KR102384311B1 - Device for managing user information based on image and method thereof - Google Patents

Abstract

An object of the present disclosure is to effectively manage user information by easily modifying, adding, and/or checking user information based on an image representing the user information. A device according to an embodiment includes a display for displaying an avatar-based image, a controller for correcting the avatar-based image displayed on the display according to a user input, and updating user information based on the modified avatar-based image.

Description

DEVICE FOR MANAGING USER INFORMATION BASED ON IMAGE AND METHOD THEREOF

The present disclosure relates to a device and method for managing user information, and more particularly, to a device and method for managing user information based on an image representing the user information.

With the development of communication technology and smart devices, it is becoming possible to collect various user information. The user information refers to information related to a user, such as user profile information and/or user life log information. The user's life log information may include, but is not limited to, user schedule information or user health information.

The user information may be collected using a user input and/or a sensing function of the device, a detection function of the device, or/and a data analysis function of the device, but the method of collecting user information is not limited thereto.

There is a need for a technology capable of effectively managing user information collected in such a variety of ways.

The above-mentioned background art is information possessed by the inventor for the derivation of the present disclosure or acquired during the derivation of the present disclosure, and cannot necessarily be said to be a known technique disclosed to the general public prior to the filing of the present disclosure.

Embodiments of the present disclosure are for effectively managing user information by easily modifying, adding, and/or checking user information based on an image representing user information.

In addition, embodiments of the present disclosure are for effectively managing user information by easily checking and using a service that can be provided based on an image representing user information.

As a technical means for achieving the above technical problems, a first aspect of the present disclosure is a device for managing user information, comprising: a display for displaying an avatar-based image; controlling the display so that the avatar-based image according to user schedule information is displayed on the display, modifying the avatar-based image according to a user input indicating a request for modification of the avatar-based image, and adding the modified avatar-based image Accordingly, it is possible to provide a device including a control unit for updating the user schedule information.

In addition, the display further includes a user input unit that displays a plurality of avatar-based images by being controlled by the control unit and receives a user input requesting user information for a specific period, wherein the control unit responds to the user input. In response, the device may be controlled to display a plurality of avatar-based images according to user information of the specific period, and the plurality of avatar-based images may be determined according to a criterion for dividing the specific period.

In addition, the display further includes a user input unit that displays a plurality of avatar-based images by being controlled by the control unit and receives a user input requesting user information for a specific period, wherein the control unit responds to the user input. In response, the device may be controlled to display a plurality of avatar-based images according to the user information of the specific period, and the plurality of avatar-based images may be determined according to a theme item included in the user information of the specific period.

In addition, the device further includes a user input unit for receiving a user input for requesting display of editable information, and the display is controlled by the controller to respond to the user input, and is modifiable from among the avatar-based images. You can mark parts as distinct from non-modifiable parts.

Also, the controller may check the provision period when the provision period for the avatar-based image is set, and destroy the avatar-based image when the provision period expires.

In addition, the display displays an avatar-based image according to the user's health information, and the device further includes a user input unit for receiving a user input indicating a medical information request, and the control unit, in response to the user input Medical information related to the user's health information is provided through the display, and when a user input regarding medical information selected based on the provided medical information is received through the user input unit, the user's schedule according to the selected medical information Information can be updated.

In addition, the device further includes a user input unit for receiving a user input indicating addition of information on the avatar-based image, wherein the display is controlled by the controller to display an input window for adding the information, The controller may add an image corresponding to the information received through the input window to the avatar-based image, and update the user's schedule information according to the avatar-based image including the added image.

In addition, the display is controlled by the controller to display an avatar-based image according to the user's future hope information, and the device further includes a user input unit for receiving a user input requesting information related to the future hope information The control unit may provide information related to future hope information through the display according to the user input, and update the schedule information of the user according to information selected from information related to the future hope information.

In addition, the display is controlled by the controller to display a plurality of avatar-based images according to usage history information of the device, and the device receives a user input requesting usage history information for each usage period of the device and a user input unit, wherein in response to the user input, the control unit may control the display to display the plurality of avatar-based images on the display.

The display is controlled by the controller to display an avatar-based image according to user information registered in social media, and the device further includes a user input unit for receiving a user input requesting correction, the controller may modify the avatar-based image in response to the user input, and update user information registered in the social media according to the modified avatar-based image.

As a technical means for achieving the above-described technical problems, a second aspect of the present disclosure is a user information management method executed by a computing device, wherein an avatar-based image according to user schedule information is displayed on the display of the computing device. step; modifying the avatar-based image according to a user input indicating a modification request for the avatar-based image; and updating the user schedule information according to the modified avatar-based image.

In addition, the displaying of the avatar-based image may include displaying a plurality of avatar-based images according to user information of a specific period, and the plurality of avatar-based images may be determined according to a criterion for dividing the specific period. .

In addition, the displaying of the avatar-based image includes displaying a plurality of avatar-based images according to user information of a specific period, wherein the plurality of avatar-based images are themes included in the user information of the specific period. It may be determined depending on the item.

In addition, the method may further include displaying a modifiable part of the avatar-based image in response to a user input requesting display of modifiable information to be distinguished from a non-modifiable part.

The method may further include: if a provision period for the avatar-based image is set, checking the provision period; When the provision period expires, the method may further include destroying the avatar-based image.

In addition, the method includes: displaying an avatar-based image according to the user's health information on the display; providing medical information related to the user's health information according to a user input indicating a medical information request; and updating the user's schedule information according to medical information selected based on the provided medical information.

The method may further include: displaying an input window for adding information on the display when a user input indicating addition of information on the avatar-based image is received; adding an image corresponding to the information received through the input window to the avatar-based image; and updating the user's schedule information according to the avatar-based image including the added image.

In addition, the method includes: displaying an avatar-based image according to the user's future hope information; and providing information related to future hope information according to a user input requesting information related to the future hope information. and updating the user's schedule information according to information selected from among the information related to the future hope information.

The method may further include displaying a plurality of avatar-based images according to the usage history information in response to a user input requesting usage history information for each usage period of the computing device.

In addition, the method may include: displaying an avatar-based image according to user information registered in social media; modifying the avatar-based image according to a user input requesting modification; and updating user information registered in the social media according to the modified avatar-based image.

A third aspect of the present disclosure may provide a computer-readable recording medium recording a program for executing the method of the second aspect on a computer.

1 is an example in which a device displays an avatar-based image according to travel schedule information of a user, according to some embodiments.
2 is a flowchart of a user information management method in which a device uses an avatar-based image according to user schedule information to modify user information, according to an exemplary embodiment.
3 is an example of user modeling information for user schedule information generated by a device according to some embodiments.
4 is a diagram for describing a relationship between a plurality of object information stored in a device and a plurality of object images, according to an exemplary embodiment.
5A to 5E illustrate an example in which a device modifies an avatar-based image, according to some embodiments.
6A to 6E are other examples in which a device modifies an avatar-based image, according to some embodiments.
7 is a flowchart of a user information management method in which a device adds user information by using an avatar-based image according to user schedule information, according to some embodiments.
8A to 8C are other examples in which a device modifies an avatar-based image, according to some embodiments.
9 is a flowchart of a method for managing user information based on a plurality of avatar-based images according to user schedule information for a specific period by a device, according to some embodiments.
10 is an example of displaying a plurality of avatar-based images based on weekday user schedule information and weekend user schedule information by a device according to some embodiments.
11 is a flowchart of a method for managing user information in which a device uses a plurality of avatar-based images based on user schedule information for each theme item, according to some embodiments.
12A to 12C are examples of modifying a plurality of avatar-based images for each theme item.
13 is a flowchart of a user information management method in which a device manages the lifespan of an avatar-based image, according to some embodiments.
14 is a flowchart of a method for managing user information in which a device uses an avatar-based image according to the user's health information, according to some embodiments.
15A to 15E are diagrams illustrating an example in which a device updates user information by using an avatar-based image according to the user's health information, according to some embodiments.
16 is a flowchart of a method for managing, by a device, user information based on an avatar-based image according to a user's future desired information, according to some embodiments.
17A to 17D are diagrams illustrating a process in which a device updates user schedule information based on an avatar-based image according to a user's future desired information, according to some embodiments.
18 is a flowchart of a method for managing user information based on a plurality of avatar-based images according to device usage history information, according to some embodiments.
19A and 19B are examples in which a device provides a plurality of avatar-based images in response to a request for device usage history information, according to some embodiments.
20 is a flowchart of a method for managing user information in which a device uses an avatar-based image according to user information registered in social media, according to some embodiments.
21 is a block diagram of a user information management system including a device according to some embodiments.
22 is a flowchart illustrating a user information management method performed by a device according to some embodiments.
23A to 23F are diagrams for explaining avatar-based images displayed by a device based on user information, according to some embodiments.
24A is an example of user modeling information for user information, according to some embodiments.
24B is an example of a display size of an object included in an avatar-based image that may be displayed on a device according to some embodiments.
24C is a diagram for describing a display size of an object included in an avatar-based image according to some embodiments.
25 illustrates an example of mapping information between an object image and object identification information, according to some embodiments.
26 is a diagram for describing a process of generating an avatar-based image by using user modeling information and mapping information, according to some embodiments.
27A to 27F are examples in which a device displays a modifiable object according to some embodiments.
28 is a flowchart illustrating a process in which a device modifies an avatar-based image according to a user input, according to an exemplary embodiment.
29A to 29E illustrate examples in which an object is modified based on a candidate object provided by a device, according to some embodiments.
30 is a flowchart illustrating a process in which a device provides service information that can be provided for each object included in an avatar-based image, according to some embodiments.
31A to 31D are examples of screens for explaining the user information management method illustrated in FIG. 30 .
32 is an example of user modeling information updated by a device according to some embodiments.
33 is a flowchart of a method in which a device manages user information by modifying an avatar-based image, according to some embodiments.
34 is a flowchart of a method in which a device manages user information by modifying an avatar-based image, according to some embodiments.
35 is an operation flowchart of a method for managing user information in which a device uses an avatar-based image to correct and/or add information, according to some embodiments.
36 is a flowchart of a method of managing user information so that a device can provide a service based on an object selected from an avatar-based image, according to some embodiments.
37 is a flowchart of a method for managing user information in which a device provides a question and answer (QA) service based on an avatar-based image displayed, according to some embodiments.
38 is an example in which a device provides answer information according to received question information, according to some embodiments.
39 is a flowchart of a method for managing user information based on a device and an external device, according to some embodiments.
40 is a flowchart of a user information management method based on a device and an external device, according to some embodiments.
41 is a flowchart of a user information management method based on a device and a wearable device, according to some embodiments.
42 is a flowchart of a user information management method based on a device and a wearable device, according to some embodiments.
43 and 44 are functional block diagrams of a device according to some embodiments.
45 is a detailed functional block diagram of a control unit according to some embodiments.

Hereinafter, specific embodiments will be illustrated in the drawings and described in detail. However, the present invention is not limited to the following specific embodiments, and it should be understood to include all transformations, equivalents, and substitutes included in the spirit and scope of the invention. In the following description of specific embodiments, if it is determined that a detailed description of a related known technology may obscure the gist of the present invention, the detailed description thereof will be omitted.

Terms such as first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another.

Terms used throughout the specification are only used to describe specific embodiments, and are not intended to limit the present invention. The terms used throughout the specification have selected common terms that are currently widely used as possible while considering their functions, but these may vary depending on the intention, precedent, or emergence of new technology of a person skilled in the art. In addition, in a specific case, there is a term arbitrarily selected by the applicant, and in this case, the meaning of the selected term will be described in detail. Therefore, the terms used throughout the specification should be defined based on the meaning of the term and the content throughout the specification, rather than the simple name of the term.

The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present specification, terms such as “comprise” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, and includes one or more other features or It should be understood that the existence or addition of numbers, steps, operations, components, parts, or combinations thereof does not preclude the possibility of addition.

In the present disclosure, user information may include user profile information.

The user profile information may include information about the user's gender. The user profile information may include information about the user's name. The user profile information may include information about the user's age. The user profile information may include information about the user's nationality.

Also, the user profile information may include information about a region in which the user currently resides. The user profile information may include information about a company name where the user works. The user profile information may include information about a location of a company where the user works. The user profile information may include information about the user's hobbies.

In addition, the user profile information may include information about a place where the user was born. The user profile information may include information about the user's job. The user profile information may include information about the user's family (for example, information about the existence of parents, the existence of children, etc.). The user profile information may include information about the user's pet (eg, having a pet, and the type of pet). The user profile information may include information about the user's interests (eg, classical guitar, soccer, economy, travel, games, books, etc.). In the present disclosure, the user profile information is not limited to the above.

In addition, the user profile information may be referred to as user profile information (hereinafter abbreviated as social media profile information) registered in social media (eg, Facebook, Twitter, blog, YouTube, LinkedIn, etc.).

Social media profile information may include keywords, photos, places, times, or/and events (eg, photo exhibitions, art exhibitions, baseball games, soccer games) detected by analyzing the context, vocabulary, images, or/and tags posted by the user. games, concerts, or plays, etc.).

In the present disclosure, user information is user information (eg, member information, game level information, item information, usage history information, or / and a quest in a game) registered on a website (eg, Naver, Daum, Nate, etc.) (quest) information, etc.).

In the present disclosure, the user information may include user information registered to applications installed in the device, respectively. The application installed on the device may be, for example, a schedule management application, a game application, a service center application, a membership application, a customer center application, a market application, a life log management application, a map application, or a future wish information management application (eg, bucket list (bucket list) management application or wish list (wish list) management application), but the application installed in the device in the present disclosure is not limited thereto.

In the present disclosure, user information may include information acquired by a device based on data generated by an interaction between a device and a user or input/output.

The information obtained by the device may include search history information. The information obtained by the device may include mail history information. The information obtained by the device may include message history (eg, a short message, a multi-message, or a voice message, etc.) information. The information obtained by the device may include browsing history information.

Also, the information obtained by the device may include social media usage history information (or social network service history information). The information obtained by the device may include call history information. The information obtained by the device may include album history information. The information obtained by the device may include network history information. The information obtained by the device may include content history information. The information obtained by the device is not limited to the above.

The search history information may include at least one piece of information acquired while executing a search application installed on the device (eg, a list of search terms (or query words), the number of searches for each search word, search date, or number of search results, etc.) there is.

The mail history information includes at least one piece of information acquired while executing the mail application installed on the device (eg, the send/receive mail list, the number of sent/received mails, the number of deleted mails, the sender list, the number of mail sent by sender, the recipient list, the recipient The number of emails received by each, or the date of sending and receiving emails, etc.) may be included.

The message history information includes at least one piece of information acquired while executing a message application installed on the device (eg, a list of transmitted/received messages, the number of transmitted/received messages, the number of deleted messages, a list of senders, the number of messages sent by each sender, a list of recipients) , the number of messages received by each recipient, or a message transmission/reception date, etc.).

The browsing history information may include at least one piece of information (eg, a Uniform Resource Locator (URL) list, the number of accesses per URL, or URL link information, etc.) acquired while executing a browsing application installed in the device.

The social media usage history information may include at least one piece of information (eg, the number of postings or the number of comments) obtained while executing a social media service application installed in the device. For example, social media usage history information may include keywords, photos, places, times or/and events (e.g., photo exhibitions) detected by analysis of at least one of context, vocabulary, images, or/and tags posted by the user. , art exhibition, baseball game, soccer game, concert, or play, etc.) based on historical information (eg, keyword list, number of occurrences by keyword, place list, number of visits by place, hourly social network service basis, etc.) content of the user's activity, or the number of visits per event).

The call history information includes at least one piece of information (eg, number of video calls, video call time, video call target, number of calls per video call target, call time per video call target, and number of voice calls) acquired while executing a call application installed in the device. , a voice call time, a voice call target, the number of calls for each voice call target, or a call time for each voice call target).

Album history information includes information about still images and/or videos acquired while executing a camera application installed in the device (eg, a shooting location, shooting target information (eg, people, objects, or animals, etc.), or shooting date, etc.).

The network history information may include at least one network connection information of the device. Network connection information may include, for example, whether or not to use Wi-Fi, the number of times of Wi-Fi access, the name of the connected Wi-Fi device, information about the Wi-Fi network, whether to use Bluetooth, the number of times of Bluetooth access, the name of the connected Bluetooth device, data usage, NFC use It may include whether setting, the number of times of NFC use, NFC use history, beacon connection history, or Zigbee connection history, but the network connection information in the present disclosure is not limited thereto.

The content history information may include content generated using a device and information about the content. The content history information may include content shared with other devices and information about the content. The content history information may include content transmitted to another device and information about the content.

In addition, the content history information may include content received from another device and information about the content. In the present disclosure, the content history information is not limited as described above. Information on content included in the content history information may include, for example, content creation date, content data amount, content search keyword, name of a device that generated content, name of a device that transmitted content, or content. It may include the name of the received device.

In the present disclosure, user information may include environment information of a device set in the device. The environment information of the device may include, for example, the name of the device, account information, a backup account, or the name of an external device (or peripheral device), but the environment information of the device in the present disclosure is not limited thereto.

In the present disclosure, user information may be stored in a device. In the present disclosure, user information may be received from at least one external device connected to the device. The external device may include a wearable device (eg, a smart watch, smart glasses, etc.). The external device may include an appcessory. The external device may include a medical device. The external device may include a vehicle. The external device may include a server (eg, a question and answer (QA) server, or a game server). The external device may include an Internet of Thing (IoT) network-based device (hereinafter abbreviated as IoT device). The external device may include an IoT network-based sensor (hereinafter abbreviated as IoT sensor). The external device may include a device (eg, a game console, etc.).

The user information provided from the external device may include a value sensed (or measured value) by the external device. The user information provided from the external device may include information collected by the external device. The information collected by the external device may include information detected or analyzed by the external device. The user information provided from the external device is not limited as described above.

The value sensed by the external device may include the user's health information. The value sensed by the external device may include user activity information. The value sensed by the external device is not limited as described above.

The user's health information may include information about exercise time. The user's health information may include information about a heart rate during an exercise period. The user's health information may include information about pulse oximetry during the exercise period. The user's health information may include information on calorie consumption during an exercise period. The user's health information may include information about the user's moving distance during the exercise period.

In addition, the user's health information may include information about the temperature of the user's skin during the exercise period. The user's health information may include information about the number of steps the user takes in a day. The user's health information may include information about the user's galvanic skin response during the exercise period. The user's health information may include information about the user's duration of sleep. The user's health information may include information about the user's sleep patterns. The user's health information may include information about the user's blood sugar level.

In addition, the user's health information may include information sensed or measured by a medical device (eg, information about body composition, blood vessel age, blood pressure, or meridian pulse, etc.).

The user's activity information may include information indicating walking. The user's activity information may include information indicating running. The user's activity information may include information indicating stay. The user's activity information may include information indicating sleeping.

In addition, the user's activity information may include information indicating a means of transportation (eg, a car, a bus, a train, an airplane, or a ship). The activity information of the user may include information indicating the time the user stayed in one place. The user's activity information may include information on a place frequently visited by the user.

In addition, the user's activity information may include information about a person the user frequently meets. The user's activity information may include information about a meeting in which the user participated. The user's activity information may include information about a meeting in which the user participated. In the present disclosure, the user's activity information is not limited to the above. The user's activity information may include a part (eg, exercise time) of the above-described user's health information.

The information collected by the external device may include information collected by the server, but is not limited thereto. The information collected by the server may include information that maps a user's question and a search result. The information collected by the server may include information (eg, game level, exercise level, etc.) obtained by the above-described device. The information collected by the server is not limited to the above.

In the present disclosure, user information may include physical information of a user. In addition, the user information may include mental information of the user. The user information may include social information of the user. The user information may include schedule information of the user. The user information may include future wish information (bucket list, or wish list information) of the user. The user information may include usage history information for each usage period of the device.

The user's physical information may include face-related information (eg, face shape, eye color, nose shape, mouth shape, eye shape, and/or eyebrow shape, etc.). The user's physical information may include information about the user's height. The user's physical information may include information about the user's hair style (eg, short hair, long hair, curly hair, straight hair, or wavy hair). The user's physical information may include user's race information. The user's physical information may include the user's weight information. In the present disclosure, the user's physical information is not limited as described above. The user's physical information may include, for example, information about the user's gender and/or information about the user's age.

The mental information of the user may include information about the user's preference (eg, food preference information). The user's mental information may include information about the user's disposition (eg, bookmark information, frequently used application information, application execution time information, book purchase history information, video rental history information, magazine subscription information, music download information, or / and user's tendency information based on information such as product purchase history information).

In addition, the mental information of the user may include information on the user's preference (eg, preference information based on information such as web site search history information, and/or hobby information). The mental information of the user may include information about the user's emotions (eg, information indicating joy, sadness, fear, anger, surprise, greeting, or disgust). The user's formal information is not limited to the above. For example, the user's formal information may include information on the above-mentioned interests.

The user's social information may include information about the user's educational background. The user's social information may include information about the user's workplace. The user's social information may include information about the user's work content. The user's social information may include information about the user's meeting. In the present disclosure, the user's social information is not limited to the above.

The user's schedule information may include information input by the user using a schedule management application, but is not limited thereto. The user's schedule information may include information about the user's past, present, and future. The user's schedule information may include user information based on at least one of time, day of the week, month, year, and season.

The user's schedule information may be classified based on a theme item. The theme items are, for example, during travel, company, study, bicycle, meeting, date, movie viewing, exercise, shopping, hospital, beauty care, meal, game, meeting, mountaineering, nightlife, birthday, performance, or business trip. It may include at least one. In the present disclosure, the theme item is not limited thereto.

The user information may include all or part of the user information managed by the user's PIMS (Personal Information Management Software). The user information may include at least one of all or part of information included in the user's life log information.

User information is in the form of User Modeling information or User Personal Information in order to more effectively perform the purpose of the device, the external device of the device, or the service provider (eg, to provide a user-customized service). can be expressed

User information may be expressed as an avatar-based image expressed as a 2D model or a 3D model. The user information may be expressed as a virtual image, a virtual self graphic icon, or the like.

Hereinafter, the present disclosure will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating an example in which the device 100 displays an avatar-based image according to travel schedule information of a user and displays information indicating an image that can be modified in the avatar-based image, according to an exemplary embodiment.

Referring to FIG. 1 , the device 100 may display an avatar-based image according to the user's travel schedule information. The avatar-based image shown in FIG. 1 includes travel period information 101, an avatar 102 representing a user, a photo 103 of a travel place, a picture 104 of a bag, and a photo 103 of a travel place. contains information 105 indicating that it is a modifiable image, information 106 indicating that the picture 104 of the bag is a modifiable image, and information 107 indicating that the travel duration information 101 is a modifiable image do.

The avatar-based image according to the user's travel schedule information is not limited to the bar shown in FIG. 1 . For example, the avatar-based image according to the user's travel schedule information may include pictures, photos, or texts related to a plurality of travel preparations. The avatar-based image according to the user's travel schedule information may not include the travel period information 101 . The avatar-based image according to the user's travel schedule information may further include information indicating that the avatar 102 is a modifiable image.

The device 100 may display the information 105 , 106 , and 107 representing the modifiable image shown in FIG. 1 in a different form. For example, the device 100 may change the brightness of the modifiable image at a constant cycle or a variable cycle. Also, the device 100 may display the editable image as a highlight.

The device 100 may display information for indicating a modifiable image differently depending on the image. For example, the device 100 changes the brightness of the photo 103 of the travel place at a constant or variable cycle, highlights the picture 104 of the bag, and periodically blinks the travel period information 101 . can be

2 is a flowchart of a user information management method in which the device 100 displays an avatar-based image according to user schedule information and updates the user schedule information by modifying the displayed avatar-based image, according to some embodiments.

In step S201, the device 100 displays an avatar-based image according to the user schedule information. The user schedule information may be information stored in a schedule management application installed in the device 100 , but is not limited thereto. For example, the user schedule information may be received from an external device connected to the device 100 .

The device 100 may generate an avatar-based image according to ontology-based user modeling information. Ontology can describe a word (or concept) and a relationship between words (or a relationship between concepts) in a formal language.

The formal vocabulary may include, for example, hasHobby, hasSize, isA, cause, and the like, but the formal vocabulary in the present disclosure is not limited thereto. hasHobby is a formal vocabulary for indicating that a relationship between words is a hobby. hasSize is a formal vocabulary indicating that the relationship between words is a size. isA is a formal vocabulary that indicates the containment relationship between words. Cause is a formal vocabulary that indicates a causal relationship between words. For example, “I need a break due to overwork” can be expressed as (overwork, rest) using Cause.

The above-described user modeling information may be generated by an external device that may be connected to the device 100 .

3 is an example of user modeling information for user schedule information generated by the device 100 according to some embodiments.

Referring to FIG. 3 , user modeling information is an example generated in a resource description framework (RDF) language. The device 100 may generate user modeling information using Ontogogy Web Language (OWL) or Semantic Web Rule Language (SWRL).

Referring to FIG. 3 , user modeling information includes a subject field, a predicate field, and an object field. The subject field is a field related to an object of user information. Referring to FIG. 3 , the target of user information is user 1 (user_1). It indicates that user 1 (user_1) is a person by using a class and a resource. A resource is an instance of a class.

The pledikit field shown in FIG. 3 is a formal vocabulary field indicating a relationship between a subject field and a word included in the object field. Referring to FIG. 3 , the formal vocabulary included in the Pledicit field includes a formal vocabulary about gender (gender), a formal vocabulary about a schedule (travel), and a formal vocabulary about preparation (preparation). In the present disclosure, the formal vocabulary included in the pledikit field is not limited as described above.

The device 100 may express a woman in the object field as the gender of the user 1 according to the gender-related formal vocabulary. The device 100 may represent the travel places 1 to 5 in the object field as the travel places of the user 1 according to the formal vocabulary related to the schedule. The device 100 may express the preparations 1 to 4 in the object field as the travel preparations of the user 1 according to the formal vocabulary about the travel bag.

The device 100 may generate an avatar-based image by using object information (a woman shown in FIG. 2 , travel places 1 to 5, and preparations 1 to 4) of an object field included in the user modeling information. Information of the object field may be used as identification information for generating an avatar-based image.

In order to generate an avatar-based image, the device 100 may store a plurality of object images. The plurality of object images may be stored in an external device connectable to the device 100 .

The plurality of object images may include images corresponding to the number of object information included in the object field shown in FIG. 3 . The plurality of object images may include an avatar, a photo, a moving picture, a picture, and/or a character, but the plurality of object images is not limited thereto in the present disclosure.

4 is a diagram for describing a relationship between a plurality of object information and a plurality of object images stored in the device 100 according to some exemplary embodiments.

Referring to FIG. 4 , each object information included in the object field has a corresponding object image. The device 100 may read the object image from the memory of the device 100 using the object information as identification information. When the plurality of object information shown in FIG. 4 and the object image corresponding thereto are stored in an external device connected to the device 100 , the device 100 may receive the object image from the external device using the object information.

The device 100 may combine the generated object images to generate and display one avatar-based image as shown in FIG. 1 .

Meanwhile, in step S202 , the device 100 may receive a user input indicating a request to modify the avatar-based image being displayed as shown in FIG. 1 . The user input may include a touch-based user input, a user's voice-based user input, or a user's gesture-based user input, but the user input is not limited thereto.

5A to 5E illustrate an example in which the device 100 modifies an avatar-based image according to some embodiments. 5A to 5E are examples of modifying travel arrangements.

Referring to FIG. 5A , upon receipt of a user input indicating a selection for information 106 indicating that the picture 104 for the bag is a modifiable image, the device 100 may put into the bag as shown in FIG. 5B , as shown in FIG. 5B . Images for preparations 1 to 4 (501 to 504) are displayed together with an avatar-based image. Additionally, device 100 may display an edit button 505 , an add button 506 , or/and a done button 507 .

When a user input indicating selection of the edit button 505 in FIG. 5B is received, the device 100 changes the display state of the preparations 1 to 4 (501 to 504) as shown in FIG. 5c. Accordingly, the user can know that the preparations 1 to 4 (501 to 504) are in a modifiable state.

When a user input indicating selection of the first preparation 501 in FIG. 5C is received, the device 100 displays detailed information on the first preparation 501 as shown in FIG. 5D .

Referring to FIG. 5D , detailed information on the first preparation 501 may be provided by displaying a pop-up window 508 in a position adjacent to the first preparation 501, but the present disclosure is not limited thereto. Since the detailed information provided through the pop-up window 508 is detailed information on the preparation 1 501, the user can delete or modify the preparation 1 501 while viewing the detailed information provided through the pop-up window 508 . The device 100 may display the detailed information in a display area other than the form of the pop-up window 508 . For example, detailed information may be displayed using an area in which the travel period information 101 is displayed.

If the pop-up window 508 displaying detailed information is clicked on, the device may display more detailed information. The device 100 may provide more detailed information using another page. When more detailed information is provided using another page, the device 100 may switch the screen illustrated in FIG. 5E to a page providing more detailed information.

6A to 6E are other examples in which the device 100 modifies an avatar-based image according to some embodiments. 6A to 6E are examples of modifying a travel route.

Referring to FIG. 6A , when a user input indicating selection of the above-described information 105 is received, the device 100 displays images 601 to 605 representing travel places 1 to 5 and travel as shown in FIG. 6B . Information (arrow) indicating the order is displayed together with the avatar-based image. The images 601 to 605 may include text information. The text information may include a travel destination name. The images 601 to 605 may include thumbnails of travel symbols, marks, icons, or representative images.

Referring to FIG. 6B , the device 100 displays an image 601 representing the travel location 1 so that the user can easily recognize that the photo 103 about the travel location included in the avatar-based image corresponds to the travel location 1. ) to be distinguished from the images 602 to 605 representing the travel places 2 to 5. For example, in FIG. 6B , the device 100 may highlight the image 601 representing the travel location 1 .

The device 100 may display the edit button 606 , the add button 607 , or/and the done button 608 along with the avatar-based image. The completion button 608 is a button capable of receiving a user input indicating completion of a correction and/or addition operation on the avatar-based image being displayed on the device 100 .

When a user input indicating selection of the edit button 606 in FIG. 6B is received, the device 100 displays the display state of the images 601 to 605 representing the travel places 1 to 5 as shown in FIG. 6C . change Accordingly, the user can easily recognize that the images 601 to 605 representing the travel places 1 to 5 are in a modifiable state.

When a user input indicating movement of image 601 representing travel location 1 to image 602 representing travel location 2 in FIG. 6D is received, device 100 performs a travel based on travel location 1 as shown in FIG. 6D . While changing the photo 103 of the place to the photo 609 of the travel place corresponding to the travel place 2, the arrangement of images 601 to 605 representing the travel places 1 to 5 is changed as shown in FIG. 6E do. Accordingly, as the user first visits the travel location 2, it can be confirmed that the travel route has been changed.

After confirming the changed travel route shown in FIG. 6E , by selecting the Done button 608 , in step S203 , the device 100 displays travel location 2 -> travel location 1 -> travel as shown in FIG. 6e . The travel schedule information is updated according to the avatar-based image so that the travel schedule information is changed in the order of place 3 -> travel place 4 -> travel place 5.

7 is a user information management method of updating user schedule information as the device 100 displays an avatar-based image according to user schedule information and adds information based on the displayed avatar-based image, according to some embodiments. It is a flow chart.

Referring to FIG. 7 , in step S701 , the device 100 displays an avatar-based image according to user schedule information. The device 100 may display an avita-based image according to the user schedule information as in step S201 described above.

In step S702, when a user input indicating addition is received, the device 100 displays an input window. The device 100 may display the input window as an independent window.

In step S703, when information is received through the input window, the device 100 adds an image corresponding to the received information to the avatar-based image. For example, if the received information is a passport, the device 100 may add an image corresponding to the passport to the avatar-based image. To this end, the device 100 may store an image corresponding to the received information in advance. An image corresponding to the received information may be stored in an external device connectable to the device 100 . A search keyword or search identification information may be assigned to each image corresponding to the received information. The device 100 may search for an image corresponding to the received information using the above-described search keyword or search identification information.

The device 100 may search for and provide at least one candidate image (or a plurality of candidate images) that may correspond to the received information in units of one letter (or one alphabet). When the selection signal for the candidate image is received, the device 100 may add the selected candidate image to the avatar-based image.

In step S704 , the device 100 updates user schedule information according to the avatar-based image including the added image.

8A to 8C are other examples in which the device 100 modifies an avatar-based image according to some embodiments. 8A to 8C are examples of adding preparations.

When a user input indicating addition of information based on the add button 506 in FIG. 8A is received, the device 100 displays an input window 801 as shown in FIG. 8B . When a user input for inputting information on preparation 5 through the input window 801 and selecting a button 802 representing the input is received, the device 100 displays an image corresponding to preparation 5 as shown in FIG. 8c . Add 803 to the avatar-based image.

The device 100 may display the added image 803 to be distinguished from other images 501 to 504 . Accordingly, the user can easily recognize that the image 803 corresponding to the preparation 5 has been added.

When a user input for selecting the completion button 507 in FIG. 8C is received, the device 100 updates the user schedule information according to the avatar-based image to which the image 803 corresponding to the preparation 5 is added. Accordingly, materials 1 to 4 (501 to 504) for travel A included in the user schedule information are changed to materials 1 to 5 (501 to 504, 803).

9 is a flowchart of a user information management method in which the device 100 displays a plurality of avatar-based images according to user schedule information for a specific period and updates user schedule information according to the modification of the avatar-based image, according to some embodiments.

In step S901, the device 100 receives a user input indicating a user schedule information request for a specific period. The specific period may be set based on date information. The specific period may be set based on the day of the week. A specific period can be set on a weekly basis. A specific period can be set as the time period. The specific period in the present disclosure is not limited as described above.

In step S902, a plurality of avatar-based images according to weekday and weekend user schedule information is displayed. Weekdays and weekends are weekdays and weekends included in the specified period described above. For example, if the specific period is from February 2 to February 15, 2015, the device 100 may display an avatar-based image according to the weekday user schedule information from February 2 to February 6, February Avatar-based image according to weekend user schedule information from 7th to February 8th, avatar-based image according to weekday user schedule information from February 9th to 13th, weekend users from February 14th to February 15th Each of the avatar-based images according to the schedule information can be displayed.

When a plurality of avatar-based images are being displayed and a user input indicating a correction is received in step S903 , the device 100 modifies the avatar-based images.

In operation S904 , the device 100 may update user schedule information according to the avatar-based image including the modified image.

10 is an example in which the avatar-based image 1001 based on weekday user schedule information and the avatar-based image 1002 based on weekend user schedule information are displayed together by the device 100 according to some embodiments.

11 is a diagram in which the device 100 according to some embodiments displays a plurality of avatar-based images according to user schedule information for each theme item in response to a user schedule information request for a specific period, and modifies the displayed avatar-based image to schedule a user schedule. It is a flowchart of a user information management method for updating information.

Referring to FIG. 11 , upon receiving a user input indicating a user schedule information request for a specific period in step S1101, the device 100 displays a plurality of avatar-based images according to the user schedule information for each theme item in step S1102. The theme item refers to a theme item included in user schedule information for a specific period.

In step S1103, the device 100 modifies the avatar-based image according to the user input indicating the correction. In step S1104, the device 100 updates the user schedule information according to the avatar-based image including the modified image.

12A to 12C are examples of displaying a plurality of avatar-based images for each theme item included in user schedule information for a specific period mentioned in FIG. 11 and modifying the displayed avatar-based image.

Referring to FIG. 12A , the device 100 displays a plurality of avatar-based images 1201 , 1202 , 1203 , and 1204 for each theme item from February 1, 2015 to February 8, 2015 together.

12A, when a user input for selecting one avatar-based image 1201 from among a plurality of avatar-based images 1201, 1202, 1203, and 1204 is received as shown in FIG. 12B, the device 100 may enlarge and display the selected avatar-based image 1201 on the entire screen as shown in FIG. 12C .

Referring to FIG. 12C , the avatar-based image 1201 may include an edit button 1202 , an add button 1203 , and/or a complete button 1204 . The scope of modification in FIG. 12C may include changing biking to another theme item (or other user activity) or canceling biking, but the scope of modification in the present disclosure is not limited as described above. When changing from cycling to sleep, the device 100 may change the avatar-based image 1201 to the avatar-based image 1204 shown in FIG. 12A .

Additional scope in FIG. 12C may include adding a Tate theme item with B to Riding a Bicycle. In this case of adding, the device 100 may add an image related to B to the avatar-based image 1201 . Further scope in the present disclosure is not limited as described above.

13 is a flowchart of a user information management method in which the device 100 manages the lifespan of an avatar-based image, according to some embodiments.

In step S1301, if the provision period for the avatar-based image is set, the device 100 checks the provision period for the avatar-based image. The provision period may be counted based on the date. The provision period may be counted on a time basis. The provision period may be counted as the number of times of provision period. In the present disclosure, the count criterion for the provision period is not limited as described above.

In step S1302, when it is determined that the provision period has expired, the device 100 destroys the corresponding avatar-based image. Deleting the avatar-based image refers to deleting the avatar-based image stored in the device 100 . The avatar-based image may be stored in an external device connected to the device 100 . When the avatar-based image is stored in the external device, the device 100 may transmit a delete command for the avatar-based image to the external device.

14 is a flowchart of a user information management method in which the device 100 displays an avatar-based image according to the user's health information and updates user schedule information according to the selected medical information, according to some embodiments.

In step S1401. The device 100 displays an avatar-based image according to user health information. The user's health information may be received from an external device, but the present disclosure is not limited thereto. The user's health information may be obtained by executing a health application set in the device 100 .

In step S1402 , the device 100 provides medical information related to the user's health information according to a user input indicating a medical information request. For example, when the user's health information indicates that the user has a cold, the device 100 may provide cold-related medical information.

In step S1403, the device 100 updates the user's schedule information according to the medical information selected based on the provided medical information.

15A to 15E show that the device 100 displays an avatar-based image according to the user's health information, provides medical information according to the user's health information, and schedules information of the user according to the selected medical information, according to some embodiments. It is a diagram showing an example of updating .

In FIG. 15A , the device 100 displays an avatar-based image with fever and runny nose according to the user's health information received from the smart arm 1500 .

When a user input indicating a medical information request is received, the device 100 includes a hospital item 1511 , a pharmacy item 1512 , a medication item 1513 , and a self-treatment item 1514 as shown in FIG. 15B . menu items are provided. For example, as shown in FIG. 15A , when a user input of touching an avita-based image according to the displayed user's health information is received for a predetermined time, the device 100 provides menu items as shown in FIG. 15B . can do. The menu items provided in the present disclosure are not limited as described above.

When a user input for selecting the hospital item 1511 in FIG. 15B is received, the device 100 displays sub-items of the hospital item 1511 (Hospital B 1521 and Hospital C 1522 ) as shown in FIG. 15C . , D hospital (1523)) is provided. For example, in FIG. 15B , when a user input of touching the hospital item 1511 for a predetermined time is received, the device 100 displays sub-items of the hospital item 1511 (Hospital B 1521 ) as shown in FIG. 15C . ), C hospital 1522, and D hospital 1523) may be provided.

When a user input for selecting hospital C 1522 in FIG. 15D is received, the device 100 provides reservation information for hospital C 1522 in FIG. 15E . For example, when a user input of touching the hospital C 1522 in FIG. 15D is received for a predetermined time, the device 100 may provide reservation information for the hospital C 1522 in FIG. 15E .

After checking the reservation information provided in FIG. 15E , when a user input indicating a reservation is received, the device 100 performs a reservation procedure for the hospital C 1522 . For example, after checking the reservation information provided in FIG. 15E , when a user input of touching the reservation button 1532 for a predetermined time is received, the device 100 may perform a reservation procedure. The reservation procedure refers to registering reservation information with hospital C based on the provided reservation information and user information.

When the reservation procedure is completed, the device 100 updates user schedule information based on the reservation information. Accordingly, reservation information for hospital C may be registered in the user schedule information stored in the device 100 .

16 is a flowchart of a user information management method in which the device 100 displays an avatar-based image according to the user's future hope information and updates user schedule information according to information selected from information related to future hope, according to some embodiments. .

In step S1601, the device 100 displays an avatar-based image according to the user's future hope information. The future hope information may indicate future job information of the user, but the future hope information is not limited thereto in the present disclosure. For example, the future wish information may include the user's bucket list or the user's wish list.

In step S1602 , the device 100 provides information related to future hope information according to a user input requesting information related to the user's future hope information. For example, when a user input of touching and dragging an avita-based image according to the displayed user's future hope information for a predetermined time is received, the device 100 may provide information related to the future hope information. .

In step S1603, the device 100 updates the user's schedule information according to information selected from information related to future desired information. For example, when a user input of touching one of the information related to the provided future hope information for a predetermined time and dragging it in one direction is received, the device 100 may update the user's schedule information according to the information received by the user input. there is.

17A to 17D show that the device 100 displays an avatar-based image according to the user's future hope information, and provides related information in response to a request for providing the information related to the future hope information, according to some embodiments, and is provided by the user. It is a diagram explaining a process of updating user schedule information according to selected information.

Referring to FIG. 17A , the device 100 displays an avatar-based image representing an art therapist according to the user's future wish information.

In FIG. 17A , when a user input 1701 requesting information related to future desired information is received, the device 100 displays a qualification test schedule item 1711, an authentication center item 1712, as shown in FIG. 17B , or/and the related event item 1713 is provided as information related to future hope information. For example, when a user input 1701 of touching and dragging an image displayed in FIG. 17A for a certain period of time is received, the device 100 displays information related to future hope information as shown in FIG. 17B . provide information (qualification exam schedule item 1711 , certification center item 1712 , or/and related event item 1713 ).

When a user input for selecting a related event item 1713 in FIG. 17B is received, the device 100 provides an event 1 item 1721 , an event 2 item 1722 , and an event 3 item 1723 . For example, when a user input of touching the related event item 1713 in FIG. 17B is received for a predetermined time, the device 100 displays an event 1 item 1721 registered in a sub-layer of the related event item 1713, An event 2 item 1722 and an event 3 item 1723 may be provided.

In FIG. 17C , when a user input for selecting the event 2 item 1722 is received, the device 100 provides detailed information on the event 2 item 1722 . For example, when a user input of touching the event 2 item 1722 for a predetermined time is received in FIG. 17C , the device 100 may provide detailed information on the event 2 item 1722 .

The device 100 may provide detailed information on the event 2 item 1722 by opening a separate window 1724 adjacent to the event 2 item 1722 . The separate window 1724 may be displayed as a pop-up window having a horse bag shape, but the present disclosure is not limited thereto.

In FIG. 17D , when a user input for selecting the participation OK button 1725 included in a separate window 1723 is received, the device 100 uses information related to the event 2 item 1722 to provide the user's schedule information update For example, when a user input of touching the participation OK button 1725 included in a separate window 1723 in FIG. 17D is received for a predetermined time, the device 100 transmits information related to the event 2 item 1722 can be used to update the user's schedule information. Accordingly, information related to the event 2 item 1722 is registered in the user's schedule information.

18 is a flowchart of a method of managing user information by displaying a plurality of avatar-based images according to usage history information of the device 100 according to some embodiments.

In step S1801 of FIG. 18 , when the device 100 receives a user input requesting usage history information for the device 100 , in step S1802 , based on a plurality of avatars according to usage history information for each usage period of the device 100 . Display the image. For example, in step S1801, when a user input indicating selection of a request item for use history information of the device 100 included in a menu item provided by the device 100 is received, in step S1802, the device 100 A plurality of avatar-based images according to usage history information for each usage period may be displayed. In operation S1801, information specifying a usage period of the device 100 may be received. Information specifying the period of use may be input based on a date, time, or day of the week.

19A and 19B are examples of providing usage history information for each usage period as a plurality of avatar-based images when a user input according to a request for usage history information of the device 100 is received by the device 100 according to some embodiments.

Referring to FIG. 19A , the device 100 displays a plurality of avatar-based images according to usage history information of the device 100 on weekdays 1910 and weekends 1920 .

Referring to the weekday 1910 of FIG. 19A , on the weekday 1910, the device 100 may know that the app used as the first priority is the phone app and the app used as the second priority is the Internet app. Referring to the weekend 1920 of FIG. 19A , it can be seen that the device 100 uses the DTV app as the first priority and the band as the second priority on the weekend 1920 .

In FIG. 19A , when a user input 1911 requesting the next usage history information for the device 100 is received on a weekday 1910, the device 100 returns to the usage history information for the device 100 on a weekday 1910. Based on the avatar-based image for the weekday 1910 as shown in FIG. 19B , it is possible to change the avatar-based image. Referring to the weekday 1910 of FIG. 19B , it can be easily seen that the device 100 uses the KakaoTalk app as the 3rd priority and the camera app as the 4th priority on the weekday 1910 .

19A and 19B provide usage history information of the device 100 based on an application set in the device 100, but in the present disclosure, the usage history information of the device 100 is not limited as described above.

20 is a diagram showing the device 100 according to some embodiments displays an avatar-based image according to user information registered in social media, and when the displayed avatar-based image is modified according to a user input, the modified avatar-based image is displayed. It is a flowchart of a user information management method for updating user information registered in social media according to the following.

In step S2001, the device 100 displays an avatar-based image according to user information registered in social media. User information registered in social media may be received from a social media server connected to the device 100 . For example, user information registered in Facebook may be received from a Facebook server connected to the device 100 .

User information registered in social media may be provided from a social media server by a request of the device 100 , but may be automatically provided from the social media server when the device 100 is connected to the social media server. In the present disclosure, a method for the device 100 to receive user information registered in social media is not limited as described above.

In step S2002 , the device 100 modifies the avatar-based image according to a user input requesting image correction. The method of modifying the avatar-based image in step S2002 may be performed in the same manner as described in step S202 of FIG. 2 described above.

In step S2003, the device 100 may update user information registered in social media according to the modified avatar-based image. The device 100 may request an update of user information registered in social media while converting the modified avatar-based image into text information and transmitting the text information to the social media server. Accordingly, the social media server may update the registered user information. When the update of the user information is completed, the social media server may transmit an update completion message to the device 100 . In the present disclosure, a method of updating user information registered in social media is not limited as described above.

21 is a block diagram of a user information management system 2100 including the device 100 according to some embodiments.

Referring to FIG. 21 , a system 2100 includes a device 100 , a network 2110 , and an external device 2120 .

The device 100 may include, for example, a portable device, a wearable device, a home Internet appliance, or a mobile device, but in the present disclosure, the device (100) is not limited as described above.

Portable devices include smart phones, notebooks, smart boards, tablets, personal computers (PCs), handheld devices, handheld computers, media players, and e-book devices. , or a device such as a personal digital assistant (PDA), the portable device in the present disclosure is not limited thereto.

Wearable devices include smart glasses, smart watches, smart bands (eg, smart waist bands and smart hair bands, etc.), smart clips, various smart accessories (eg, smart rings, smart bracelets, smart feet, smart hairpins, and smart collars, etc.), various smart body guards (eg, smart knee guards, and smart elbow guards). devices such as, but not limited to, smart shoes, smart gloves, smart clothing, smart hats, smart prostheses for the disabled, or smart prostheses for the disabled.

Home Internet appliances include smart TVs, personal computers (PCs), desktop personal computers (PCs), smart refrigerators, smart washing machines, smart lighting, smart cleaners, smart health devices, smart ovens, smart dishwashers, and multimedia consoles. ), a game console, or a home device constituting an M2M (Machine to Machine) or IoT (Internet of Things), such as a smart water purifier, but the home Internet appliance in the present disclosure is not limited as described above.

The mobile device may include a device such as a vehicle constituting an machine to machine (M2M) or the Internet of Things (IoT), but the mobile device is not limited thereto in the present disclosure.

The network 2110 is, for example, an intranet, an internetwork, the Internet, a Personal Area Network (PAN) (eg, Bluetooth, short-range wireless communication such as HomeRF), a Campus Area Network (CAN), a LAN ( It may include a network such as a local area network), a wide area network (WAN), or a game network, but in the present disclosure, the network 2110 is not limited thereto.

The network 2110 may use at least one of a wired transmission medium and a wireless transmission medium. The wired transmission medium may include, for example, at least one of a coaxial cable, a twisted pair cable, and/or a fiber-optic, but in the present disclosure, the wired transmission medium is as described above. not limited The wireless transmission medium may include, for example, at least one of a radio wave (Radio Frequency Wave), a terrestrial microwave (Terrestrial Microwave), a satellite microwave, and/or an infrared (Infrared), etc., but in the present disclosure The wireless transmission medium is not limited as described above.

The connection type of the network 2110 may be any one of a shape, a net type, a tree type, a bus type, and a ring type, but the connection type of the network 2110 is not limited as described above.

The network 2110 is at least one of a network access device such as a local area network (LAN) card, a hub, a switch, a bridge, a gateway, a repeater, or a router. One may be used, but in the present disclosure, the network access device is not limited as described above.

The external device 2120 refers to a device connectable to the device 100 from the outside of the device 100 . The external device 2120 may include a user's cloud server, a user's information management server, a question and answer (QA) server, a service providing server (eg, a game server, an application server, etc.), or a social media server. However, in the present disclosure, the external device 2120 is not limited as described above.

The external device 2120 may include a server, a medical device, a computing device, an appcessory, a wearable device, an IoT sensor, or an IoT device. The external device 2110 may be the same device as the device 100 . A user of the external device 2110 may be the same as a user of the device 100 . A user of the external device 2110 may be different from a user of the device 100 .

The medical device may include a function of sensing (or measuring) the user's health information and a communication function. Medical devices include household medical devices (eg, at least one device such as a body temperature monitor, a blood pressure monitor, a cholesterol monitor, a blood sugar monitor, a uric acid monitor, and a massager), a medical device for a hospital (eg, an ultrasound device, a body fat monitor, at least one device, such as a stethoscope, and an electrocardiogram), or a medical device such as a recycled medical device (eg, at least one device such as a walking belt, transfer belt, knee guard, paraffin bath, etc.) The device is not limited to the above.

The appcessory is a device that is controlled using an application installed on the device 100 . The appcessory includes, for example, at least one device such as a smart light bulb, a smart printer, a smart wireless blood pressure monitor, a smart recorder, a smart location tracker, a smart microphone, a smart thermostat, a smart alarm device, or a smart toy. .

The device 100 may perform a user information management method according to some embodiments. The device 100 may manage user information stored in the device 100 and/or user information stored in the external device 2120 connected through the network 2110 . User information stored in the device 100 may be received from the external device 2120 and stored. User information stored in the external device 2120 may be received from the device 100 and stored. User information stored in the device 100 may be the same as user information stored in the external device 2120 . User information stored in the device 100 may be different from user information stored in the external device 2120 . Part of the user information stored in the device 100 may be the same as the user information stored in the external device 2120 .

The device 100 may update user information stored in the device 100 or the external device 2120 by using the user information corrected using the user information management method according to some embodiments. The device 100 may transmit the stored user information to the external device 2120 and request that the external device 2120 manage the user information.

22 is a flowchart illustrating a user information management method performed by the device 100 according to some embodiments. 23A to 23F are diagrams for explaining avatar-based images displayed by the device 100 based on user information according to some embodiments.

Referring to FIG. 22 , in step S2201 , the device 100 displays an avatar-based image based on user information. For example, the device 100 may generate and display the avatar-based image 2310 based on the user information 2300 of FIG. 23A . The avatar-based image 2310 may be said to be an image representing the user of the device 100 . The avatar-based image 2310 may be expressed as an avatar expressed as a 2D model or a 3D model, a cyber character, a virtual image of a user, a graphic icon of a user's virtual self, or an icon of the user. A method of expressing the avatar-based image 2310 in the present disclosure is not limited as described above.

In step S2201 , the device 100 may display a plurality of avatars based on user information as shown in FIGS. 23B to 23F .

Referring to FIG. 23B , the device 100 displays an avatar-based image (2340, hereinafter abbreviated as current image) based on current user information and an avatar-based image (2341, hereinafter abbreviated as future image) based on future user information together. Display.

The current image 2340 has a different hairstyle from the future image 2341 . This means that the hairstyle information (eg, the user's hairstyle is a cut) included in the current user information is different from the hairstyle information (eg, the user's hairstyle is a wave) included in the future user information. indicates that

Hair style information included in the current user information may be based on photo information, but the present disclosure is not limited thereto. For example, hair style information included in the current user information may be input by the user. Hair style information included in future user information may be based on the user's hair salon reservation information, but the present disclosure is not limited thereto. For example, hair style information included in future user information may be input by the user.

The device 100 may display the current image 2340 and the future image 2341 by dividing the user information into units of units, days, weeks, months, or years.

For example, in the case of a unit of one day, the device 100 generates and displays a current image 2340 based on today's user information (eg, user's today's schedule information), and tomorrow's user information (eg, , a future image 2341 may be generated and displayed based on the user's tomorrow schedule information).

For example, in the case of a week, the device 100 generates and displays the current image 2340 based on user information for the past week based on the current time point, and displays the current image 2340 based on the current time point for the future week. A future image 2341 may be generated and displayed based on the information.

The current image 2340 and the future image 2341 displayed on the device 100 may be the same. This indicates that current user information and future user information are the same.

The device 100 may not generate the current image 2340 and the future image 2341 in the above-described units. For example, the device 100 may display a current image 2340, for example, based on information about a current job (eg, a student) and information about a future hope (eg, a police officer) included in the user information. For example, an avatar-based image wearing a school uniform) and a future image 2341 (eg, an avatar-based image wearing a police uniform) may be generated.

Also, the device 100 may generate and display the future image 2341 based on the user's bucket list or wish list included in the user information. For example, when 10 items are included in the bucket list, the future image 2341 may generate and display an avatar-based image according to 10 different items. For example, when generating and displaying an avatar-based image according to the above-described 10 different items, the device 2341 divides the area in which the future image 2341 is displayed and displays it simultaneously or divides it into 10 pages. display, but the present disclosure is not limited thereto.

23C is an example in which the device 100 simultaneously displays an avatar-based image 2342 (hereinafter abbreviated as past image), a current image 2340, and a future image 2341 according to past user information according to some embodiments. .

Referring to FIG. 23C , the device 100 simultaneously displays a past image 2342 , a current image 2340 , and a future image 2341 having different hairstyles. This means that the user's hair style information included in the past user information, the current user information, and the future user information is different.

The device 100 generates a past image 2341, a current image 2340, and a future image 2341 based on user information in units of units, units, weeks, months, or years as shown in FIG. 23B. It can be created and displayed at the same time. For example, the device 100 generates a current image 2340 based on the current time and user information for a week before the current time, and a future image 2341 based on user information for a week after the current time. , and based on user information used to provide the current image 2340 , a past image 2342 may be generated based on user information for the previous week.

23D shows an avatar-based image (2343, hereinafter abbreviated as morning image) based on user information in the morning and an avatar-based image (2344, hereinafter abbreviated as afternoon image) based on user information in the afternoon of the device 100 according to some embodiments. This is an example of simultaneously displaying . The device 100 may generate the morning image 2343 and the afternoon image 2344 based on one of the units of units, units of units, units of weeks, units of months, or units of years; and the unit of generation of the afternoon image 2344 is not limited thereto.

Referring to FIG. 23D , the morning image 2343 indicates that user information includes information about “going to work in the morning”. The afternoon image 2344 indicates that user information includes information about “taking a violin lesson after work”.

23E shows an avatar-based image based on user information on weekdays (2345, hereinafter abbreviated as weekday image) and an avatar-based image based on user information on weekends (2346, hereinafter abbreviated as weekend image) of the device 100 according to some embodiments. This is an example of simultaneously displaying .

Referring to FIG. 23E , the device 100 generates a weekday image 2345 based on user information from Monday to Friday. Also, the device 100 may generate a weekend image 2346 on Saturday and Sunday based on user information. The device 100 may generate the weekday image 2345 and the weekend image 2346 based on one of a week, a few weeks, a monthly, a year, or a few years, but in the present disclosure, a weekday image The units of provision of 2345 and the weekend image 2346 are not limited as described above.

The weekday image 2345 indicates that user information includes information about “going to work from Monday to Friday”. The weekend image 2346 indicates that user information includes information about “jogging on the weekend”. In the present disclosure, the weekday image 2345 and the weekend image 2346 displayed on the device 100 are not limited to those shown in FIG. 23D .

23F illustrates an avatar-based image based on user information in spring (2347, hereinafter abbreviated as spring image) and an avatar-based image based on user information in summer (2348, hereinafter abbreviated as summer image) of the device 100 according to some embodiments. , an avatar-based image 2349 based on user information in autumn (hereinafter abbreviated as autumn image), and an avatar-based image 2350 based on user information in winter are simultaneously displayed.

The device 100 may generate, for example, a spring image 2347 based on user information from March to May. The device 100 may generate, for example, a summer image 2348 based on user information from June to August. The device 100 may generate, for example, an autumn image 2349 based on user information from September to November. The device 100 may generate a winter image 2350 based on user information from December to February.

A spring image 2347 and an autumn image 2349 indicate that user information includes information about “commuting to work by bicycle”. A summer image 2348 and a winter image 2350 indicate that user information includes information about “automatically going to work”. The device 100 may generate and display different spring images 2347 , summer images 2348 , autumn images 2349 , and winter images 2350 according to user information.

The device 100 may generate and display a spring image 2347 , a summer image 2348 , an autumn image 2349 , and a winter image 2350 as monthly images or monthly images.

The device 100 includes more various objects in the images 2310, 2340, 2341, 2342, 2343, 2344, 2345, 2346 shown in FIGS. 23A to 23F according to user information, and can be generated and displayed. .

Generating and displaying the avatar-based image by the device 100 mentioned in some embodiments described above means that the device 100 provides the avatar-based image.

Hereinafter, for convenience of description, the avatar-based image displayed on the device 100 will be referred to as the image 2310 illustrated in FIG. 23A . However, in the present disclosure, the avatar-based image is not limited thereto. For example, the avatar-based image 2310 being displayed on the device 100 may include the avatar-based images 2340, 2341, 2342, 2343, 2344, 2345, and 2346 shown in FIGS. 23B to 23F according to user information. can be one of In addition, the avatar-based image 2310 displayed on the device 100 is one of the avatar-based images 2340, 2341, 2342, 2343, 2344, 2345, and 2346 shown in FIGS. 23A to 23F according to user information. It may be an image to which at least one object is added.

The device 100 may express the avatar-based image 2310 as at least one of a static image and a dynamic image. For example, the device 100 may represent the avatar-based image 2310 as a static image when a first time elapses after expressing the avatar-based image 2310 as a dynamic image. After a second time elapses after expressing the avatar-based image 2310 as a static image, the device 100 may express the avatar-based image 2310 as a dynamic image. The first time and the second time may be set in advance. The first time and the second time may be the same time, but may be different times.

Referring to FIG. 23A , the user information 2300 includes information 2301 about the user's hobbies, the user's name 2302 , and information 2303 about the user's gender. In the present disclosure, the user information 2300 is not limited to that shown in FIG. 23A .

For example, the user information 2300 may further include at least one of the user information mentioned in the introduction of the specification. User information 2300 may include less information than shown in FIG. 23A . For example, the user information 2300 may include information 2301 about the user's hobbies and the user's name 2302 .

In order to generate the avatar-based image 2310 , the device 100 may generate user modeling information 2400 regarding the user information 2300 . The user modeling information 2400 may be referred to as a user modeling information table.

The device 100 may generate the user modeling information 2400 for the user information 2300 based on an ontology. A method of generating the user modeling information 2400 for the user information 2300 is not limited to an ontology basis.

24A is an ontology-based example of user modeling information 2400 for user information 2300 . The ontology is as described above with reference to FIG. 3 .

Referring to FIG. 24A , user modeling information 2400 includes a subject field 2401 , a predicate field 2402 , and an object field 2403 .

The subject field 2401 shown in FIG. 24A is a field related to the object of the user information 2300 . Referring to FIG. 24A , the target of user information 2300 is user 1 (user_1). 24A shows that user 1 (user_1) is a person using a class 2404 and a resource 2405 . The resource 2405 is an instance of the class 2404 .

The predicate field 2402 shown in FIG. 24A is a field indicating a formal vocabulary indicating a relationship between words included in the subject field 2401 and the object field 2403 . Accordingly, the predicate kit field 2402 shown in FIG. 24A includes a formal vocabulary about a name (Name), a formal vocabulary about a gender (Gender), and a formal vocabulary about a hobby (hasHobby).

“A” included in the object field 2403 can be expressed as the name of “user 1” by the formal vocabulary (Name) regarding the name. "Female" in the object field 2403 can be expressed as the gender of "user 1" by the gender-related formal vocabulary (Gender). "Viroline" in the object field 2403 can be expressed as the hobby of "User 1" by the standard vocabulary about hobby (hasHobby). The information included in the object field 2403 shown in FIG. 24A is user information of user 1 corresponding to the formal vocabulary included in the predicate field 2402 .

The user modeling information 2400 of FIG. 24A is an example generated in a Resource Description Framework (RDF) language. However, the device 100 may generate ontology-based user modeling information for the user information 2300 in languages such as Ontogogy Web Language (OWL) and Semantic Web Rule Language (SWRL).

24B is an example of a display size of an object included in an avatar-based image 2310 that may be displayed on the device 100 according to some embodiments. 24B is an example of a violin included in the user modeling information 2400 shown in FIG. 24A .

Referring to FIG. 24B , the device 100 may determine the display size of the object according to a possible weight value assigned to the object. 24B , the device 100 may determine the display size of the violin to be included in the avatar-based image 2310 according to the weight given to the violin included in the object field 2403 . The weight may be determined according to user information. The display size according to the weight may be set in advance.

Referring to FIG. 24B , the weight range is 0 to 1. When the weight is 0.3, the user's interest in the violin is lower than when the weight is 1. For example, if it is determined based on the user information of the device 100 that the user's hobby is the violin, and it is determined that the violin is played for 1 hour every other day, the device 100 may assign a weight of 0.5 to the violin. . In addition, if it is determined that the user's hobby is the violin based on the user information of the device 100 and it is determined that the user plays the violin for 1 hour every day, the device 100 may assign a weight to the violin as 1.

When the weight for the violin is 0.5, the device 100 determines the display size of the violin to be (50×50) pixels. Accordingly, the device 100 may generate an avatar-based image 2310 including a violin image of (50×50) pixels.

Also, when the weight of the violin is 1, the device 100 determines the display size of the violin as (100×100) pixels. Accordingly, the device 100 may generate an avatar-based image 2310 including a violin image of (100×100) pixels.

24C is a diagram for describing a display size of an object included in an avatar-based image 2310 according to some embodiments. 24C is an example of a woman included in the user modeling information 2400 shown in FIG. 24A .

Referring to FIG. 24C , the device 100 may determine the display size of the avatar 2311 to be included in the avatar-based image 2310 according to weight and height. The weight and height are the weight and height of a woman included in the object field 2403 of the user modeling information 2400 . The woman's weight and height are information included in the user information. The device 100 may select one of a plurality of preset display sizes.

Referring to FIG. 24C , the device 100 sets the reference value for the weight as 50 kg and the reference value for the height as 170 cm. However, the reference values in the present disclosure are not limited to the above. If the weight of the user included in the user information satisfies the condition of “(height-100)×0.9+5 > weight”, the device 100 sets the female avatar 2311 included in the avatar-based image 2310 as an obese image. can be displayed.

Also, if the user's weight satisfies the condition of “(height-100)×0.9 < weight”, the device 100 may display the female avatar 2311 included in the avatar-based image 2310 as a skinny image. If the user's weight satisfies the "other" condition, the device 100 may display the female avatar 2311 included in the avatar-based image 2310 as a normal image.

25 illustrates an example of mapping information 2500 between an object image and object identification information according to some embodiments. Referring to FIG. 25 , the device 100 may generate an avatar-based image 2310 using mapping information 2500 and user modeling information 2400 illustrated in FIG. 24 . As described with reference to FIGS. 24B and 24C , the display size of the object image shown in FIG. 25 may be adjusted or changed based on information included in user information.

The mapping information 2500 may be configured in consideration of all or some of the user information mentioned in the introduction of the present disclosure. The mapping information 2500 may be referred to as a mapping information table.

In the present disclosure, the mapping information 2500 may be set differently from that shown in FIG. 25 . For example, the mapping information 2500 may be configured to provide an object image of a still image as shown in FIG. 25 , but may be configured to provide an object image of a moving image. The mapping information may be stored in the device 100 or may be stored in the external device 2120 .

The image corresponding to the violin or guitar may include a standard image playing a violin or a standard image playing a guitar. The standard image may also be provided in the form of a still image or a form of a moving picture. The standard image may include a musical instrument and an avatar image playing a musical instrument.

When the standard image is stored in the mapping information 2500 , the device 100 may change the avatar playing the musical instrument to the user's avatar. The device 100 may change the avatar playing the musical instrument into the user's avatar by mapping the location information on the feature point required to display the avatar playing the instrument and the location information on the feature point needed to display the user's avatar. The location information about the feature point may be expressed as (x,y) coordinate values when the avatar is a two-dimensional image. The location information on the feature point may be expressed as (x, y, z) coordinate values when the avatar is a 3D image.

26 is a diagram for explaining a process of generating an avatar-based image 2310 using user modeling information 2400 and mapping information 2500 according to some embodiments.

26 , the device 100 detects information A, a woman, and a violin included in the object field 2303 of the user information 2300. The device 100 detects the detected information Each of the corresponding object images 2610 and 2620 from the mapping information 2500 using the object identification information (eg, “0000 0010” for a woman and “0000 0011” for a violin) assigned to The device 100 combines the detected object images 2610 and 2620 .

When combining the object images 2610 and 2620, the device 100 sets a coordinate value (a two-dimensional coordinate value or a three-dimensional coordinate value) used to render each of the object images 2610 and 2620 and a preset value. The binding condition information is available.

Coordinate values for each object image may be configured as a table that can be detected using object identification information included in the mapping information 2500 . For example, the device 100 may detect a coordinate value used when rendering the object image 2610 from the coordinate value table by using “0000 0010”. The coordinate value table may include position information of a feature point required to display each object image. The coordinate value table may be stored in the device 100 and/or the external device 2120 .

The coordinate value table may be a skeleton data table of an object image. For example, when the object image is an avatar like the image 2610 , the skeleton data table indicates the outline of the avatar. For example, the avatar's skeletal data may include information about face, head, neck, shoulders, chest, waist, hips, arm length, arm thickness, leg length, leg thickness, right hand, and/or left hand, etc. However, the present disclosure is not limited thereto.

For example, if the object image is a violin, such as image 2620, the skeleton data table indicates the outline of the violin. For example, the skeletal data of Viroline includes scroll, bobbin, bobbin, nut, neck, fingerboard, string, soundboard, purple, side plate, waist, braid, figure 1 hole, and string. It may include, but is not limited to, information about a pedestal, chinrest, and/or buttons, and the like.

The combining condition information may include position information to be combined between object images. The combining condition information may be configured as a table that can be detected using identification information of object images to be combined. For example, when combining the object images 2610 and 2620 as shown in FIG. 26 , the device 100 receives “0000 0010”, “0000, 0011”, which are identification information of the object images 2610 and 2620 , By using this, coupling condition information between the images 2610 and 2620 may be detected from the coupling condition information table.

When the coupling condition information includes position information indicating, for example, holding a violin bow in the right hand of the object image 2610 and holding the neck of the violin in the left hand of the object image 2610 , the device 100 may display the object image 2610 ) and the object image 2620 may be combined and provided as in the image 2630 . The location information indicates location information of a feature point required to display each object. The coupling condition information table may be stored in advance in the device 100 and/or the external device 2120 .

In order to generate the avatar-based image 2310 , the binding condition information may include location information indicating “hold a violin bow in your right hand, put a violin chin rest on your chin, and your left hand grips a violin neck”. When the violin image 2620 is a standard image playing the violin, the device 100 converts the coordinate value (or location information of the feature point) used to render the violin player to the coordinate value of the object image 2610 . can be changed

In step S2202 , the device 100 modifies the avatar-based image 2310 according to the user input. The user input may include a user input indicating a request for modification of the avatar-based image 2310 , a user input indicating object selection, or a user input indicating completion of modification, but the user input is not limited thereto in the present disclosure. The user input representing the edit request may represent the user input representing the edit request. The user input indicating completion of editing may indicate user input indicating completion of editing.

The user input may be based on at least one of a touch, a touch & drag, a gesture, a multi-modal, and a voice. The user input may be received from the external device 2120 .

When a user input indicating a modification request is received while the avatar-based image 2310 is being displayed, the device 100 may display a modifiable object.

27A to 27F are examples in which the device 100 displays a modifiable object according to some embodiments.

Referring to FIG. 27A , when a user input indicating selection of the edit request button 2701 is received, the device 100 highlights the user's name 2702 and the violin 2703 . Accordingly, the user can know that the user's name 2702 and the violin 2703 are editable objects.

In FIG. 27B , when a user input indicating selection of the correction request button 2705 is received, the device 100 overlaps and displays a message 2706 indicating the name in a partial area of the user's name 2702 . Also, the device 100 overlaps and displays a message 2707 indicating a hobby in a partial area on the violin 2703 . The message 2707 indicating the hobby may include information related to the violin 2703 (eg, playing the violin). Accordingly, the user can know that the user's name 2702 and the violin 2703 are editable objects, or the user's name 2702 and the user's hobby 2707 are editable objects.

Referring to FIG. 27C , the device 100 may display a tray 2712 including icons 2710 and 2711 for editable objects together with an avatar-based image 2310 .

In FIG. 27C , when a user input indicating selection of a modification request button 2709 is received, the device 100 provides a tray 2712 including an icon 2710 related to a hobby and an icon 2711 related to a name. do. The hobby icon 2710 may include information related to the violin 2703 . Accordingly, the user can know that the user's name 2702 and the violin 2703 or the user's name 2702 and the hobby item are editable objects. The tray 2712 may further include identification information (hobbies, names) regarding the icons 2710 and 2711 .

Referring to FIG. 27C , the device 100 displays a tray 2712 at the bottom of the screen. However, the tray 2712 may be displayed on at least one of the left side, the right side, the upper side, and the lower side of the screen. If the number of editable object icons is greater than the number of object icons that can be simultaneously displayed on the tray 2712 , the device 100 displays the object being displayed through the tray 2712 according to a scroll input based on the tray 2712 . It is possible to display icons of selectable objects while moving the icons of . The scroll input based on the tray 2712 may include a scroll input based on the area of the tray 2712 .

When the number of editable object icons is greater than the number of object icons that can be simultaneously displayed on the tray 2712 , the device 100 may provide a plurality of trays divided by preset categories.

When the device 100 provides a plurality of trays, the device 100 arranges the plurality of trays in at least one position of the left, right, upper or lower side of the screen, or arranges the plurality of trays hierarchically. structure can be placed. The category may be set according to the type of editable object, but is not limited thereto.

When an icon of one object included in the tray 2712 is selected in FIG. 27C , the device 100 may highlight an object corresponding to the icon of the selected object.

27D is an example in which the device 100 provides information about an object that can be modified according to a touch-based input to the object through a speech bag, according to some embodiments.

In FIG. 27D , when a user input indicating selection of the edit request button 2714 is received, the device 100 may activate an editable image among displayed images.

When a touch-based input 2715 indicating selection of a modifiable object (eg, violin 2703) from among the displayed images is received, the device 100 displays modifiable information about the object (violin 2703). (eg, hobby) is displayed. The device 100 may display the bag 2716 at a position adjacent to the object 2703 to indicate that the bag 2716 is related to the object 2703 . Accordingly, the user can know that the violin 2703 is a modifiable object, and user information regarding hobbies can be modified.

Referring to FIG. 27E , the device 100 provides a modifiable object as a pop-up window 2721 . In FIG. 27E , when a user input indicating selection of a correction request button 2718 is received, the device 100 displays a pop-up window 2721 including information 2719 and 2720 about a modifiable object. The pop-up window 2721 may include information about one object.

Accordingly, the user can know the information 2719 and 2720 about the modifiable object. When information on one object included in the pop-up window 2721 is selected, the device 100 may highlight and display an object corresponding to the information on the selected object.

Referring to FIG. 27F , the device 100 may display a modifiable object using an independent page. In FIG. 27F , when a user input indicating selection of the edit request button 2723 is received, the device 100 opens a page 2726 including information 2724 and 2725 on a modifiable object. Accordingly, the user can know the modifiable object. The device 100 may display information about at least one modifiable object through the page 2726 .

When a modifiable object is selected in FIG. 27E or 27F , the device 100 may highlight an object corresponding to the selected modifiable object.

27A to 27F receive a user input indicating selection of the correction request buttons 2701 , 2705 , 2709 , 2714 , 2718 , and 2723 , but in the present disclosure, the user input indicating a correction request is not limited as described above. For example, a user input indicating a correction request input may be received using another button of the device 100 or based on a user's gesture.

28 is a flowchart illustrating a process in which the device 100 modifies an avatar-based image 2310 according to a user input, according to some embodiments.

Referring to FIG. 28 , in step S2801 , the device 100 receives a first selection signal for one object among modifiable objects. In step S2802, the device 100 displays a candidate object for the selected object.

29A to 29E illustrate examples in which an object is modified based on a candidate object provided by the device 100 according to some embodiments.

As shown in the screen 2910 of FIG. 29A , when the device 100 highlights a modifiable object, a first selection signal 2715 indicating selection of the violin 2703 as shown in the screen 2920 is received. When done, the device 100 displays a candidate object related to the selected violin 2703 .

Referring to the screen 2920 of FIG. 29A , the device 100 provides a guitar item 2912 , a baseball item 2913 , and a badminton item 2914 as candidate objects related to the violin 2703 , but a candidate in the present disclosure Objects are not limited to this.

Referring to the screen 2920 of FIG. 29A , the device 100 displays information about the candidate object in the form of a pop-up window 2921 . Referring to the screen 2920 of FIG. 29A , the device 100 provides information about the candidate object in the form of text.

However, in the present disclosure, a method of providing information about a candidate object is not limited as described above. For example, as shown in the screen 2930 shown in FIG. 29B , the device 100 may provide information about the candidate object in the form of a rectangular pop-up window 2931 . The rectangular pop-up window 2931 horizontally arranges images corresponding to each candidate object. Information on the candidate object included in the rectangular pop-up window 2931 may be provided as a symbol, an icon, a representative image, or the like, but in the present disclosure, the information on the candidate object is not limited as described above.

As shown in the screen 2940 of FIG. 29B , the device 100 may provide information about the candidate object in the form of a ring-type pop-up window 2941 . The ring-type pop-up window 2941 may include information about a candidate object in a symbol form or a text form.

When the number of candidate objects is greater than the number that can be simultaneously displayed in the pop-up windows 2921, 2931, and 2941, the device 100 displays the pop-up window 2921 according to a scroll input based on the pop-up windows 2921, 2931, and 2941. , 2931 , and 2941 may provide information about the candidate object while moving the information about the candidate object displayed on the screens. The device 100 may move information about the candidate object according to a preset arrangement order.

When the number of candidate objects is greater than the number that can be simultaneously displayed in the pop-up windows 2921, 2931, and 2941, the device 100 performs an input (eg, touch & drag), the display area of the pop-up windows 2921 , 2931 , and 2941 may be expanded to provide information about the candidate object.

For example, as shown in the screen 2950 shown in FIG. 29C , the device 100 expands the pop-up window 2931 according to the touch & drag input 2951 to provide information about the candidate object. there is. The device 100 may further provide a pop-up window 2931 and an independent pop-up window 2961 according to the touch & drag input 2951 like the screen 2960 shown in FIG. 29D .

In step S2803 of FIG. 28 , the device 100 receives a second selection signal based on information about the candidate object being displayed. In step S2804 , the device 100 modifies the selected object as the selected candidate object according to the second selection signal.

When the second selection signal for selecting the baseball item 2913 is received on the screen 2970 of FIG. 29E , as shown in the screen 2980 , the device 100 displays the avatar-based image playing the violin on the left hand. Change to the avatar-based image of wearing a baseball glove and holding a baseball in his right hand.

In the present disclosure, the avatar-based image playing baseball is not limited to the bar shown on the screen 2980 . For example, as shown in screen 2990 of FIG. 29F , when device 100 displays an avatar-based image holding violin 2703 in its left hand, indicating selection for baseball item 2913 , When the user input is received, the device 100 may display an avatar-based image holding the baseball bat 2996 in the left hand. In addition, when a user input indicating selection of a movie item is received through the pop-up window 2911 , the device 100 may display an avatar-based image holding a movie CD or movie tape in the left hand.

When a user input indicating selection of the modified button 2704 is received on the screen 2980 of FIG. 29E or the screen 2995 of FIG. 29F , the device 100 displays the modified image 2320 illustrated in FIG. 23A can be created and displayed. The modified image 2320 shown in FIG. 23A is an avatar-based image playing a guitar.

30 is a flowchart illustrating a process in which the device 100 provides service information that can be provided for each object included in the avatar-based image 2310, according to some embodiments.

Referring to FIG. 30 , in step S3001 , the device 100 receives a first selection signal indicating selection of an object. In step S3002, the device 100 provides at least one piece of service information based on the selected object.

31A to 31D are examples of screens for explaining the user information management method illustrated in FIG. 30 . When an avatar-based image is being displayed as shown in the screen 3110 of FIG. 31A , when a user input indicating selection of the edit request button 2701 is received, the device 100 is modified as shown in the screen 3120 of FIG. 31A . Highlight possible objects.

When the first selection signal for the violin 2703 is received on the screen 3120 of FIG. 31A , the device 100 provides service information available based on the violin 2703 as shown on the screen 3130 of FIG. 31B . is provided through a pop-up window 3131 . Referring to the pop-up window 3131 , the available service information may include a concert reservation item, a community access item, and/or a specialized shopping mall item, but the service information available in the present disclosure is not limited as described above.

In step S3003, upon receiving the second selection signal 3132 indicating selection of the selected object (violin 2703 in FIG. 31B), in step S3004, the device 100 selects the selected object (violin 2703). Provides information about at least one candidate object. Information on the candidate object may be provided in the form of a pop-up window 3141 as shown in the screen 3140 of FIG. 31B .

Accordingly, the device 100 may recognize the first selection signal in step S3001 as a preliminary selection for the desired object, and recognize the second selection signal in step S3003 as a final selection for the desired object. . The preliminary selection indicates a selection waiting state for an object. The final selection indicates the selection completion status for the object.

In step S3005, the device 100 receives a third selection signal based on the information on the candidate object. Accordingly, in step S3006, the device 100 provides at least one piece of service information for the selected candidate object.

Referring to FIG. 31C , when a baseball item is selected from information on a candidate object included in the pop-up window 3141 provided through the screen 3150 , the device 100 displays a baseball as shown on the screen 3160 . Service information that can be provided based on the item is provided through the bag 3151 .

In step S3007 , the device 100 receives a fourth selection signal for the selected candidate (baseball). For example, a double touch 3152 as shown in the screen 3160 of FIG. 31C may be received as the fourth selection signal.

The device 100 recognizes the third selection signal for the candidate object in step S3005 as preliminary selection for the candidate object, and converts the fourth selection signal for the candidate object selected in step S3007 to the final selection for the selected candidate object. can recognize A preliminary selection indicates a selection waiting state, and a final selection indicates a selection completion state.

In operation 3008 , the device 100 modifies the selected object as the selected candidate object. Accordingly, the device 100 may display an avatar-based image based on a baseball item as shown on the screen 3170 of FIG. 31D . When a user input indicating selection of the edit completion button 2704 is received on the screen 3170 of FIG. 31D , the device 100 completes the image correction.

According to the flowchart shown in FIG. 30 , the user of the device 100 may check service information that can be provided based on the candidate object before modifying the avatar-based image.

On the other hand, when the avatar-based image 2310 is modified according to the user input in step S2202 of FIG. 22 , in step S2203 , the device 100 transmits the user information 2300 based on the modified avatar-based image 2320 to the user information ( 2330) is updated as shown. The user information 2330 includes information about a user's hobbies (guitar playing) 2331 , a user name 2332 , and information about a user's gender (female) 2333 .

As the user information 2300 is updated with the user information 2330 , the user modeling information 2400 shown in FIG. 24A may be updated with the user modeling information 3200 shown in FIG. 32 . 32 is an example of user modeling information 3200 updated by the device 100 according to some embodiments.

When the display size of the avatar 2311 included in the avatar-based image 2310 is changed, the device 100 may change user information according to the changed display size. For example, when the display size of the avatar 2311 included in the avatar-based image 2310 is normal and the display size of the avatar 2311 is changed to diamond according to a user input, the device 100 displays the user information The included user's weight and height can be updated with values that can be expressed in a lean state.

In step S2201 of FIG. 22 , when the avatar-based image 2210 is being displayed by the device 100 , the device 100 may also provide a sound based on the user information 2300 . The sound may be provided as background music of the avatar-based image 2310 or may be provided as a voice of the avatar 2311 . The voice of the avatar 2311 may provide, for example, guide information related to the user information 2300 (eg, “My name is A, I am a woman, and my hobby is playing the violin.”).

The device 100 may provide the voice of the avatar 2311 in various modes based on the female voice. The various modes may include, for example, a mode such as a speech mode, a musical mode, or a theater mode, but is not limited to the above-described various modes based on voice in the present disclosure. The voice mode may be set as a default in the device 100, but may be set or changed by a user.

When the sound is provided as background music, the device 100 may provide a violin performance song based on the user's hobby information 2301 (violin performance) included in the user information 2300 . The violin performance song may be determined by the device 100 or the server 2120 based on user information other than the user information 2300 shown in FIG. 23A .

The device 100 may further include the background image of the avatar 2311 in the avatar-based image 2310 based on the user information 2300 to provide it. The background image may be expressed as a violin performance stage image based on the user's hobby information (2301, playing violin) or as a single color image (eg, pink) based on the user's gender information (2303, female). In the present disclosure, the background image is not limited as described above. For example, when it is recognized that the user mainly plays the violin 2312 in the evening based on user information, the device 100 may display a night image as a background image. The device 100 may determine, for example, that the user mainly plays the violin 2312 in the evening based on the violin playing time period included in the user information.

As described above, when at least one of a background image and a sound is further included in the avatar-based image 2310 and provided, the device 100 may provide the background image and the sound as a modifiable object.

If the background image and sound are provided by user information other than the information included in the user information 2300, and the background image and/or sound is modified, the device 100 responds to the modified background image and/or the modified sound. User information can be updated based on

33 is a flowchart of a method of managing user information by modifying an avatar-based image by the device 100, according to some embodiments. 33 is an example of updating user information according to a user input accepting a modified avatar-based image.

In step S3301 of FIG. 33 , the device 100 displays the avatar-based image 2310 based on user information as in step S2201 described above. In step S3302, the device 100 modifies the avatar-based image according to the user input as in the above-described step S2202.

In step S3303 , the device 100 displays the modified avatar-based image 2320 . In step S3304, when a user input indicating acceptance of the modified avatar-based image 2320 is received, the device 100 proceeds to step S3305. A user input indicating acceptance of the modified avatar-based image 2320 may be input through the device 100 , but may be input using an external device 2120 . In order to receive a user input indicating acceptance of the modified avatar-based image 2320 , the device 100 may display a message asking whether to accept the modified avatar-based image 2320 .

In step S3304, if a user input indicating acceptance of the modified avatar-based image 2320 is not received, the device 100 proceeds to step S3301 to display the avatar-based image 2310 before being modified.

In step S3304, if a user input indicating acceptance of the modified avatar-based image 2320 is not received, the device 100 proceeds to step S3303 to display the modified avatar-based image 2320 while requesting additional modifications. A waiting state may be maintained for the indicated user input.

In operation S3304 , if a user input is not received for a preset period of time, the device 100 may determine that a user input indicating acceptance of the modified avatar-based image 2320 has not been received.

In step S3304 , when a user input indicating to accept the modified avatar-based image 2320 is received, the device 100 updates user information based on the modified avatar-based image 2320 in step S3305 .

34 is a flowchart of a method for managing user information by modifying an avatar-based image by the device 100, according to some embodiments.

Referring to FIG. 34 , in step S3401 , the device 100 receives user information from the external device 2120 . The device 100 may receive user information from the external device 2120 according to a user input requesting user information. For example, the device 100 may request user information from the external device 2120 according to a user input requesting user information registered in the social media server, and thus receive user information from the external device 2120 . there is.

Also, user information may be received from the external device 2120 without a request from the device 100 . For example, when the device 100 is in a sleep mode or an application A is running, the user's health information may be received from the external device 2120 . The user information received in step S3401 may include at least one of the user information mentioned in the introduction of the present disclosure. The received user information may be stored in the device 100 .

In step S3402, the device 100 generates and displays an avatar-based image based on the received user information. Generating the avatar-based image based on the received user information may be performed as in step S2201 of FIG. 22 described above.

In step S3403, the device 100 modifies the displayed avatar-based image according to a user input indicating a modification request. Modifying the avatar-based image according to the user input may be performed as in step S2202 of FIG. 22 described above.

In step S3404, the device 100 updates the user information according to the modified avatar-based image. The device 100 may change the user information received in step S3401 according to the modified avatar-based image and request an update of the user information while transmitting the changed user information to the external device 2120 . Accordingly, the external device 2120 may update user information. After requesting the update of the user information, the device 100 may automatically delete information stored in the device 100 as the user information is received in step S3401.

35 is an operation flowchart of a method for managing user information in which the device 100 corrects and/or adds information using an avatar-based image, according to some embodiments.

In step S3501 , the device 100 displays an avatar-based image 2310 according to user information. Step S3501 may be performed similarly to step S2201 of FIG. 22 described above.

In operation S3502 , the device 100 may modify the displayed avatar-based image 2310 according to a user input. The modification of the avatar-based image 2310 performed in step S3502 may be performed similarly to the above-described step S2202 of FIG. 22 .

Also, in step S3502 , the device 100 may add new information (or a new object) to the displayed avatar-based image 2310 according to a user input. Adding new information may be performed based on image information corresponding to addable information provided by the device 100 . The image information may include at least one of an image, a symbol, or an icon representing additional information.

When an image corresponding to the addable information is selected, the device 100 may provide a modified image 2320 to which the selected image is added. The process of adding new information in the present disclosure is not limited as described above.

For example, when an image for additional information (eg, an accessory) is selected, the device 100 may provide an input window for inputting information (eg, an earring) related to the selected image. When information related to the selected image is received through the provided input window, the device 100 generates an image (eg, an earring) of the selected information based on the received information, and a modified image 2320 including the generated image ) can be provided.

In step S3503 , the device 100 updates user information according to the avatar-based image to which the modified and/or image is added. For example, when travel preparations are added as shown in FIGS. 8A to 8C , the device 100 updates user information so that information on the added preparations is included in the user information.

36 is a flowchart of a method for managing user information so that the device 100 may provide a service based on an object selected from an avatar-based image, according to some embodiments. In FIG. 36 , an object refers to an image and/or text corresponding to editable user information included in the avatar-based image.

In step S3601, the device 100 displays an avatar-based image according to user information. The device 100 may display the avatar-based image as in step S2201 of FIG. 22 described above. For example, the device 100 may display the avatar-based image shown in FIG. 31A . In this case, the device 100 may display an avatar-based image that does not include the correction request button 2701 and the correction completion button 2704 shown in FIG. 31A .

In step S3602 , when a user input indicating selection of one object 2703 from the avatar-based image 3110 being displayed on the device 100 is received, the device 100 provides service information based on the selected object 2703 . provides For example, when a user input indicating selection of an object 2703 is received as shown in the screen 3120 of FIG. 31A , as shown in the screen 3130 of FIG. 31B , the device 100 displays the selected object A pop-up window 3131 including at least one service information based on 2703 is opened.

Referring to FIG. 31B , service information included in the pop-up window 3131 includes a concert reservation item, a community access item, and a specialized shopping mall item. In the present disclosure, service information included in the pop-up window 3131 is not limited as described above. In addition, service information based on the object 2703 selected in the present disclosure may be provided in a form included in the tray 2712 of FIG. 27C .

In step S3603, when a user input indicating selection of one piece of service information is received, the device 100 provides the selected service. For example, when a user input indicating selection of a concert reservation item included in the pop-up window 3131 is received, the device 100 may execute an application providing a violin concert reservation service.

37 is a flowchart of a method for managing user information in which the device 100 provides a question and answer (QA) service based on an avatar-based image displayed, according to some embodiments.

In step S3701 of FIG. 37 , the device 100 displays an avatar-based image according to user information. For example, the device 100 may display the avatar-based image 2310 shown in FIG. 23A according to user information.

In step S3702, when question information is received, in step S3703, the device 100 provides answer information to the received question information.

The question information in step S3702 refers to question information related to the displayed avatar-based image. For example, the question information could be “Tell me the violin concert I have booked”, “Tell me if there is a violin lesson tomorrow,” “What time was the violin lesson yesterday?”, or “Just before tomorrow’s violin lesson. What schedule do you have?” and so on.

The question information received in step S3702 may use the user's voice signal. If the device 100 cannot process (or recognize) the question information received in step S3702, the device 100 may display an error message or output an error message as a voice signal.

Also, when the device 100 fails to recognize the received question information, the device 100 may provide guide information on the inputable question information. The guide information for the question information may provide inputable question information in the form of a list. The guide information for the question information may provide information about the type of question that can be entered.

When recognizing the received question information, the device 100 may consider user information. For example, if the received question is “Give me information about Lincoln”, the device 100 may determine whether “Lincoln” is a human “Lincoln” or a car “Lincoln” based on user information. When it is determined based on the user information that the user of the device 100 is a person interested in cars, the device 100 may recognize “Lincoln” included in the question information as “Lincoln” of the car.

On the other hand, when the question information received in step S3702 is “Tell me information about the violin concert I have reserved”, in step S3703, the device 100 responds to the screen 3810 or the screen 3820 shown in FIG. 38 as answer information can be provided as For example, if the question information received in step S3702 is “Tell me about the concert information I recently updated”, in step S3703, the device 100 performs the concert information 3810 before the update shown in FIG. 38 and After the update, concert information 3820 may be provided together. Accordingly, the user can easily check the concert information before the update and the concert information after the update.

38 is an example in which the device 100 provides answer information according to received question information, according to some embodiments. Referring to FIG. 38 , it is an example of simultaneously providing answer information based on user information before being updated and answer information based on user information after being updated.

The device 100 may provide the screen 3810 and the screen 3820 by partially overlapping each other. The device 100 may provide the screen 3810 and the screen 3820 in the form of an unfolded e-book. When the screens 3810 and 3820 are provided in the form of an unfolded e-book, when a user input of touch & drag is received based on the screen 3810 or a corner portion or side of the screen 3820, the device 100 may close the screens 3810 and 3820 in the dragged direction. For example, when a user input of touching the rightmost side of the screen 3820 and dragging it toward the left side is received, the device 100 may close the screen in such a way that the screen 3820 covers the screen 3810 .

39 is a flowchart of a user information management method based on the device 100 and the external device 2120 according to some embodiments. 39 is a case in which the external device 2120 generates an avatar-based image.

In step S3901, the external device 2120 generates an avatar-based image based on user information. The external device 2120 may generate an avatar-based image based on user information as in step S2201 of FIG. 22 described above. For example, if the external device 2120 is a social media server, the external device 2120 may generate an avatar-based image based on user profile information registered in the social media server.

In step S3902 , the external device 2120 transmits the generated avatar-based image to the device 100 . Step S3902 may be performed according to a user information request from the device 100 .

In step S3903, the device 100 displays an avatar-based image. In step S3904, the device 100 modifies the displayed avatar-based image. Step S3904 may be performed when a user input indicating a modification request is received based on the avatar-based image being displayed. The method of modifying the avatar-based image may be performed as in step S2202 of FIG. 22 described above.

If the avatar-based image is modified, in step S3905 , the device 100 transmits the modified avatar-based image to the external device 2120 . In step S3906, the external device 2120 updates user information based on the received modified avatar-based image. Step S3906 may be performed like step S2203 of FIG. 22 .

40 is a flowchart of a user information management method based on the device 100 and the external device 2120 according to some embodiments. 40 is a case in which the device 100 generates an avatar-based image.

In step S4001, when user information is transmitted from the external device 2120 to the device 100, in step S4002, the device 100 generates an avatar-based image based on the received user information.

In step S4003, the device 100 displays the generated avatar-based image. In step S4004, the device 100 modifies the displayed avatar-based image according to the received user input.

In step S4005, the device 100 updates user information based on the modified avatar-based image. In step S4006 , the device 100 transmits the updated user information to the external device 2120 . The external device 2120 updates user information stored in the external device 2120 based on the received updated user information.

41 is a flowchart of a user information management method based on the device 100 and the wearable device 4110 according to some embodiments. 41 is an example of displaying information about an object included in an avatar-based image through the wearable device 4110. Referring to FIG.

In step S4101 , the device 100 displays the avatar-based image according to the user information, and in step S4102 , the device 100 transmits information on a modifiable object included in the avatar-based image to the wearable device 4110 . do.

In step S4103, the wearable device 4110 displays information about the received object. In the present disclosure, information about an object may be displayed in the form of an image, but is not limited thereto. For example, the wearable device 4110 may display information about the object in the form of text. When a selection signal based on information about the object being displayed is received in step S4104 , the wearable device 4110 transmits information about the selected object to the device 100 in step S4105 .

In step S4106, the device 100 searches for information about a candidate object about the selected object from the stored information. Information about the candidate object may be retrieved from an external storage device of the device 100 . Information about the candidate object may be retrieved from the external device 2120 connected to the device 100 . The device 100 may search for information about the candidate object using identification information of the selected object as a keyword.

In operation S4107 , the device 100 transmits information about the searched candidate object to the wearable device 4110 . In step S4108, the wearable device 4110 displays information about the received candidate object.

For example, if the object selected in step S4104 is the violin 2703 of FIG. 29A described above, the candidate object that can be displayed in step S4108 is an item included in the pop-up window 2921 shown in the screen 2920 of FIG. 29A can take

For example, if the object selected in step S4104 is an icon 2710 or a hobby item of FIG. 27C , the candidate object that can be displayed in step S4108 is a hobby-related item (eg, swimming, cycling, or photos, etc.).

The information on the candidate object may be displayed in the form of an image and/or text, but the display form of the information on the candidate object in the present disclosure is not limited as described above.

In operation S4109 , when a selection signal for a candidate object is received, in operation S4110 , the wearable device 4110 transmits information about the selected candidate object to the device 100 .

In step S4111, the device 100 modifies the avatar-based image by using the selected candidate object. In step S4112, the device 100 updates user information based on the modified avatar-based image. Updating the user information in step S4112 may be performed as in step S2203 of FIG. 22 described above.

42 is a flowchart of a user information management method based on the device 100 and the wearable device 4210 according to some embodiments. 42 is an example of modifying an avatar-based image based on user information obtained by the wearable device 4210 .

In step S4201, the device 100 displays an avatar-based image according to user information. In step S4201, the device 100 may display the avatar-based image as in step S2201 of FIG. 22 described above.

In step S4202, the wearable device 4210 acquires user information. The user information acquired by the wearable device 4210 may include, for example, user health information or location information of the wearable device 4210, but information that may be acquired by the wearable device 4210 in the present disclosure is not limited thereto.

The above-described steps S4201 and S4202 may be performed simultaneously. Step S4202 may be performed before step S4201.

In step S4203 , the wearable device 4210 transmits the obtained user information to the device 100 . The device 100 modifies the avatar-based image being displayed in step S4201 based on the received user information.

For example, if the received user information is information obtained by measuring the user's body temperature and it is determined that the measured body temperature is higher than the reference value, the device 100 partially changes the face color of the avatar-based image displayed in step S4201. It can be edited in red. Accordingly, the user can confirm that the user is heating up through the avatar-based image being displayed through the device 100 .

For example, if the received user information is information indicating a user's happy emotion, the device 100 may change the face of the avatar-based image displayed in step S4201 to a smiling face. Accordingly, the user can confirm that the user is in a good mood through the avatar-based image displayed through the device 100 .

As described above, the device 100 may variously modify the avatar-based image displayed in step S4201 according to information measured and transmitted by the wearable device 4210 . The image modification may modify at least one modifiable object among objects included in the image as well as the face of the image. The modifiable object may include at least one of a background image and a sound, but the modifiable object in the present disclosure is not limited as described above.

43 is a functional block diagram of the device 100 according to some embodiments. Referring to FIG. 43 , the device 100 may include a user input unit 4310 , a control unit 4320 , a display 4330 , and a memory 4340 .

The user input unit 4310 may include a user input indicating a user information request, a user input indicating a request to modify user information, a user input indicating a request to add user information, a user input indicating completion of modification of user information, and a user indicating object selection input, a user input indicating a service information request, a user input indicating a service information selection, a user input indicating acceptance of a modified avatar-based image, or a user input capable of controlling the device 100 In , the user input is not limited as described above.

The user input unit 4310 may be mounted on the device 100 . The user input unit 4310 that may be mounted on the device 100 may include, for example, a key pad, a dome switch, and a touch pad (contact capacitive type, pressure resistance type, infrared rays). It may include at least one of a sensing method, a surface ultrasonic conduction method, an integral tension measurement method, a piezo effect method, etc.), a jog wheel, and/or a jog switch, but the user input unit 4310 in the present disclosure is not limited to the above. does not

The user input that may be received through the user input unit 4310 may include a touch-based user input, a movement-based user input, a vision-based user input, and/or a user voice-based user input. At least one of the inputs may be included, but in the present disclosure, the user input is not limited as described above.

A touch-based user input is, for example, a tap (tap or touch), a long tap (long tap or long touch), a touch and hold, a touch and drag, and a double tap (double tap). tap), drag, panning, flick, drag and drop, or sweep, but in the present disclosure, touch-based user input is limited as described above. doesn't happen

A movement-based user input may include, for example, a user input based on a motion-based user gesture of the device 100 (eg, shake the device, rotate the device, or lift the device, etc.). However, in the present disclosure, the motion-based user input is not limited as described above.

The vision-based user input may include a user input recognized by analyzing an input image obtained by a camera (not shown) included in the device 100 without contact between the device 100 and the user. The vision-based user input is not limited to the above.

The controller 4320 refers to a processor that controls the operation of the device 100 . The controller 4320 may operate by loading a program for executing the user information management method according to some embodiments stored in the memory 4340 . The controller 4320 may store a program for executing the user information management method according to some embodiments in the memory 4340 . The controller 4320 may update a program for executing the user information management method according to some embodiments stored in the memory 4340 .

The controller 4320 may generate an avatar-based image based on user information according to a user information management method according to some embodiments, and display it on the display 4330 . When a user input requesting to modify the avatar-based image is received from the user input unit 4310, the controller 4320 modifies the avatar-based image according to the received user input. The controller 4320 may update user information based on the modified avatar-based image.

2, 7, 9, 11, 13, 14, 16, 18, 20, 22, 28, 30, 33, 34, and FIG. 35 , 36 , 37 , 39 , 40 , 41 , or 42 may perform an operation of the device 100 based on the flowchart shown in FIG.

In addition, when a user input requesting user information of a specific period is received through the user input unit 4310, the controller 4320 displays a plurality of avatar-based images according to the user information of the specific period in response to the user input. The display 4330 is controlled to be displayed on the 4330 . A plurality of avatar-based images displayed on the display 4330 may be determined according to a criterion for dividing a specific period.

In addition, when a user input requesting user information for a specific period is received through the user input unit 4310 , the controller 4320 displays a plurality of avatar-based images according to the user information of the specific period in response to the user input 4330 . ) to control the display 4330 to be displayed. A plurality of avatar-based images displayed on the display 4330 may be determined according to a theme item included in user information of a specific period.

In addition, when a user input requesting display of editable information is received through the user input unit 4310 , the controller 4320 corrects the editable part of the avatar-based image displayed on the display 4330 in response to the user input It can be marked to distinguish it from the impossible part.

Also, if the provision period for the avatar-based image is set, the controller 4320 checks the provision period. When the provision period expires, the controller 4320 may destroy the avatar-based image. The above-described information regarding the provision period and the avatar-based image may be stored in the memory 4340 , but the present disclosure is not limited thereto. For example, the information regarding the above-described provision period and the avatar-based image may be stored in the external device 2120 .

Also, when an avatar-based image based on the user's health information is being displayed on the display 4330 and a user input indicating a medical information request is received through the user input unit 4310, the controller 4320 responds to the user input Medical information related to the user's health information is displayed on the display 4330 .

When a user input regarding medical information selected based on the medical information provided through the display 4330 is received through the user input unit 4310, the controller 4320 may update the user's schedule information according to the selected medical information. .

Also, when a user input indicating addition of information to the avatar-based image is received through the user input unit 4310 , the controller 4320 displays an input window for adding information on the display 4330 . When information is received through the input window, the controller 4320 adds an image corresponding to the received information to the avatar-based image displayed on the display 4330 . The controller 4320 may update the user's schedule information according to the avatar-based image including the added image being displayed on the display 4330 . The user's schedule information may be stored in the memory 4340 and/or the external device 2120 .

In addition, an avatar-based image according to the user's future desired information may be displayed on the display 4330 by being controlled by the controller 4320 . When an avatar-based image according to the user's future hope information is being displayed on the display 4330 , when a user input requesting information related to the future hope information is received through the user input unit 4310 , the controller 4320 controls the user input In response, information related to future hope information is provided through the display 4330 . When a user input for selecting at least one of information related to future desired information provided through the display 4330 through the user input unit 4310 is received, the control unit 4320 may update the user's schedule information according to the selected information. can

In addition, when a user input requesting usage history information for each usage period of the device 100 is received through the user input unit 4310 , the control unit 4320 responds to the received user input, and uses history information of the device 100 . A plurality of avatar-based images according to . may be displayed on the display 4330 .

In addition, an avatar-based image according to user information registered in social media may be displayed on the display 4330 by being controlled by the controller 4320 . At this time, when a user input requesting correction is received through the user input unit 4310 , the controller 4320 corrects and modifies the avatar-based image displayed on the display 4330 in response to the received user input. User information registered on social media can be updated according to the avatar-based image. The user information registered in social media may include user profile information registered in the above-described social media server. In the present disclosure, user information registered in social media is not limited as described above.

The display 4330 may display information processed by the device 100 . The display 4330 is the above-described FIGS. 1, 5A to 5E, 6A to 6E, 8A to 8C, 10, 12A to 12C, 15A to 15E, 17A to 17D, and FIGS. 19A, 19B, 23A to 23F, 27A to 27F, 29A to 29F, 31A to 31D, or FIG. 38 may display a GUI (Graphic User Interface) screen.

The display 4330 may be used as an output device and an input device when the touch pad is configured as a touch screen by forming a layer structure. The display 4330 may include a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, and a three-dimensional display (3D). display), and/or at least one of an electrophoretic display (EPD).

Depending on the implementation of the device 100 , the device 100 may include two or more displays 4330 . In this case, two or more displays 4330 may be disposed to face each other using a hinge. The display 4330 may be configured as a transparent display.

The memory 4340 may store a program for processing and control of the controller 4320 . The program stored in the memory 4340 may include an operating system (OS) program and various application programs. The various application programs may include an application capable of executing at least one of a method for managing user information according to some exemplary embodiments and a method for providing a service according to an avatar-based image, but the application is not limited thereto.

The memory 4340 stores information managed by an application program. For example, user information according to some embodiments, user modeling information based on user information, coordinate values for each object, or/and combination condition information for each object may be stored, but in the present disclosure, it is stored in the memory 4340 The information is not limited to the above.

The memory 4340 may include a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (eg, SD or XD memory), and a RAM. (RAM, Random Access Memory) SRAM (Static Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, magnetic disk , or at least one type of optical disk may include a storage medium.

The device 100 may be configured to use a web storage or a cloud server (not shown) that performs a storage function of the memory 4340 on the Internet. In this case, information about the GUI displayed on the display 4330 may be provided from the above-described web storage or cloud server.

In the device 100, not all of the components shown in FIG. 43 are essential components. The device 100 may include more components than those illustrated in FIG. 43 or may include fewer components than those illustrated in FIG. 43 . For example, the device 100 may be configured as shown in FIG. 44 .

Referring to FIG. 44 , the device 100 includes a user interface unit 4410 , a control unit 4420 , a sensor unit 4430 , a memory 4440 , an image processing unit 4450 , an audio output unit 4460 , and an audio input unit ( 4470 ), a camera 4480 , and a communication unit 4490 may be included.

The device 100 may further include a battery (not shown). The battery may be included in the device 100 as a built-in type or may be included in the device 100 in a detachable form. The battery may supply power to all components included in the device 100 . The device 100 may receive power from an external power supply. The external power supply may be a battery built into the external device 2120 .

The user input unit 4411 , the display 4412 , the control unit 4420 , and the memory 4440 included in the user interface unit 4410 shown in FIG. 44 include the user input unit 4310 and the control unit 4320 shown in FIG. 43 . ), display 4330 , and memory 4340 may be referred to as similar or identical components.

The sensor unit 4430 includes a proximity sensor 4431 that detects whether a user approaches the device 100 , an illuminance sensor 4432 (or a light sensor) that detects lighting around the device 100 , and the device 100 . A mood scope sensor 4433 for detecting the user's mood, a position sensor for detecting the position of the device 100 (Position Sensor, for example, a Global Positioning System (GPS) receiver) 4434, a device ( A gyroscope sensor 4435 for measuring the azimuth of 100), an accelerometer sensor 4436 for measuring the inclination and acceleration of the device 100 with respect to the ground surface, and the device 100. It may include a geomagnetic sensor 4437, a gesture sensor 4438 for recognizing a hand gesture by detecting infrared rays reflected on the palm, and/or a temperature/humidity sensor 4439 for recognizing ambient temperature and humidity. When the location sensor 4434 is configured as a GPS receiver, the location sensor 4434 may be referred to as being included in the communication unit 4490 . The gesture sensor 4438 may be referred to as a motion detection sensor that detects activity.

The sensor unit 4430 is not limited to that shown in FIG. 44 . For example, the sensor unit 4430 may include a gravity sensor, an altitude sensor, a chemical sensor (eg, an odor sensor), an atmospheric pressure sensor, a fine dust measurement sensor, an ultraviolet sensor, an ozone sensor, and a carbon dioxide sensor. a carbon (CO2) sensor, or/and a network sensor (eg, a network sensor based on WiFi, Bluetooth, 3G, Long Term Evolution (LTE), or Near Field Communication (NFC), etc.).

The sensor unit 4430 includes a pressure sensor (eg, a touch sensor, a piezoelectric sensor, a physical button, etc.), a status sensor (eg, an earphone terminal, a Digital Multimedia Broadcasting (DMB) antenna, a standard terminal (eg, A terminal that can recognize whether charging is in progress, a terminal that can recognize whether a PC (Personal Computer) is connected, a terminal that can recognize whether a dock is connected), a time sensor, or/and a health sensor (for example, a biosensor, a heartbeat sensor, a blood flow sensor, a diabetes sensor, a blood pressure sensor, a stress sensor, etc.), but the sensor that may be included in the sensor unit 4430 in the present disclosure is not limited thereto.

The result (or sensed value, or sensed signal) sensed by the sensor unit 4430 is transmitted to the controller 4420 .

The controller 4420 may generate an avatar-based image based on the sensed value received from the sensor 4430 and display it on the display 4412 . The controller 4420 may modify the avatar-based image displayed on the display 4412 according to a user input indicating a modification request received through the user input unit 4411 . The controller 4420 may update user information stored in the memory 4440 based on the modified avatar-based image. The controller 4410 may update user information stored in the external device 2120 connected through the communication unit 4490 based on the modified avatar-based image. As described above, the control unit 4420 may operate according to some embodiments, like the control unit 4320 of FIG. 43 .

The controller 4420 controls the overall operation of the device 100 . For example, the control unit 4420 executes programs stored in the memory 4440 , and thus the user interface unit 4410 , the sensor unit 4430 , the memory 4440 , the image processing unit 4450 , and the audio output unit 4460 . , the audio input unit 4470 , the camera 4480 , and/or the communication unit 4490 may be generally controlled.

The control unit 4420 is similar to the control unit 4320 of FIG. 43 in FIGS. 2, 7, 9, 11, 13, 14, 16, 18, 20, 22, 28, and FIG. 30, 33, 34, 35, 36, 37, 39, 40, 41, or 42 may perform an operation of the device 100 based on the flowchart shown in FIG.

A plurality of control units 4420 may be configured according to the operating environment of the device 100 . For example, the controller 4420 may include a processor operated in a low power mode of the device 100 and a processor operated in a normal mode of the device 100 . The controller 4420 may include a plurality of processors classified according to components included in the device 100 .

Programs stored in the memory 4440 may be classified into a plurality of modules according to their functions. For example, programs stored in the memory 4440 may be classified into a UI module 4441 , a notification module 4442 , and/or an application module 4443 , but in the present disclosure, programs stored in the memory 4440 . They are not limited to the above. For example, when the display 4412 is combined with a touch panel layer, the plurality of modules may include a touch screen module.

The UI module 4441 may provide GUI information for correcting user information mentioned in a preferred embodiment to the controller 4420 . The UI module 4441 may provide a UI, GUI, etc. specialized for each application installed in the device 100 to the controller 4420 .

The notification module 4442 may generate a signal notifying the device 100 of a modification request, an additional request, a modification completion, an object selection, a service information request, or a service selection, etc., but the notification generated by the notification module 4442 is described above. As long as it is not limited to

The notification module 4442 may output a notification signal in the form of a video signal through the display 4412 and may output a notification signal in the form of an audio signal through the audio output unit 4460, but is not limited thereto.

The application module 4443 may include at least one of a user information management application set in the device 100 and/or a service providing application based on an avatar-based image.

The user information management application according to some embodiments may be operated in connection with another application set in the device 100 . For example, when the question information is received in step S3702 of FIG. 37, the user information management application is set in the device 100 to provide answer information (eg, a search result) based on the question information received in step S3703 It may be operated in connection with a QA (Question & Answer) application.

The image processing unit 4450 processes image data received from the communication unit 4490 or stored in the memory 4440 to be displayed on the display 4412 . For example, FIGS. 1, 5A to 5E, 6A to 6E, 8A to 8C, 10, 12A to 12C, 15A to 15E, 17A to 17D, and 19A described above. , 19B, 23A to 23F, 27A to 27F, 29A to 29F, 31A to 31D, or 38 may display a GUI (Graphic User Interface) screen shown on the display 4412. Thus, the image processing unit 4450 may perform signal processing on the image data.

The audio output unit 4460 may output a sound according to user information according to some embodiments. The audio output unit 4460 may output audio data received from the communication unit 4490 or stored in the memory 4440 . The audio output unit 4460 may output an acoustic signal (eg, a notification sound) related to a function performed by the device 100 . The audio output unit 4460 may include, but is not limited to, a speaker, a buzzer, and the like.

The audio input unit 4470 may be configured as a microphone. The audio input unit 4470 may receive an audio signal input from the outside of the device 100 , convert the received audio signal into an electrical audio signal, and transmit it to the controller 4420 . The audio input unit 4470 may be configured to perform operations based on various noise removal algorithms for removing noise generated in the process of receiving an external sound signal. The audio input unit 4470 may recognize the user's voice.

The camera 4480 may obtain an image frame such as a still image or a moving image through an image sensor in a video call mode or a shooting mode. The camera 4480 may obtain an image of the user's finger gesture of the device 100 and transmit it to the controller 4480 . Based on the user's finger gesture image, the controller 4420 may include a user input indicating a request for modification of the avatar-based image, a user input indicating an additional request for an avatar-based image, a user input indicating completion of modification of the avatar-based image, and an avatar-based image based on the user's finger gesture image. It may be determined whether a user input indicating object selection from an image, a user input indicating a service information provision request based on the avatar-based image, and a user input indicating a service execution request based on the avatar-based image are received. In the present disclosure, the user's finger gesture image is not limited to being used to determine whether the above-described user input is received. The image captured through the image sensor may be processed through the controller 4420 or a separate image processing unit (not shown).

The image frame processed by the camera 4480 may be stored in the memory 4440 or transmitted to the outside through the communication unit 4490 . Two or more cameras 4480 may be provided according to the configuration of the device 100 .

The communication unit 4490 may include one or more components for communication between the device 100 and at least one external device 2120 . For example, the communication unit 4490 may include at least one of a short-range communicator 4491 , a mobile communicator 4492 , and/or a broadcast receiver 4493 . Components included in the communication unit 4490 in the present disclosure are not limited as described above. For example, the communication unit 4490 may include a component capable of performing at least one communication described in the network 2110 of FIG. 21 .

Short-range wireless communicator (short-range wireless communicator) 4491, Bluetooth communication, BLE (Bluetooth Low Energy) communication, short-range wireless communication (Near Field Communication unit, RFID), WLAN (Wi-Fi) communication, Zigbee (Zigbee) communication, Ant+ Communication, WFD (Wi-Fi Direct) communication, or UWB (ultra wideband) communication may be configured to include a module capable of at least one communication. In the present disclosure, the communication module included in the short-range communicator 4491 is not limited to the above. For example, the short-range communicator 4491 may include an infrared data association (IrDA) communication module. When receiving the beacon signal, the short-range communicator 4491 may use low-power Bluetooth communication.

The mobile communicator 4492 transmits/receives wireless signals to and from at least one of a base station, an external device, and/or a server over a mobile communication network. Here, the wireless signal may include various types of data according to transmission and reception of a voice call signal, a video call signal, or a text/multimedia message.

The broadcast receiver 4493 receives a broadcast signal and/or broadcast-related information from the outside through a broadcast channel. The broadcast channel may include, but is not limited to, at least one of a satellite channel, a terrestrial channel, and a radio channel.

The communication unit 4490 may transmit the avatar-based image generated by the device 100 and/or the modified avatar-based image to the external device 2120 according to some embodiments. The communication unit 4490 may receive at least one of user information, user modeling information, avatar-based image, service information, coordinate information for each object, and/or coupling condition information between objects from the external device 2120 according to some embodiments. can Information, data, and signals transmitted and received through the communication unit 4490 are not limited as described above.

45 is a detailed functional block diagram of the controllers 4320 and 4420 according to some embodiments. The following description will be based on the controller 4420 of FIG. 44 for convenience, but the following description may be applied to the controller 4320 of FIG. 43 .

Referring to FIG. 45 , the controller 4420 includes an information collection processor 4501 , a user modeling information generation processor 4502 , an image generation processor 4503 , an image display processor 4504 , an image modification processor 4505 , and a user modeling information modification processor 4506 .

The information collection processor 4501 may collect user information. The information collection processor 4501 may collect user information based on information received through the user input unit 4411 , the sensor unit 4430 , or the communication unit 4490 . User information is the same as mentioned at the beginning of the present disclosure. The user input received through the user input unit 4411 may be based on selection of an image-type object provided by the information collection processor 4501 , but the user input in the present disclosure is not limited as described above. The information collection processor 4501 may store the collected user information in the memory 4440 or transmit it to the external device 2120 through the communication unit 4490 .

The user modeling information generation processor 4502 generates user modeling information by using the user information collected by the information collection processor 4501 . The user modeling information may be configured as shown in FIG. 3 or FIG. 24A . The user modeling information in the present disclosure is not limited to the above. The user modeling information generation processor 4502 may store the generated user modeling information in the memory 4440 or transmit it to the external device 2120 through the communication unit 4490 .

The image generation processor 4503 generates the avatar-based image referred to in the present disclosure based on the user modeling information generated by the user modeling information generation processor 4502 . The avatar-based image may be said to be an image that represents the user. The image generation processor 4503 may store the generated avatar-based image in the memory 4440 or transmit it to the external device 2120 through the communication unit 4490 . The image stored in the memory 4440 or the external device 2120 may be temporarily stored.

The image display processor 4504 displays the avatar-based image generated by the image generation processor 4503 through the display 4412 .

The image modification processor 4505 modifies the avatar-based image based on the avatar-based image being displayed through the display 4412 according to some embodiments described above. When the modification of the avatar-based image is completed, the image modification processor 4505 may update the image stored in the memory 4440 or the image stored in the external device 2120 .

The user modeling information correction processor 4506 updates user information stored in the memory 4440 or the external device 2120 based on the image corrected by the image correction processor 4505 .

The controllers 4320 and 4420 illustrated in FIGS. 43 and 44 may include fewer or more components than those illustrated in FIG. 45 . For example, when receiving user information from the external device 2120 , the controllers 4320 and 4420 may not include the information collection processor 4501 .

When receiving user modeling information from the external device 2120, the control unit 4320, 4420 includes a user modeling information receiving processor instead of the information collection processor 4501 and the user modeling information generating processor 4502, user modeling Instead of the information modification processor 4506, a user modeling information transmission processor may be included. For example, when receiving an avatar character image based on user information from the external device 2120 , the controllers 4320 and 4420 may include an information collection processor 4501 , a user modeling information generation processor 4502 , and an image generation processor (4503) may include an image receiving processor instead.

As described above, the detailed functional block diagrams of the controllers 4320 and 4420 shown in FIGS. 43 and 44 are not limited to those shown in FIG. 45 . Also, the function of the external device 2120 may be determined based on the functional block diagram of the controllers 4320 and 4420 .

One or more programs including instructions to perform the method of implementing the method according to the embodiments of the present disclosure by a computer may be recorded as computer-readable codes in a computer-readable recording medium. The computer-readable recording medium includes all types of storage devices in which data readable by a computer system is stored. Examples of the computer-readable recording medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. In addition, the computer-readable recording medium may be distributed in network-connected computer systems, and may be stored and executed as computer-readable codes in a distributed manner.

So far, the present disclosure has been focused on preferred embodiments thereof. Those of ordinary skill in the art to which the present disclosure pertains will understand that the present disclosure may be implemented in a modified form without departing from the essential characteristics of the present disclosure. Therefore, the disclosed embodiments should be considered in an illustrative rather than a restrictive sense. The scope of the present disclosure is indicated in the claims rather than the foregoing description, and all differences within the scope equivalent thereto should be construed as being included in the present disclosure.

Claims (20)

In the device for managing user information,
a display that displays an avatar-based image;
a user input unit for receiving a user input;
controlling the display so that an avatar-based image according to user schedule information is displayed on the display;
modifying the avatar-based image according to the user schedule information when a user input indicating a request for modification of the avatar-based image according to the user schedule information is received through the user input unit;
a control unit for updating the user schedule information according to the modified avatar-based image;
The control unit is
controlling the display so that an avatar-based image according to the user's health information is displayed on the display;
When a user input indicating a medical information request based on an avatar-based image according to the user's health information is received through the user input unit, the display is controlled so that medical information related to the user's health information is displayed on the display, ,
The device further configured to update the user schedule information based on the selected medical information when information on medical information selected based on the displayed medical information is received through the user input unit. According to claim 1, wherein the control unit, when a user input requesting user information of a specific period is received through the user input unit, the display is configured to display a plurality of avatar-based images based on the user information of the specific period is further configured to control
The plurality of avatar-based images are determined according to a criterion for dividing the specific period. According to claim 1, wherein the control unit, when a user input requesting user information of a specific period is received through the user input unit, the display is configured to display a plurality of avatar-based images based on the user information of the specific period is further configured to control
The plurality of avatar-based images is determined according to a theme item included in the user information of the specific period. According to claim 1, wherein the control unit,
and when a user input requesting display of editable information is received through the user input unit, control the display to display an editable portion of the avatar-based image to be distinguished from a non-modifiable portion. According to claim 1, wherein the control unit,
If the provision period for the avatar-based image is set, check the provision period,
and when the provision period expires, destroy the avatar-based image. delete According to claim 1, wherein the control unit,
When a user input indicating addition of information to the avatar-based image is received through the user input unit, the display is controlled to display an input window for adding the information;
adding an image corresponding to the information received through the input window to the avatar-based image;
The device further configured to update the user schedule information according to the avatar-based image including the added image. The method of claim 1, wherein the control unit provides information related to the user's future hope information through the display as a user input requesting information related to the user's future hope information is received through the user input unit,
The device is further configured to update the user schedule information according to selected information among information related to the user's future wish information. The method of claim 1, wherein the control unit is configured to display a plurality of avatar-based images based on the usage history information in response to a user input requesting usage history information for each period of use of the device through the user input unit. A device further configured to control the display. The method of claim 1, wherein the controller controls the display to display an avatar-based image according to user information registered in social media,
In response to a user input requesting modification being received through the user input unit, the avatar-based image is modified according to user information registered in the social media,
The device further configured to update user information registered in the social media according to the modified avatar-based image. A user information management method executed by a computing device, comprising:
displaying an avatar-based image according to user schedule information on a display of the computing device;
modifying the avatar-based image according to the user schedule information according to a user input indicating a request for modification of the avatar-based image according to the user schedule information;
updating the user schedule information according to the modified avatar-based image;
displaying an avatar-based image according to the user's health information on the display;
displaying medical information related to the user's health information on the display when a user input indicating a medical information request based on an avatar-based image according to the user's health information is received; and
When information on medical information selected based on the displayed medical information is received, updating the user schedule information based on the selected medical information;
A method for managing user information, including. The method of claim 11, wherein the displaying of the avatar-based image according to the user schedule information comprises:
Displaying a plurality of avatar-based images according to user information of a specific period,
The plurality of avatar-based images are determined according to a criterion for dividing the specific period. The method of claim 11, wherein the displaying of the avatar-based image according to the user schedule information comprises:
Displaying a plurality of avatar-based images according to user information of a specific period,
The plurality of avatar-based images are determined according to a theme item included in the user information of the specific period. 12. The method of claim 11, wherein the method comprises:
and displaying an editable portion of the avatar-based image to be distinguished from a non-modifiable portion of the avatar-based image in response to a user input requesting display of the editable information. 12. The method of claim 11, wherein the method comprises:
checking the provision period when a provision period for the avatar-based image is set;
When the provision period expires, the step of destroying the avatar-based image
A method for managing user information further comprising. delete 12. The method of claim 11, wherein the method comprises:
displaying an input window for adding information on the display when a user input indicating addition of information on the avatar-based image is received;
adding an image corresponding to the information received through the input window to the avatar-based image; and
The user information management method further comprising the step of updating the user schedule information according to the avatar-based image including the added image. 12. The method of claim 11, wherein the method comprises:
displaying an avatar-based image according to the user's future wish information; and
providing information related to the user's future hope information in response to a user input requesting information related to the user's future hope information; and
The user information management method further comprising the step of updating the user schedule information according to information selected from the information related to the user's future desired information. 12. The method of claim 11, wherein the method comprises:
As a user input requesting usage history information for each usage period for the computing device is received
The user information management method further comprising the step of displaying a plurality of avatar-based images based on the usage history information. 12. The method of claim 11, wherein the method comprises:
displaying an avatar-based image according to user information registered in social media;
modifying an avatar-based image according to user information registered in the social media in response to a user input requesting modification; and
The user information management method further comprising the step of updating the user information registered in the social media according to the modified avatar-based image.

Images