KR102120936B1 - System for providing customized character doll including smart phone - Google Patents

Abstract

The present invention provides a system for providing a customized character doll including a smart phone. The system includes: a user terminal that registers a user using user information and uploads a picture that can identify the user; and a character doll production service providing server including: a registering unit that registers the user by storing user information from the user terminal; an upload unit to upload the picture from the user terminal; a selection unit that selects a 3D character that matches the user from pre-stored 3D character data by inputting the user information and image as a query; and an ordering unit that designates the selected 3D character as a user character and sends an order to create the 3D character doll to a production terminal.

Description

Character doll production service providing system equipped with the function of a smart terminal {SYSTEM FOR PROVIDING CUSTOMIZED CHARACTER DOLL INCLUDING SMART PHONE}

The present invention relates to a system for providing a character doll production service equipped with a function of a smart terminal, and provides a platform for producing a character doll including a smart terminal using a user's image.

Recently, in a simulation game, research results have been reported that users using the same character as themselves experience a higher level of spatial reality, social reality, immersion and empathy than using animation characters. In other words, you can feel a higher sense of realism in the game using real-life character games and animation characters that appear as the user himself in the game. Spatial realism tends to depend on the technical characteristics of the medium, and the technical realization of these media makes it the same as reality. When one's figure appears, the real feelings of users can be maximized. In addition, social realism, which is another sub-dimensionality of the established realism, also showed a high level in the live-action character to which one's figure was applied. Similar results also have a significant social presence that emphasizes interactivity.

At this time, the technology of combining the character doll and the smart terminal based on the IoT was researched and published. In this regard, in the prior art, Korean Patent Registration No. 10-1791942 (announced on November 1, 2017), the terminal and the driving unit are combined. Therefore, a mobile smart doll and a mobile smart doll driving system are provided to increase the applicability of control of a smart doll and diversify the appearance of the doll by using various display and sound output functions of the terminal. A drive unit that can move in response to a signal, a character unit that forms the exterior of the driving unit, a mounting terminal that is mounted on the driving unit and transmits a movement command signal to the driving unit, and a control terminal that communicates with the mounting terminal to transmit a predetermined command signal to the mounting terminal. It includes.

However, the above-described character is only a character preset by the manufacturer, and is not a character doll reflecting the user's character. As can be seen from the above-mentioned research results, when the user's own character is applied to virtual reality or reality, it is possible to further increase immersion as well as realism, while providing everyday functions, while constantly using the medium There is a need for a terminal production platform capable of inducing immersion through active selection acts in a manner, and inducing strong user interest and concentration by combining their actions and media reactions.

According to an embodiment of the present invention, a 3D character doll is generated as in real life using a user's information and a photo, a character doll that characterizes the information and a photo is generated, or a group that has the greatest similarity to the information and photo is stored. By providing a platform for selecting a character and making a request for a character doll terminal, and configuring the character doll terminal to perform the pet function of the character doll in addition to including the basic functions of the smart terminal, It is possible to provide a character doll production service providing system equipped with a function of a smart terminal capable of maximizing the user's presence and immersion. However, the technical problem to be achieved by the present embodiment is not limited to the technical problem as described above, and other technical problems may exist.

As a technical means for achieving the above-described technical problem, an embodiment of the present invention registers a user using user information, uploads a user identifiable picture, and stores user information from the user terminal. Registration section to register users, upload section to upload photos from user terminals, selection section to select 3D characters that match users from pre-stored 3D character data by entering user information and photos as queries, selected 3D characters And a character doll production service providing server including an ordering unit that designates a user character and transmits a 3D character doll production order to a production terminal.

According to any one of the above-described problem solving means of the present invention, the user's information and photos are used to create a 3D character doll as in real life, or a character doll is created by characterizing information and photos, and the information and photos are simulated. It provides a platform for making and requesting character puppet terminals by selecting pre-stored characters that show great similarity, and in addition to the basic features of the smart puppet terminal, the character puppet terminal performs the original pet function. By configuring it to be possible, it is possible to maximize the user's realism and immersion, and to increase the customer's emotional quality.

1 is a view for explaining a character doll production service providing system equipped with a function of a smart terminal according to an embodiment of the present invention.
FIG. 2 is a block diagram illustrating a character doll production service providing server included in the system of FIG. 1.
3 is a view for explaining an embodiment in which a character doll production service equipped with a function of a smart terminal according to an embodiment of the present invention is implemented.
4 is a view for explaining an embodiment in which a character doll production service equipped with a function of a smart terminal according to another embodiment of the present invention is implemented.
5 is an operation flowchart for explaining a method for providing a character doll production service equipped with a function of a smart terminal according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art to which the present invention pertains may easily practice. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. In addition, in order to clearly describe the present invention in the drawings, parts irrelevant to the description are omitted, and like reference numerals are assigned to similar parts throughout the specification.

Throughout the specification, when a part is "connected" to another part, this includes not only "directly connected" but also "electrically connected" with another element in between. . Also, when a part is said to "include" a certain component, it means that the component may further include other components, not exclude other components, unless specifically stated otherwise. However, it should be understood that the existence or addition possibilities of numbers, steps, actions, components, parts or combinations thereof are not excluded in advance.

The terms "about", "substantially", and the like used throughout the specification are used in or at a value close to the value when manufacturing and substance tolerances unique to the stated meaning are given, and the understanding of the present invention. To aid, accurate or absolute figures are used to prevent unscrupulous use of the disclosed disclosure by unscrupulous infringers. The term "~(step)" or "step of" as used in the entire specification of the present invention does not mean "step for".

In the present specification, the term “unit” includes a unit realized by hardware, a unit realized by software, and a unit realized by using both. Further, one unit may be realized using two or more hardware, and two or more units may be realized by one hardware.

In the present specification, some of the operations or functions described as performed by the terminal, the device, or the device may be performed instead on a server connected to the corresponding terminal, device, or device. Similarly, some of the operations or functions described as being performed by the server may be performed in a terminal, apparatus, or device connected to the corresponding server.

In this specification, some of the operations or functions described as mapping or matching with the terminal means that the unique number of the terminal or identification information of the individual, which is identification data of the terminal, is mapped or matched. Can be interpreted as

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 is a view for explaining a character doll production service providing system equipped with a function of a smart terminal according to an embodiment of the present invention. Referring to FIG. 1, a character doll production service providing system 1 equipped with a function of a smart terminal includes at least one user terminal 100, a character doll production service providing server 300, and at least one production terminal 400. ), and at least one character doll terminal 500. However, since the character doll production service providing system 1 equipped with the function of the smart terminal of FIG. 1 is only an embodiment of the present invention, the present invention is not limitedly interpreted through FIG. 1.

At this time, each component of FIG. 1 is generally connected through a network (network, 200). For example, as illustrated in FIG. 1, at least one user terminal 100 may be connected to the character doll production service providing server 300 through the network 200. In addition, the character doll production service providing server 300 may be connected to at least one user terminal 100, at least one production terminal 400, and at least one character doll terminal 500 through the network 200. . Also, the at least one production terminal 400 may be connected to the character doll production service providing server 300 through the network 200. Also, the at least one character doll terminal 500 may be connected to at least one user terminal 100 and the character doll production service providing server 300 through the network 200.

Here, the network means a connection structure capable of exchanging information between each node such as a plurality of terminals and servers, and examples of such a network include RF, 3GPP (3rd Generation Partnership Project) network, and LTE (Long Term) Evolution (Evolution) network, 5GPP (5th Generation Partnership Project) network, WIMAX (World Interoperability for Microwave Access) network, Internet (Internet), Local Area Network (LAN), Wireless Local Area Network (LAN), Wide Area Network (WAN) , PAN (Personal Area Network), Bluetooth (Bluetooth) network, NFC network, satellite broadcasting network, analog broadcasting network, DMB (Digital Multimedia Broadcasting) network, and the like.

In the following, the term at least one is defined as a term including singular and plural, and even if the term at least one does not exist, each component may exist in singular or plural, and may mean singular or plural. It will be self-evident. In addition, it may be said that each component is provided in a singular or plural form, depending on the embodiment.

The at least one user terminal 100 uses user information and user pictures to produce the character doll terminal 500 using a web page, app page, program, or application related to a character doll production service equipped with a function of a smart terminal. It may be a terminal that registers a user by registering and requests production. In this case, the at least one user terminal 100 may be a terminal that receives a matching result of a user and a character from the character doll production service providing server 300 and confirms the received result. And, the at least one user terminal 100, the character doll terminal 500 is completed, and the character doll terminal 500 to perform synchronization with the user terminal 100, or the character doll terminal 500 In order to use the user terminal 100 as a substitute, it may be a terminal that moves all data of the user terminal 100 to the character doll terminal 500.

Here, the at least one user terminal 100 may be implemented as a computer capable of accessing a remote server or terminal through a network. Here, the computer may include, for example, navigation, a laptop equipped with a web browser (WEB Browser), a desktop, a laptop, and the like. At this time, the at least one user terminal 100 may be implemented as a terminal capable of accessing a remote server or terminal through a network. The at least one user terminal 100 is, for example, a wireless communication device in which portability and mobility are guaranteed, navigation, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), Personal Handyphone System (PHS), Personal Digital Assistant (PDA), International Mobile Telecommunication (IMT)-2000, Code Division Multiple Access (CDMA)-2000, W-Code Division Multiple Access (W-CDMA), Wireless Broadband Internet ) It may include all kinds of handheld-based wireless communication devices such as a terminal, a smartphone, a smartpad, and a tablet PC.

The character doll production service providing server 300 may be a server that provides a character doll production service web page, app page, program, or application equipped with a function of a smart terminal. In addition, the character doll production service providing server 300 may be a server in which images, characteristic data, and the like of at least one character are built into a database. In addition, the character doll production service providing server 300, when user information and a user's photo are registered from the user terminal 100, and a production request of the character doll terminal 500 is received, user information in a pre-built database And a list with the highest similarity to the picture is transmitted to the user terminal 100 by listing up the matching list, or selected by the user terminal 100, or by selecting a character with the highest similarity to the user terminal 100 for a confirmation request. It can be a sending server. In addition, the character doll production service providing server 300 transmits a production order to at least one production terminal 400 when a character is determined, and transmits a production situation received from the production terminal 400 to the user terminal 100 It can be a server sharing with. In addition, the character doll production service providing server 300 provides a paying or synchronization interface between the character doll terminal 500 and the user terminal 100, driving the character doll itself, driving the character doll terminal 500, and the like. If an update or patch exists, it may be a server that publishes it.

Here, the character doll production service providing server 300 may be implemented as a computer capable of accessing a remote server or terminal through a network. Here, the computer may include, for example, navigation, a laptop equipped with a web browser (WEB Browser), a desktop, a laptop, and the like.

At least one production terminal 400 receives a production request or an order form from the character doll production service providing server 300 using a web page, an app page, a program or an application related to a character doll production service equipped with a function of a smart terminal. It may be a terminal. Here, the at least one production terminal 400 transmits the 3D drawing or 2D drawing of the character doll production for the character doll terminal 500 to the user terminal 100 through the character doll production service providing server 300 and finally If a request for confirmation or a modification exists, it may be a terminal received from the user terminal 100 through the character doll production service providing server 300. Also, at least one production terminal 400 may be a terminal that shares the tracking number to the user terminal 100 through the character doll production service providing server 300 when it is sent.

Here, the at least one production terminal 400 may be implemented as a computer capable of accessing a remote server or terminal through a network. Here, the computer may include, for example, navigation, a laptop equipped with a web browser (WEB Browser), a desktop, a laptop, and the like. At this time, the at least one production terminal 400 may be implemented as a terminal capable of accessing a remote server or terminal through a network. The at least one production terminal 400 is, for example, a wireless communication device in which portability and mobility are guaranteed, navigation, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), Personal Handyphone System (PHS), Personal Digital Assistant (PDA), International Mobile Telecommunication (IMT)-2000, Code Division Multiple Access (CDMA)-2000, W-Code Division Multiple Access (W-CDMA), Wireless Broadband Internet ) It may include all kinds of handheld-based wireless communication devices such as a terminal, a smartphone, a smartpad, and a tablet PC.

The at least one character doll terminal 500 may be a smart terminal using a web page, an app page, a program or an application related to a character doll production service equipped with a function of a smart terminal. At this time, the at least one character doll terminal 500 may be a terminal further comprising a control device for moving at least one part of the character doll. Further, since at least one character doll terminal 500 is a smart terminal, it may be a terminal including the basic configuration of the smart terminal.

Here, the at least one character doll terminal 500 may be implemented as a computer that can access a remote server or terminal through a network. Here, the computer may include, for example, navigation, a laptop equipped with a web browser (WEB Browser), a desktop, a laptop, and the like. At this time, the at least one character doll terminal 500 may be implemented as a terminal capable of accessing a remote server or terminal through a network. The at least one character puppet terminal 500 is, for example, a wireless communication device in which portability and mobility are guaranteed. Navigation, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular) , PHS(Personal Handyphone System), PDA(Personal Digital Assistant), IMT(International Mobile Telecommunication)-2000, CDMA(Code Division Multiple Access)-2000, W-CDMA(W-Code Division Multiple Access), Wibro(Wireless Broadband Internet) terminal, a smart phone (smartphone), a smart pad (smartpad), a tablet PC (Tablet PC) and the like may include all kinds of handheld (Handheld)-based wireless communication device.

FIG. 2 is a block diagram for describing a character doll production service providing server included in the system of FIG. 1, and FIG. 3 is a character doll production service equipped with a function of a smart terminal according to an embodiment of the present invention. FIG. 4 is a view for explaining an embodiment, and FIG. 4 is a view for explaining an embodiment in which a character doll production service equipped with a function of a smart terminal according to another embodiment of the present invention is implemented.

Referring to FIG. 2, the character doll production service providing server 300 may include an upload unit 310, a selection unit 320, and an ordering unit 330.

Character doll production service providing server 300 according to an embodiment of the present invention or another server (not shown) operating in conjunction with at least one user terminal 100, at least one production terminal 400, and at least one When transmitting a character doll production service application, program, app page, web page, etc. equipped with the function of a smart terminal to the character doll terminal 500 of at least one user terminal 100, at least one production terminal 400 ), and the at least one character doll terminal 500 may install or open a character doll production service application, a program, an app page, a web page, etc. equipped with a function of a smart terminal. In addition, the service program may be driven in at least one user terminal 100, at least one production terminal 400, and at least one character doll terminal 500 using a script executed in a web browser. Here, the web browser is a program that enables the use of the world wide web (WWW) service, and refers to a program that receives and displays hypertext described in a hypertext mark-up language (HTML), for example, Netscape. , Explorer, Chrome, and the like. Further, the application means an application on the terminal, and includes, for example, an app that is executed on a mobile terminal (smartphone).

Referring to Figure 2,

The registration unit 310 may register a user by storing user information from the user terminal 100.

The upload unit 320 may upload a photo from the user terminal 100. To this end, the user terminal 100 may register a user using the user information and upload a picture that identifies the user. At this time, the character doll according to an embodiment of the present invention may be interpreted in two meanings. The first may mean a doll that characterizes a user picture, and the second may mean a character doll that already exists to match user information and a picture as much as possible. In the former case, it is divided into two methods: making a 3D character doll in real life according to the user's photo, or making a doll by characterizing the user's photo. In the latter case, it will be described later.

The former case will be described. First, a 3D character can be generated only by performing a 3D modeling method from a 2D face of a picture uploaded by a user. In general, the most accurate method of generating a face is a method using a laser scanner, which is a device capable of obtaining 3D data, but this requires a large amount of data to be processed and expensive equipment. Accordingly, in one embodiment of the present invention, a method using a parameter may be used, which can generate a face model using a small number of control points, and if a number of parameters is increased, a method of synthesizing various faces may be used. have. Alternatively, in an embodiment of the present invention, a method of obtaining a parameter for generating a 3D face by receiving a user's picture and obtaining a parameter, and replacing the face of the character doll with the resulting 3D face may be used.

To this end, a facial feature point is selected as a control point using AAM (Active Appearance Models) in a two-dimensional front face image, and a 3D average face can be transformed through a parameter update function to complete the 3D face of the input image. . Control points for face modeling can express eyebrows, eyes, noses, mouths, and jawlines by adding feature points for more detailed expression based on feature points defined in MPEG-4 (MPEG-4 Facial Animation). At this time, in order to control the 3D face, it is necessary to calculate the control points in the X, Y, and Z axis directions. In an embodiment of the present invention, PCA (Principal Component Analysis) may be used to reduce the amount of calculation, and input images (uploaded pictures) ), you can create a parameter update function through learning to modify the control point. The parameter update function defines a relationship between an error value between a 3D average face projected in 2D and a control point of a 2D image and a PCA coefficient value in a 3D image. That is, when a 2D input image is received, a 3D control parameter can be calculated using this function and a 3D average face, and a 3D face can be generated using the 3D control parameter. At this time, for learning data and evaluation data, face pictures of a plurality of users may be separately used as input images.

In the embodiment of making a doll by characterizing a user picture, a method of characterizing or animating the user's picture may be used. First, the face must be recognized in the user's picture, and a PCA method may be used among the face recognition algorithms. PCA, which is one of the appearance-based methods, is a statistical method that makes it easy to handle by reducing a high-level signal that is difficult to handle to a low level by treating the component as a main component rather than individually separating it from the face. Comparing similarities by applying the PCA algorithm is effective when the face is recognized as a whole. At this time, in one embodiment of the present invention, it is more computationally efficient to compare the characters to be partially compared to the overall similarity. Here, the similarity comparison algorithm that can be used can be applied to the PCA-SIFT algorithm with the SIFT algorithm applied to the PCA, which is a little more complementary than the PCA. As a reduction method, it has the advantage of having a high computational speed and high accuracy. Furthermore, in order to increase the simple recognition rate for character creation, a face is detected through a Haar-like feature and a pixel is compared 1:1 based on the most basic L2-Norm theory of similarity comparison (Pixels Matching, 1 :1 PM), but is not limited thereto.

1:1 PM refers to the process of finding the sum of the pixel value differences at the same location of the face image data to be compared with the image pixel values in the database and finding the smallest value. At this time, an open operation may be used to extract an image such as an eye or mouth extracted from the original image as a clean binary image, and the image in the database should also be a black and white image for comparison. When the eye is detected through Haar, since the starting positions of the eyes are all different, and the total size of the detected images may be different, it is possible to perform a task of changing the relative size to an absolute size to solve this. It is possible to determine the starting position of the eye by detecting the feature point and cut the ratio of the region of interest based on the point to 4:3 to set a consistent ratio, but the ratio is not limited to the above. And, by detecting the feature point, the operation of changing the position of the region of interest can be performed so that the actual eye position is in the middle, and the most important resizing operation can be performed at 1:1 PM. Calculating the difference between the distance between the start and end positions of the eye, found by the detection of the feature points that were previously performed, can be compared to the length of the image eyes in the database, so the difference can be seen. At this time, the eye image detected based on the image size of the database may be small or large. The difference in image size can be set as a scale constant so that different operations are performed according to the two results. At this time, if the resized image is smaller than the image in the database, blending (compositing images of different sizes) is performed to match the size of the database image. In the opposite case, the image is cropped to match the size of the database image according to the position where the eye was detected. In order to compare the completed image with the original image, the pixel values are loaded into a matrix, and the values obtained after the sqrt operation are added. The image with the lowest similarity among the character eye images in the database has the highest similarity. to be. That is, after comparing the pixel values obtained by binarizing the detected eye image with the pixel values of the eye in the database, and adding all the extracted values of the pixel values at the matching positions, the image having the smallest value has the highest similarity. Judge. Then, the database having the highest similarity can be selected and the image of the character can be synthesized to the existing character. Also, similarity analysis and matching of personality may be further added. Of course, it will be apparent that various methods can be used in addition to the above-described method.

The latter case will be explained. The selection unit 320 may input a user information and a picture as a query to select a 3D character matching the user from the 3D character data previously stored. First, the selection unit 320 may collect character image data necessary for learning, classification, and search by using iCralwer, a kind of web crawler. For example, there are a lot of useless images due to the iCralwer feature, which uses Google to retrieve images by entering search keywords. If you simply collect images and perform non-supervised clustering, there is a high possibility of performing inefficient clustering. Rather, the method of storing image files collected by crawling for each image character on the web in a folder with the name of each character is collected, and naturally labeled data is collected during this process, and at the same time, it is possible to learn more easily by using this point. Make it possible.

Next, character search and classification work is performed using deep learning. In order to find the most suitable neural network, it is necessary to select a suitable neural network model. For example, Deep Residual Learning (ResNet) 4 may be used. Here, the performance of ResNet is not only superior to other models, but also shows a figure that exceeds the error of human cognitive ability, so it may be appropriate for the factors in which human cognition and perception are revealed and expressed. In addition, for characters, ResNet, which has a deep layer using the Residual Module structurally, is more advantageous for extracting these features because several features are well-recognized in the image. Then, data pre-processing is performed, and a character can be searched using a user's picture based on ResNet, and classification for character search must be performed on an input image (user picture) using ResNet. Unlike other CNN models, ResNet is characterized by higher performance through a large number of layers. However, contrary to the tendency of performance improvement when many layers are stacked, too many layers of a certain level or more will lower performance. Residual connections can be made because even the test data, as well as the training data, may deteriorate. Of course, in addition to the above-described method, since there may be a method of classifying and finding a character in various ways, it will be apparent that one embodiment of the present invention is not limited to those described above.

The ordering unit 330 may designate the selected 3D character as a user character and transmit a 3D character doll production order to the production terminal 400. At this time, the production terminal 400 may share the result of creating a character doll to correspond to the modeled or selected character in the former or latter method described above with the user terminal 100, and a modification instruction of the user terminal 100 Or you can get a confirmation.

On the other hand, the character doll terminal 500 produced by the producer of the production terminal 400 is paired with the user terminal 100 to store data of the user terminal 100, CS (Circuit Switched) network, PS (Packet Switched) ) It is connected to network and LTE network to transmit and receive data. That is, it is configured to include all the basic functions of the smart terminal. At this time, depending on whether the user uses the character doll terminal 500 with the user terminal 100 or the character doll terminal 500 as a substitute for the user terminal 100, functions such as interlocking or pairing are different. That is, in the former case, while using the user terminal 100, the character doll terminal 500 is used only when receiving a call at home or used as a plurality of mobile terminals, and when a call is received, the user terminal 100 In addition, since the character doll terminal 500 is also paired, the user terminal 100 may perform a role such as ringing a telephone. It is one phone number, but it is the same as two terminals connected. In the latter case, since the user terminal 100 will not be used, it refers to a case in which the identities of the user terminal 100 are all transferred to the character doll terminal 500. At this time, when making a call or sending a message to the user's phone number, only the character doll terminal 500 operates. At this time, since the operation or function of transferring data of the user terminal 100 is a known technique, detailed description will be omitted.

In addition, the character puppet terminal 500 may convert the caller voice of the calling terminal to a voice previously stored in the character puppet terminal 500 when the call reception event occurs to establish a call connection with the call terminal. have. When the voice of the character doll itself exists, the voice of the caller can be converted into the voice of the character. To this end, a data-driven speech synthesis method can be used. The first is a non-parametric, example-based method known as concatenative speech synthesis. And the second is a parametric, model-based approach known as statistical parametric speech synthesis. The editing synthesis is performed in a voice waveform step without any parameterization process by collecting desired voice units from a large number of pre-recorded voice databases and synthesizing the voices, so the sound quality is excellent. The statistical parameter-based synthesis method synthesizes speech by outputting vocoder parameters from a pre-trained generative model in a statistical manner. In the process of converting the linguistic feature output from the text analysis to an acoustic feature, an acoustic model is used to perform speech feature prediction. At this time, the acoustic model is trained by a statistical technique based on a probability model from a large number of speech databases, and the training framework may be a hidden markov model or a deep neural network. Since the Hidden Markov Model uses HTK (Hidden Markov Model Toolkit), it is known as HTS with the meaning of Synthesis. Basically, it is a voice parameter using vocoder. And a new voice can be output through audible and cognitively meaningful correction. Voices can be mixed by interpolation between previously trained acoustic models, and attempts can be made to mimic the voice through adaptive techniques when adaptation data is given. At this time, although the statistical parameter-based method is inferior to the non-parametric method in terms of sound quality, it is possible to create various voices, various emotions, expressive voices, etc. through free transformation of sound models or voice parameters. It can also be used in a conversation agent to be described later.

The character doll terminal 500 may operate at least one part of the character doll terminal 500 to correspond to the caller voice of the calling terminal when a call reception event occurs and a call connection is made with the calling terminal. . At this time, when operating at least one part of the character doll in the character doll terminal 500, it is necessary to first understand the utterance of the caller. Operating any part without knowing its meaning can cause discomfort due to actions that do not fit the situation or context. To this end, text-to-speech (TTS) is required, which refers to a technology that converts incoming speech, text, or a string of text into a speech signal that produces a natural speech. Here, the TTS is composed of a text analysis unit at the front end and a speech synthesis unit at the rear end, and the text analysis unit generally includes sentence segmentation, word segmentation, text normalization, and part-of-speech tagging. A number of natural language processing (NLP) techniques such as of-speech tagging and grapheme-to-phoneme conversion can be used. That is, the text analysis part receives a series of word strings and outputs phoneme strings having various language contexts. Next, the speech synthesis unit receives a phoneme string containing language context information output as an input and outputs a synthesized speech waveform, which generally includes rhyme prediction and speech waveform generation, and through the series of processes described above. The speaker's intention, situation, context, etc. can be grasped, and in response, the character doll terminal 500 may perform an operation such as moving any one part or changing an expression.

Accordingly, the character doll terminal 500 may operate at least one part of the character doll terminal 500 when at least one event occurs. At this time, at least one event and the operation of each part may be pre-mapped and stored. Here, since each event is pre-programmed and predictable since it is generated in the smart terminal, implementation of the above-described process is not difficult. The problem is when content, context, and situation included in one event are different. For example, the character event is the same, but the character dolls have the same expression when the content contained within the character is different from "I'm promoted" in the first character and "My mother-in-law is dead" in the second character. If you build or perform the same movement, the user is not satisfied with the emotional quality. Accordingly, the character puppet terminal 500 may perform natural language processing, such as word embedding, LSTM (Long Short Term Memory), seqtoSeq (Sequence-to-Sequence), and attention mechanism (Attention Mechanism). It may be further performed, and emotional analysis (Opinion Mining) may be further performed. In the text that humans write, there will be a topic that is the main subject of the article, and there is an author's opinion on the topic. The task of finding the subject that is the subject of an article is called topic modeling, and this technique provides a way to automatically calculate what subject the text is dealing with. Likewise, grasping the writer's opinion on the subject is called Opinion Mining or Sentiment Classification. By using this, it is possible to determine which article reflects the subjective opinion of the author, whether it is an objective article, or if it is subjective, positive or negative.

In addition, the character doll terminal 500 outputs a feeding dose alarm in a predetermined cycle or in real time, accumulates a feeding dose event when a feeding dose event is input to correspond to the feeding dose alarm, and responds to the accumulated feeding dose event. Thus, the growth level can be increased. For example, the character doll terminal 500 basically has a function of a smart terminal, but may have characteristics of the character doll itself. In other words, if you do not perform a management action such as feeding on time, washing a bath, walking, etc., as if you are managing a pet by applying the function of a pet, give an alarm for this or keep indifferent If the management operation is not input, the pet can be set to die, and a penalty of failing to perform the basic functions of the smart terminal may be given.

The character doll terminal 500 includes an artificial intelligence conversation agent, and the artificial intelligence conversation agent recognizes the user's speech and converts it into machine language through natural language processing, generates an answer to the converted machine language, and then creates a character doll terminal It is controlled to utter an answer using a character voice previously stored in the 500, and the character doll terminal 500 may control at least one part of the character doll terminal 500 to correspond to the content of the answer. At this time, the chat agent (Chatting Agent, or chatbot) is a platform that exchanges information between humans and machines through voice or text, and was created to show the communication superficiality between humans and machines, and uses pattern matching and alternative methodology. Conversation can be made, and automatic speech recognition technology that converts human speech into text based on human conversation, a dialogue controller for manipulating interaction with humans, and a platform for providing information to humans Can be. In this case, the conversation agent may be configured to physically analyze the user's query using the artificial intelligence algorithm in the doll production service providing server 300 and the user may be provided with information through an app or the web. In the artificial intelligence conversation service providing server 300, technologies such as natural language processing (NLP), situational awareness, and big data analysis may be used to provide an appropriate answer to a user's query. Further, a distributed computing technology such as hadoop, which processes large-scale message traffic, may be further used between the doll production service providing server 300 and the user. In addition, the conversation agent can use a platform to which an ontology-based knowledge base is applied, not a simple match (pattern match, rule-based)-based, natural language processing (NLP) and ontology-based search You can use techniques to process commands.

In addition, the AI character will act according to the user's choice and the vast amount of pre-defined decision trees. Here, context is all information used to specify a situation, and context awareness is adopted according to changes in objects and objects over time as well as nearby people and objects and places used. Also, according to the behavioral model of cognitive psychology, understanding and perceiving the world is possible only through subjective perception. In other words, context recognition according to the behavioral model proceeds not only through people, objects, and places, but also changes in objects over time through selection based on a subjective viewpoint. Therefore, the character-based interactive AI according to an embodiment of the present invention is defined as an interactive AI capable of understanding the context of a conversation through performing continuous conversation and expressing subjective opinions and emotions based on this.

At this time, continuous conversation can be made by learning the response sentence for the input sentence, such as interactive artificial intelligence using machine learning, or by adding up to one sentence immediately before the conversation to the above method through the Generative Hierachical Neural Network Models. It might be. It is also possible to introduce multiple agents for character-based interactive AI. For example, it is divided into recognition, decision making, planning/behavior, etc., and each agent is further subdivided into a knowledge base and an inference engine. Therefore, in a platform composed of a plurality of agents, the conversation of the other party can be read, grasped, and the overall conversation situation in consideration of one's emotional state is recognized, a response strategy is devised, and an execution plan is established and executed. .

At this time, the target language analysis agent is an agent that analyzes the meaning of the target and the emotion of the target through sentences received from the other party. The agent grasps the meaning and emotions of the other party through fragment analysis and in-depth analysis from the feedback of the decision-making agent and the other party's remarks, and delivers it to the conversation situation determination agent. Fragment analysis refers to semantic and emotional analysis through sentences currently received from the other party. In-depth analysis is an analysis module for inferring an opponent's actual intention and emotion by accumulating the opponent's intention and emotion updated through feedback received from the situational awareness agent. In-depth analysis is composed of only the value of fragment analysis at the beginning of the dialogue, so it reflects the preconceptions about the object, and is updated according to the intention and emotion of the actual object as the conversation progresses.

The self-emotion grasping agent is an agent for modeling the change in emotions of artificial intelligence itself through sentences received from the other party. Like the target language analysis agent, this agent also grasps the emotion of the artificial intelligence from the other person's remarks through fragment analysis and in-depth analysis, and delivers it to the conversation situation judgment agent. Modeling through a fragmented reaction is an instantaneous emotional change that occurs through a strong reaction of a certain level or more. This affects the sudden change of emotion caused by strong stimulation or the expression of instant emotion. Modeling through an in-depth response models emotional changes that occur and accumulate from a situation in which a conversation with the other party is in progress.

The dialogue situation determination agent grasps the context of the conversation through the meaning and emotion state received from the target language analysis agent and the self-emotion identification agent, and the feedback from the decision agent, and delivers it to the decision agent. In addition, it provides feedback for the situation identified in the target language analysis agent and the self-sensibility identification agent for updating of each deep analysis module.

The decision-making agent is an agent that decides the intention of the next remark, including the purpose of the utterance of artificial intelligence, in the situation identified through the dialogue situation judgment agent. The decision-making agent provides feedback to the inference engine of the target language analysis agent and the dialogue situation determination agent to achieve a subjective interpretation of the given situation. In order to achieve the purpose of artificial intelligence, the decision is made through simulation through the emotion model created by the target language analysis agent and the self-emotion identification agent.

The response generation agent is an agent that generates the intention of the next speech at the level of the sentence determined through the decision-making agent. The sentence expression agent is an agent that transforms the result sentence of the response generation agent to match the grammar and a specific tone (spoken dialect, etc.). Accordingly, when all characters, personalities, and personalities for each relationship are defined and agents are defined so that interaction is possible, a conversation agent can be created that simulates the personality of an object that has a relationship with the user, and accordingly, the user is a character doll. It is possible to chat with a conversation agent of the terminal 500.

Hereinafter, the operation process according to the configuration of the character doll production service providing server of FIG. 2 will be described in detail with reference to FIGS. 3 and 4 as an example. However, the embodiment is only one of various embodiments of the present invention, it will be apparent that it is not limited thereto.

Referring to FIG. 3, (a) character doll production service providing server 300 uploads user information and photos from user terminal 100, and (b) performs 3D modeling from photos to make production requests or extract them as photos A match request can be made by selecting the final character by matching the character that most closely matches the personality of the face image and user information. In this process, (c) the character doll production service providing server 300 may request confirmation to the user terminal 100, and (d) send an order request to at least one production terminal 400.

Referring to FIG. 4, (a) the character doll terminal 500 may be paired with the user terminal 100 or transfer data of the user terminal 100, and (b) when a call is received from the calling terminal, the caller It is also possible to output the speech by converting it to speech, and (c) if a pet function is further included, the management request can be output as an alarm periodically or in real time, and (d) each character doll is analyzed and output. The site may be controlled.

The method for providing a character doll production service equipped with the functions of the smart terminals of FIGS. 2 to 4 is not described above. Since the contents can be easily inferred from the same or described contents, the following description will be omitted.

5 is a flowchart illustrating a method for providing a character doll production service equipped with a function of a smart terminal according to an embodiment of the present invention. 5, the character doll production service providing server stores user information from a user terminal to register a user (S5100), uploads a photo from the user terminal (S5200), and queries user information and pictures (Query) 3D character matching with the user is selected from the previously stored 3D character data by inputting (S5300), the selected 3D character is designated as a user character, and a 3D character doll production order is transmitted to the production terminal (S5400).

The order between the above-described steps (S5100 to S5400) is only an example, and is not limited thereto. That is, the order between the above-described steps (S5100 ~ S5400) may be mutually variable, some of which may be executed or deleted simultaneously.

Such a method for providing a character doll production service equipped with the function of the smart terminal of FIG. 5 is not described above. A method of providing a character doll production service equipped with the function of the smart terminal through FIGS. 1 to 4 has been described above. Since the contents can be easily inferred from the same or described contents, the following description will be omitted.

A method for providing a character doll production service equipped with a function of a smart terminal according to an embodiment described with reference to FIG. 5 is a form of a recording medium including instructions executable by a computer, such as an application or program module executed by a computer Can also be implemented as Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. Also, the computer-readable medium may include any computer storage medium. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

The method for providing a character doll production service equipped with a function of a smart terminal according to an embodiment of the present invention described above may include an application installed in the terminal (this may include a program included in a platform or an operating system basically installed in the terminal). It may be executed by an application (i.e., a program) installed directly on the master terminal through an application providing server such as an application store server, an application, or a web server related to a corresponding service. In this sense, the method for providing a character doll production service equipped with a function of a smart terminal according to an embodiment of the present invention described above is implemented as an application (that is, a program) basically installed in a terminal or directly installed by a user, and is provided in the terminal. It may be recorded on a computer readable recording medium.

The above description of the present invention is for illustration only, and a person having ordinary knowledge in the technical field to which the present invention pertains can understand that it can be easily modified to other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the above detailed description, and it should be interpreted that all changes or modifications derived from the meaning and scope of the claims and equivalent concepts thereof are included in the scope of the present invention. do.

Claims (7)

A user terminal that registers a user using user information and uploads a picture that identifies the user;
The registration unit that registers a user by storing the user information from the user terminal, an upload unit that uploads the photo from the user terminal, and inputs the user information and a picture as a query to query the user from among 3D character data previously stored. A character doll production service providing server including a selecting unit for selecting a matching 3D character, and an ordering unit for designating the selected 3D character as a user character and transmitting a 3D character doll production order to a production terminal; And
Includes a character doll terminal that is paired with the user terminal to store data of the user terminal, and is connected to a CS (Circuit Switched) network, a PS (Packet Switched) network, and an LTE network to transmit and receive data.
The character doll terminal converts the caller voice of the calling terminal into a voice previously stored in the character doll terminal and outputs it when a call reception event occurs and a call connection is made with the calling terminal,
The character doll terminal, when the call (Call) receiving event occurs and the call connection with the calling terminal, characterized in that to operate at least one part of the character doll terminal to correspond to the caller voice of the calling terminal Character doll production service providing system equipped with the function of a smart terminal.
delete delete delete delete delete According to claim 1,
The character doll terminal outputs a feeding administration alarm in a predetermined cycle or in real time, accumulates the feeding administration event when a feeding administration event is input to correspond to the feeding administration alarm, and responds to the accumulated feeding administration event. Growth level is increased,
The character doll terminal includes an artificial intelligence conversation agent,
The AI conversation agent recognizes the user's speech, converts it into machine language through natural language processing, generates an answer to the converted machine language, and controls to utter the answer using a character voice previously stored in the character doll terminal. Become,
The character doll terminal is a character doll production service providing system equipped with a function of a smart terminal characterized in that it controls at least one part of the character doll terminal to correspond to the content of the answer.

Images