KR20200032419A - Recording Midium - Google Patents

Abstract

According to an embodiment of the present invention, a recording medium storing a message service providing program comprises the functions of: providing a message interface to a user terminal, processing a message transmitted and received through the message interface, and providing one or more expression image items which can be included in the message to the user terminal through the interface; storing the one or more expression image items; classifying and managing the one or more expression image items stored in a database according to element information; and processing machine learning of the expression image item database based on classification and management information of a visual item management unit to generate an expression image item learning model and generating, by using the learning model, a combined-image item obtained by selecting and combining first and second expression image items stored in the database. Therefore, the present invention can provide the recording medium storing the program for processing a message including combined expression image items capable of providing emoji or emoticons reflecting maximized diversity and feedback.

Description

Recording medium {Recording Midium}

The present invention relates to a recording medium. More specifically, the present invention relates to a recording medium recording a program for processing a message providing a combined image item for expressing.

In a message service such as a chat service, various types of expression methods such as emoticons and emojis are used to express emotions of users.

The emoticon is a symbol used to express the emotions of service users in the chat service. It is a combination of emotion and icon. The initial emoticon was created by a simple combination of various symbols and characters on the keyboard. In recent years, it has developed into a form that expresses the user's emotion through various types of character images.

In addition, unlike the emoji method based on the combination of characters, the method of outputting a pictorial character embedded in the font form of the platform system, a method of outputting an emotion expression image corresponding to the input emoji code is illustrated.

As the demand of service users for these emojis and emoticons increases, it has become common to designate and provide various types of pseudo-expression image items to service users in message services.

However, the existing pseudo-expression image items such as emojis and emoticons must be randomly generated by the designer for images matching each pseudo-expression, and matched with the pseudo-expression to be generated and stored on the platform system. There is an inconvenience that can be entered on a message only when it is called and selected.

This is because in the current environment where the demand for emoji and emoticons is constantly increasing, the design cost increases in proportion to the number of generations, the system construction cost increases by platform update, and unnecessary images increase when the user inputs. This can cause problems that take a long time or multiple unwanted images appear.

Due to these limitations and inconveniences, it is common to update emojis and emoticons for each system OS or messenger platform over a period of time, and it is common to update large-scale, especially in such environments, even if problems or complaints about specific images occur. There is also a problem that cannot be reflected in real time.

The present invention has been devised to solve the above-described problems, and it is possible to gradually and naturally generate pseudo-expression images according to accumulation of learning data according to machine learning without additional design time or cost, and provide real-time feedback corresponding thereto. The purpose is to provide a recording medium recording a program for processing a message including a combinatorial expression image item that can reflect and provide emoji or emoticons that reflect maximum diversity and feedback.

The recording medium according to an embodiment of the present invention for solving the above problems provides a message interface to a user terminal, processes messages transmitted and received through the message interface, and can be included in the message through the interface. A function of providing one or more pseudo-expression image items to the user terminal; A function of storing the one or more pseudo-expression image items; A function of classifying and managing one or more pseudo-expression image items stored in the database according to element information; And processing the machine learning of the pseudo-expression image item database based on the classification management information of the visual item management unit to generate a pseudo-expression image item learning model, and using the learning model, the first pseudo-expression image item stored in the database. And a function of generating a combined image item in which a second pseudo-expression image item is selected and combined.

According to an embodiment of the present invention, based on the element classification information of the pseudo-expression image items and the machine learning process using the image as an input value, a combination pseudo-expression images are generated, and a message service using the combination-expression image items is provided. can do.

Accordingly, according to an embodiment of the present invention, natural pseudo-expression images may be produced according to accumulation of learning data according to machine learning without additional design time or cost, and real-time feedback corresponding thereto may be reflected, thereby diversifying It is possible to provide a recording medium recording a program for processing a message including a combinatorial expression image item capable of providing personalized emoji or emoticons.

1 is a view schematically showing an entire system according to an embodiment of the present invention.
2 is a block diagram showing a service providing apparatus according to an embodiment of the present invention in more detail.
3 is a block diagram showing a combination image item generating unit according to an embodiment of the present invention in more detail.
4 is a flowchart illustrating a method of generating a combined image item according to an embodiment of the present invention.
5 is an example of generating a combined image item according to an embodiment of the present invention.
6 to 7 are diagrams for explaining a message interface provided according to an embodiment of the present invention.

The following merely illustrates the principles of the present invention. Therefore, those skilled in the art may implement various principles included in the concept and scope of the present invention and implement the principles of the present invention, although not explicitly described or illustrated in the present specification. In addition, it is understood that all conditional terms and examples listed in this specification are, in principle, intended only for the purpose of understanding the concept of the present invention, and are not limited to the examples and states specifically listed in this way. Should be.

In addition, it should be understood that all detailed descriptions that list the principles, aspects and embodiments of the invention as well as specific embodiments are intended to include structural and functional equivalents of these matters. It should also be understood that these equivalents include all currently invented equivalents as well as equivalents to be developed in the future, ie all devices invented to perform the same function regardless of structure.

Thus, for example, it should be understood that the block diagrams herein represent conceptual views of exemplary circuits embodying the principles of the invention. Similarly, all flow diagrams, state transition diagrams, pseudo-codes, etc. are understood to represent various processes performed by a computer or processor, whether or not the computer or processor is clearly depicted, which may be substantially represented on a computer-readable medium. Should be.

Also, the clear use of terms presented in terms of processors, controls, or similar concepts should not be interpreted exclusively by reference to hardware capable of executing software, and without limitation, digital signal processor (DSP) hardware, ROM for storing software. It should be understood as implicitly including (ROM), RAM and non-volatile memory. Other hardware for governors may also be included.

The above objects, features and advantages will become more apparent through the following detailed description in connection with the accompanying drawings, and accordingly, those skilled in the art to which the present invention pertains can easily implement the technical spirit of the present invention. There will be. In addition, in the description of the present invention, when it is determined that the detailed description of the known technology related to the present invention may unnecessarily obscure the subject matter of the present invention, the detailed description will be omitted.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a view schematically showing an entire system according to an embodiment of the present invention.

Referring to FIG. 1, a system according to an embodiment of the present invention may include a plurality of user terminals 100 and a service providing device 200.

More specifically, the user terminals 100 and the service providing device 200 may be connected by wire or wirelessly through a network, and each user terminal 100 and the service providing device 200 may be connected to the Internet for mutual communication between networks. Data can be transmitted and received through networks, LANs, WANs, Public Switched Telephone Networks (PSTNs), Public Switched Data Networks (PSDNs), cable TV networks, WIFI, mobile networks, and other wireless networks. In addition, each user terminal 100 and the service providing device 200 may include respective communication modules for communicating using a protocol corresponding to each communication network.

In addition, the user terminal 100 described herein includes a mobile phone, a smart phone, a laptop computer, a terminal for digital broadcasting, PDA (Personal Digital Assistants), PMP (Portable Multimedia Player), and navigation. Although it may be included, the present invention is not limited to this, and may be various devices capable of user input and information display.

In such a system, the user terminal 100 may be connected to the service providing device 200 to receive a message interface service with other user terminals.

For this processing, the service providing apparatus 200 provides a message interface to the user terminal 100, delivers a message received through the message interface to another user terminal, or receives a message received from another user terminal. The processing delivered to (100) can be performed.

For example, the service providing device 200 may be a chat server device, the user terminal 100 may be a portable terminal in which a chat application is installed, and the service providing device 200 may be used or used for each user terminal. It provides a chat interface in the group generated by, and can send and receive messages transmitted and received through the chat interface.

The message processed by the service providing device 200 may include various message contents, and the content may include data in an email, short or long text message format, and the service providing device 200 may display the message content. The e-mail, short or long text message format data may be transmitted through a mobile communication network or a dedicated internet network.

Meanwhile, the service providing device 200 includes a communication protocol conversion, a file server, a collect server, a push gateway server, a transport server, an account server, an admin server, a reception server, a conversion server, a messaging server, It may be implemented as a server device that performs an overall role of a push receiving server, a push sending server function, and the like.

In addition, the communication network to which the user terminal 100 and the service providing device 200 are connected may be different networks, and accordingly, the service providing device 200 may provide data conversion processing according to each network protocol. In addition, the service providing apparatus 200 may encrypt and manage all or part of the message information as necessary.

In addition, the service providing apparatus 200 according to an embodiment of the present invention may provide various additional information and functions processed according to a message service to the user terminal 100, provide service subscription and affiliate processing, or a financial server It may further perform billing processing and the like in order to provide an affiliate service with a plurality of operator servers by communicating with.

In addition, a separate application may be installed in the user terminal 100, and a general mobile terminal including a separate control unit (not shown), a storage unit (not shown), an output unit (not shown), and a communication unit (not shown). It can basically include the configuration of.

Through such a system, when the service providing apparatus 200 provides a relay service of message content transmitted and received through a message interface, the message interface may include one or more pseudo-expression image items.

The pseudo-expression image item may include an image representing a user's emotion, mood, state, and other intention to express, and the image may include, for example, a pictorial emoji format matching a specific pseudo-expression information, a text emoticon It may consist of data in a format or a separately specified picture file format, or a file format that allows movement. In addition, the expression information may include emotion, mood, state or other intention keywords, and data representing additional descriptions, and one image may match one or more keywords and data representing additional descriptions.

Accordingly, the service providing device 200 may fuse the pseudo-expression information and the image, and store and manage the fused information as a pseudo-expression image item, and the service providing device 200 generates a list of pseudo-expression image items By providing it through the message interface output from the user terminal 100.

The user terminal 100 may input a message through the message interface or select and input a desired pseudo-expression image item from the list of pseudo-expression image items, and a user terminal having a different message including the input pseudo-expression image item Or can be registered as a post in a group chat room.

In addition, the service providing apparatus 200 may generate an image item learning model by performing a machine learning process corresponding to pre-stored existing pseudo-expression image items, and based on the learning model, a newly combined combinatorial pseudo-expression image item You can create

Here, the learning model may be a model for deriving condition setting information for generating a combination of image items from existing pseudo-expression image items. In order to learn the condition setting information derivation algorithm, training is performed by processing pseudo-expression image items. It is possible to obtain element classification information as information and perform machine learning processing using the element classification information as an input value.

Then, the service providing apparatus 200 selects a first pseudo-expression image item and a second pseudo-expression image item from the pseudo-expression image database, and combines elements of each pseudo-expression image item according to the learned condition setting information. It is possible to generate a combined pseudo-expression images, store the generated combined pseudo-expression images, and provide a message service using the combined pseudo-expression image items.

Accordingly, according to an embodiment of the present invention, natural pseudo-expression images may be produced according to accumulation of learning data according to machine learning without additional design time or cost, and real-time feedback corresponding thereto may be reflected, thereby diversifying It is possible to provide a message processing method and apparatus including a combinatorial expression image item capable of providing personalized emoji or emoticons.

2 is a block diagram more specifically showing a service providing apparatus according to an embodiment of the present invention, and FIG. 3 is a block diagram more specifically showing a combination image item generating unit according to an embodiment of the present invention.

2 and 3, the service providing apparatus 200 includes a control unit 210, a communication unit 220, a message interface unit 225, a database 240, a visual item management unit 250, and an analysis unit 260 And a combination image item generator 230.

The control unit 210 controls the overall operation of each module to provide operation processing according to a service providing method according to an embodiment of the present invention, and may include one or more microprocessors for this.

The communication unit 220 may include one or more modules that enable wired or wireless communication between networks in which the user terminal 100 or the service providing device 200 is located. For example, the communication unit 220 may include at least one of a mobile communication module, a wired Internet module, a wireless Internet module, and a short-range communication module.

In addition, the message interface unit 225 may register user information in response to users using a service according to an embodiment of the present invention, and relay, store, and manage messages transmitted and received. The user information may include user identification information for sending and receiving messages, and the user information may be newly registered, updated, and managed according to a user information registration or update request received from the user terminal 100. Also, all or part of the user information may be encrypted and stored.

Meanwhile, the database 240 may store a program for the operation of each component, and may store input / output data (eg, a pseudo-expression image item, a combination image item, etc.), for learning processing You can also store and manage separate classified databases.

The visual item management unit 250 may match and store the pseudo-expression image items stored through the database 240 according to the classification information. The classification information may include attribute information for each element of the image item, and the attribute information for each element may include, for example, pseudo-expression type information, image type information, usage frequency information for each user, feedback information, and the like. .

Then, the visual item management unit 250 may receive the feedback information received from the user terminal 100 through the message interface unit 225 and apply it to the classification information of the pseudo-expression image item, and the feedback information is an example. For example, it may include at least one of evaluation information, report information, or usage rate information for each element corresponding to a pseudo-expression image item.

Particularly, when the report information is received, the visual item management unit 250 may perform a correction or deletion process of a pseudo-expression image item corresponding to the report information, and the database 240 update is processed in real time. User feedback corresponding to the pseudo-expression image item may be actively reflected.

Meanwhile, the visual item management unit 250 may provide classification information for generating a combination image of the combination image item generation unit 230, and the combination image item generated by the combination image item generation unit 230 to the database 240 You can also register and manage new ones.

More specifically, referring to FIG. 3, the combined image item generation unit 230 includes an image item combination unit 231, a first machine learning processing unit 233, a feedback information weight applying unit 235, and a second machine learning A processing unit 237 is included, and an image item generation learning model 232 generated according to machine learning may be provided through the database 240.

The first machine learning processing unit 233 processes the first machine learning of the pseudo-expression image item database 240 based on the classification management information of the visual item management unit 250 to generate an initial image item learning model, The generated image item generation learning model 232 is processed to be stored in the database 240.

Here, the image item generation learning model may include a learning model for deriving a combination setting information matching algorithm for generating an image item, and may include classification information of an image item as an input variable, and an output variable is a pseudo expression It may include combination setting information for combination between image items. The combination setting information may include combination information between elements, image arrangement type information, image grid position information, image scale ratio information, semantic combination information, keywords of the first pseudo-expression image item and the second pseudo-expression image item to be combined. It may include at least one of combination information, preference information, and configuration information.

The machine learning algorithm in the first machine learning processing unit 233 may use one or more processing methods, and further applies an input variable and a target variable to a combination of a plurality of machine learning algorithms and a plurality of machine learning algorithms to learn in advance. , An optimal machine learning algorithm or an optimal combined machine learning algorithm may be determined by considering at least one of accuracy, computation time, performance ratio, and model stability of each algorithm according to the learned results.

Machine learning algorithms processed by the first machine learning processing unit 233 include, for example, Ridge Regression, Logistic Regression, Generalized Linear Model, and Random Forest. , GradientBoosting Model and Neural Network can be used.

Ridge regression is a basic linear model and provides additional parameters to handle extremes or outliers. Logistic regression analysis is a basic linear model, and it is effective to use it when the target variable, that is, the object to be predicted, shows a binomial distribution. The generalized linear model is a basic linear model, and it is effective to use it when the target variable shows a Poisson distribution. This model is a random forest ensemble model composed of a large number of decision trees. Each decision tree is individually developed considering the relationship between input variables and target variables, and has more flexible characteristics than the upper linear model. The stimulus model is also called the Generalized Boosting Model or GBM, and it is often referred to as a model with higher accuracy than the Random Forest because there is a difference between an ensemble model with a tendency similar to that of a Random Forest or each decision tree being opened considering the accuracy of the decision tree that has already been created. It is also considered.

The neural network needs to be fine-tuned for parameter tuning because it can encompass both linear and non-linear patterns by adjusting parameters with a very flexible model that can predict virtually any target. In particular, the neural network is a statistical learning algorithm inspired by a neural network of biology in machine learning and cognitive science, and a deep learning method can be implemented by composing such a neural network in a complex manner.

For example, the neural network may include overall fully connected (FC), convolutional neural network (CNN) optimized with Max Pooling and convolution, and sequential neural networks (RNN) useful for time series data.

When the neural network learning process is used, for example, the first machine learning processing unit 233 generates a predicted value and a predicted value of a combined image item generated according to combination setting information from classification information of the image item database 240 And to minimize the difference between the actual classification information value, a learning process (backpropagation) of updating the weight of the connection may be performed. To this end, the first machine learning processing unit 233 finds a weight that minimizes the difference between the actual value and the predicted value by repeating the process of partially differentiating the cost function with respect to the weight and then gradually moving the weight in the gradient direction. As a method, optimal combination setting information similar to the real can be learned using a gradient descent method or the like.

Meanwhile, the first machine learning processing unit 233 may construct an image generation model that generates a composite image between image items as well as combination setting information by using a recently emerging Generative Adversarial Network (GAN) learning method. When GAN is used, the first machine learning processing unit 233 generates training data for GAN training from image item classification information and image data, and is provided with a generator and a discriminator based on the training data, respectively. By processing the process to determine the similarity of the image item data generated by the generator to the actual image item by the discriminator, but by processing the process of increasing the accuracy according to the amount of learning data by exchanging mutual information between the generator and the discrimination period, the combined image It can be processed so that it is appropriately generated to be difficult to distinguish from the actual existing image item.

When the first machine learning is processed as described above, the image item combination unit 231 determines the first pseudo-expression image item and the second pseudo-expression image item to be combined, and removes the first machine learning image item from the learning model. The combination setting information calculated corresponding to the first pseudo-expression image item and the second pseudo-expression image item is acquired, and a combination based on the combination setting information is processed to generate the combination-type image item.

The image item combination unit 231 may obtain combination information between elements of the first pseudo-expression image item and the second pseudo-expression image item from the combination setting information obtained from the learned image item generation learning model 232, Each element information of the first pseudo-expression image item and the second pseudo-expression image item may be combined according to the combination information between elements, and a combined image item according to the combination may be generated.

Here, the combination information between the elements includes at least one of image arrangement type information, image grid position information, image scale ratio information, semantic combining information, and keyword combining information of the first pseudo-expression image item and the second pseudo-expression image item. can do.

Also, the generated combined image item may be stored in the database and used recursively in the first machine learning process of the first machine learning processing unit 233.

Accordingly, the generated combined image item may be stored in the database 240 and provided to the user terminal 100 through the message interface unit 225.

For example, the message interface unit 225 may provide a list of pseudo-expression image items including the combination image item through the user terminal 100, and a combination image item selected from the pseudo-expression image item list may be displayed in the message. It can be processed to be included.

Meanwhile, the visual item management unit 250 may receive feedback information corresponding to the generated combined image item from the user terminal 100 and perform a database 240 update process corresponding to the feedback information.

The message interface unit 225 may request feedback information corresponding to the combined image item input by the user to the user terminal 100, and obtain feedback information as a response to the request and transmit it to the visual item management unit 250. . Here, the feedback information may include at least one of evaluation information, report information, and usage rate information for each element of the combined image item.

On the other hand, the feedback information weight applying unit 235, on the basis of at least one of the element-specific evaluation information and utilization information obtained from the feedback information, can process the weight tuning of the combination setting information corresponding to the feedback target combination image item. have.

Then, the second machine learning processing unit 237 processes the second machine learning of the image item learning model using the adjusted weight parameter by using the weight tuning information of the combination setting information corresponding to the feedback target combination image item. By doing so, the image item generation learning model 232 can be updated.

Accordingly, by more accurately correcting and learning the image item generation algorithm for the user, it is possible to create a more useful image item, and the utilization rate of the message service and the utilization rate of the combined image item may be increased according to an embodiment of the present invention. It has an effect.

On the other hand, the analysis unit 260 analyzes the messages transmitted and received between each user through the message interface unit 225, and provides image information, content information, setting information, additional description information of a combination image item that can be provided in response to the message, or Keywords and the like can be extracted.

In addition, the analysis unit 260 may further perform detection of a specific expression corresponding to the message content, recommendation of an image item, provision of an image item, and analysis of usage rate of an image item, using information extracted according to the analysis. .

For example, the analysis unit 260 may perform one or more of a number of natural language understanding techniques such as morphological analysis of a message, keyword recognition, speech act analysis, semantic analysis, or pseudo-expression image item analysis according to one or more messages and message ranges. By analyzing the message obtained through the message interface unit 225, a specific emotion expression phrase, which is a phrase expressing a specific emotion, is identified, and image information, content information, setting information, additional description information, or keywords corresponding to the phrase are extracted. Also, the image items matching at least one or more may be recommended or provided. In addition, the analysis unit 260 performs analysis processing corresponding to the pseudo-expressed image item, thereby corresponding image information, content information, setting information, Extract additional description information or keywords, and recommend other image items that match at least one May be provided. Fig. 4 is a flowchart illustrating a combined image created item according to an embodiment of the present invention, Figure 5 is an example of generating a combined image item in the embodiment;

4 and 5, the combined image item generator 230 determines a first pseudo-expression image and a second pseudo-expression image item to be combined (S101).

Then, the combined image item generation unit 230 obtains combination setting information calculated corresponding to the first image and the second image item from the first machine-learned image item generation learning model (S103).

Then, according to the combination setting information, the combination image item generation unit 230 determines the arrangement type between the first image and the second image (S105).

The placement type may include, for example, placement relationship information of the first image item and the second image item, and may include, for example, at least one of a left / right placement type, a top / bottom placement type, a diagonal placement type, and a front / rear placement type. The combination setting information may further include distance information between the first image and the second image, corresponding to the arrangement type. Accordingly, the arrangement type and the distance between images may be determined in advance.

Then, according to the arrangement type and the distance between the images, the combined image item generation unit 230 determines the grid positions of the first image and the second image (S107).

For example, the combined image item generation unit 230 is based on the center point of the first image and the center point of the second image, as the respective position information in the grid based on the combined image item grid, X and Y coordinates. Can decide.

For example, the grid may be a virtual grid with a variable distance between lines. For example, the location information may include location coordinates in a virtual grid created such that areas of some of the areas surrounded by the lines constituting the virtual grid are different from each other, and the distance between the lines may be varied when the virtual grid is created. You can. The virtual grid is not displayed on the display, and may be generated in correspondence to all or part of the display area.

When generating a combined image item, each location information may be recognized, for example, through a matrix index corresponding to a virtual grid. In other words, when the position of the first image is included in the area of the first column of the first row of the virtual grid, the index of (1, 1) can be recognized. Alternatively, an index that increases sequentially from left to right and from top to bottom may be used. In this case, the area of the first column of the first row of the virtual grid may have an index of '1', and the area of the first column of the second row may have an index of '2'. As such, a structure selected from among various structures may be used as an index of the virtual grid.

As described above, in this embodiment, the entire coordinate system is not used, and the position in each grid can be displayed using the index of the virtual grid in which the distance between the lines is variable.

Thereafter, the combined image item generation unit 230 determines the scale ratios of the first image and the second image according to the combination setting information (S109).

The image scale ratio may include ratio information between the first image and the second image, such as 1: 1, 1: 3, and 3: 1, and the size of each image in the combined image item is adjusted according to the scale ratio. Can be.

In addition, the combined image item generation unit 230 may generate respective size information and setting information according to a position and scale ratio determination corresponding to each image in the combined image item, and the setting information is stored in the database 240 Can be.

According to the scale ratio and location information determination, layout between image items may be determined using the grid, and the combined image item generation unit 230 may place each image item at positions not occupied in the display grid. And various processes can be used in the placement pattern and placement rules.

For example, the combined image item generator 230 may identify a match between a candidate image element and a corresponding set of one or more grid positions based on a set of placement patterns and placement rules for the image items, and placement The algorithm can determine unoccupied locations in the display grid. Positions in the display grid can be marked as occupied due to a specific area of the display grid that is designated for a particular purpose.

Also, the combined image item generator 230 may change the placement origin for the image item based on a specific placement rule applied when placing the image item on the grid.

For example, the placement rules can specify placement according to a typical English layout that runs from left to right and top to bottom. However, different placement rules may be applied to specify placement according to a typical Chinese text layout from top to bottom, right to left, or layers may be applied separately.

In addition, the combined image item generation unit 230, in the image arrangement, with respect to the starting point in the display grid that is the origin of the arrangement, the directional path of the arrangement (eg, the upper left corner of the display grid or the center point in the center of the display grid) And / or a grid-filling pattern (e.g., when the end of a row or column is reached, the pattern (1) jumps from the conventional text layout pattern to the opposite end of the next row or column, or (2) Batch processing can be performed based on one or more factors such as the zigzag pattern to U-turn to the next row or column), the priority of the placement along the first axis relative to the second axis, or the orientation of the displayable area. have. The order of filling positions in the display grid can be re-evaluated after image items are placed in the grid.

The combined image item generation unit 230 may generate a zoom-in version or a zoom-out version image of each image according to scaling, and it may be determined where the corresponding image is to be placed, and the placed image is determined. It can be laid out on the grid, or layered.

For example, the combined image item generation unit 230 may process image conversion for scaling, which includes the creation of screenshots, the generation of thumbnails, the extraction of video frames, the cropping of images, the scaling of images, and the like. Conversion of images from one format to another or file conversion (e.g., vector to raster conversion, Portable Networking Graphics (PNG), Joint Photographic Experts Group (JPEG), cross conversion between HEIF, HEVC, WebP, etc.) You can.

Then, the combined image item generator 230 combines semantic expression information between the first image and the second image item (S111).

The semantic expression information includes keyword information and additional texts, tags and related texts and images specifically representing mood, emotion, and state of each image item, such as 'pleasure', 'angry', and 'busy' It may be a variety of data information, and the combined image item generation unit 230 may obtain the combined semantic expression information by simply combining the two data information according to the combination setting information or in order.

Thereafter, the combined image item generator 230 combines classification information between the first image and the second image item (S113).

The classification information may include classification keyword information for classifying image items, for example. The classification keyword information may include keyword information indicating attributes of image items such as 'mood', 'state', and 'thing'.

Then, the combined image item generation unit 230 uses the pseudo-expression image information processed by steps S103 to S113 and the combined semantic expression information and classification information to generate the combined pseudo-expression image item (S115).

The generated combined pseudo-expression semantic item may be newly registered by the visual item management unit 250 and stored in the database 240.

Meanwhile, the message interface unit 225 provides the combined pseudo-expression image item through a message service (S117), and the visual item management unit 250 provides feedback of evaluation information and utilization information for each element corresponding to the combined pseudo-expression image item. It can be received (S121).

Accordingly, the combined image item generation unit 230 may perform combination setting information weight tuning of the learning model according to the feedback information through the feedback information weight applying unit 235, and through the second machine learning processing unit 237 , By applying the tuned combination setting information, a second machine learning corresponding to the image item generation learning model may be performed (S123).

The first machine learning and the second machine learning-based learning processing are repeatedly processed according to image generation and feedback information accumulation, thereby constructing an image item generation learning model that can sequentially and gradually generate useful image items. Make it possible.

FIG. 5 illustrates a combination image generating item as described above, and the first image and the second image item may be initial images included in the existing database 240, and one of the already generated combination image items is the first image. Alternatively, the second image may be determined.

As shown in FIG. 5 (A), when the first image is a 'face' image and the second image is a 'nose' image, according to the combination setting information, the 'nose' on the front side is an 'face' image on the back side. Combination image items in a form of being reduced to the center may be generated. In addition, the semantic expression information may be generated by combining 'nose / face'.

In addition, as shown in FIG. 5 (B), when the first image is a 'heart' image and the second image is a 'heart' image, the combination setting information matching with this is reduced according to diagonal arrangement. Combination image items of the arrangement type may be generated. Also, the semantic expression information may be generated in combination with 'heart / heart'.

In addition, as shown in FIG. 5 (C), as a combination image that has been previously generated, a second image in which two hearts are combined and a first image representing “face” may be combined. In this case, a combination image item in a form in which the 'two hearts' on the front side are reduced to the top of the 'face' image on the rear side may be generated. The semantic expression information may be generated in combination with 'heart / heart / face'.

For the combination processing, the combined image item generation unit 230 may layer and process the first image and the second image shown on the left side of FIG. 5, respectively.

In addition, the combined image item generation unit 230 may perform various transformation processes including scaling magnification, grid arrangement, front and rear arrangement, and color change information corresponding to each layer, and combining each transformed layer, Each combination image on the right side of FIG. 5 may be generated.

The combined image thus generated may be selected and / or ranked as a candidate according to relevance to the user, user preference, or other factors. For example, the combined image may be selected and / or ranked according to user evaluation information.

Also, the item generator 230 may determine attribute values between images according to user evaluation information corresponding to the combined image. Various values corresponding to the image may be illustrated as the attribute value, and for example, a connection value between images may be illustrated. For example, the item generating unit 230 may determine a connection value indicating a connection strength between images and a relevance score for the connection value according to user evaluation information.

6 to 7 are diagrams for explaining a message interface provided according to an embodiment of the present invention.

6 to 7, the message interface unit 225 provides one or more combinations of pseudo-expression image items in response to the message input window of the chat application of the user terminal 100, and prompts the user according to the user's selection. It can be processed to be input to the image.

Referring to FIG. 6, the message interface unit 225 according to an embodiment of the present invention analyzes a message input to a message input window of a chat application of the user terminal 100 through the analysis unit 260, and analyzes the expression It is possible to provide a list of recommended items including at least one image information corresponding to the information, content information, setting information, additional description information, or a combination expression image item in which keyword information is matched.

In addition, the message interface unit 225 is provided through the selection means including at least one of two or more suitable points in the message list interface or the auxiliary list window called to the right of the message input window on the message application to provide the list of recommended items. A list of recommended items can be processed to be provided, and accordingly, the selected combination pseudo-expression image item can be input through a message input window or automatically transmitted through a message application.

For example, when a plurality of service users who are using a message application includes a specific emotion expression phrase in a chat message to be posted in the message input window, the analysis unit 260 expresses a combined expression corresponding to the specific emotion expression phrase. The recommended item list including the image item is determined, and the determined recommended item list is called at a point including at least one of a suitable list window or a specific message interface unit adjacent to the input window, and the image determined by user selection. Items can be entered on the message.

For such processing, the message interface unit 225 periodically provides a combination image item to the application of the user terminal 100 in advance, or accesses the database 240 in real time to provide a combination image item that is updated in real time when a message is input. It can also be processed.

In addition, the message interface unit 225 may process a message including a combination image item to be transmitted when the message is input to a chat application, and when a message containing the combination image item is transmitted, the user terminal 100 ).

Here, in the feedback request, the combination guide information corresponding to the combined image item may be output together, and the user performs the second machine learning through the rating information for the combination guide information and the opinion input for feedback for each element. Feedback information may be input, and the message interface unit 225 may collect the rating information and the feedback information, and the usage rate information of the combined image item, and process it to be delivered to the visual item management unit 250.

On the other hand, the method according to the various embodiments of the present invention described above is implemented in the form of installation data for execution in a terminal device and stored in various non-transitory computer readable medium to each server or devices Can be provided. Accordingly, the user terminal 100 may access the server or device to download the installation data.

The non-transitory readable medium means a medium that stores data semi-permanently and that can be read by a device, rather than a medium that stores data for a short time, such as registers, caches, and memory. Specifically, the various applications or programs described above may be stored and provided in a non-transitory readable medium such as a CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, and the like.

In addition, although the preferred embodiments of the present invention have been shown and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present invention claimed in the claims. In addition, various modifications can be implemented by those skilled in the art, and these modifications should not be individually understood from the technical idea or prospect of the present invention.

Claims (3)

A function of providing a message interface to a user terminal, processing messages transmitted and received through the message interface, and providing one or more pseudo-expression image items included in the message to the user terminal through the interface;
A function of storing the one or more pseudo-expression image items;
A function of classifying and managing one or more pseudo-expression image items stored in the database according to element information; And
Processing the machine learning of the pseudo-expression image item database based on the classification management information of the visual item management unit to generate a pseudo-expression image item learning model, and using the learning model, a first pseudo-expression image item stored in the database and , A function for generating a combined image item by selecting and combining a second pseudo-expression image item,
The generating step further includes a function of determining at least one of size information, keyword information, and setting information corresponding to each of the first pseudo-expression image item and the second pseudo-expression image item of the combined image item and storing the result in the database. Containing
A computer readable recording medium comprising a program for processing a message providing a combinatorial expression image item. According to claim 1,
Analyze at least one of image information, content information, setting information, additional description information, or keywords of a combination image item that can be provided to a user by analyzing a message or image item transmitted and received between users through the message interface unit by the analysis unit A recording medium further comprising a function to do. According to claim 1,
A function of processing the first machine learning of the pseudo-expression image item database based on the classification management information of the visual item management unit to generate an initial image item learning model;
A function of processing weight tuning of combination setting information corresponding to a feedback target combination image item based on at least one of evaluation information for each element and utilization information obtained from the feedback information corresponding to the combination image item; And
And a function for processing the second machine learning of the image item learning model by using the weighted tuning information of the combination image item and the combination setting information corresponding to the feedback target combination image item.

Images