KR20230151952A - A method for message processing by providing combined expression image words - Google Patents

Abstract

A method for providing a message service according to an embodiment of the present invention provides a message interface to a user terminal, processes messages transmitted and received through the message interface, and displays one or more image items expressing opinions that can be included in the message through the interface. providing to the user terminal; storing the one or more pseudo-expressive image items; Classifying and managing one or more opinion expression image items stored in the database according to element information; and processing machine learning of the pseudo-expression image item database based on the classification management information of the visual item management unit to generate a pseudo-expression image item learning model, and using the learning model to create a first pseudo-expression image item stored in the database. and generating a combined image item by selecting and combining the second pseudo-expressive image items.

Description

A method for message processing by providing combined expression image words}

The present invention relates to a message processing method. More specifically, the present invention relates to a message processing method that provides combined image items for expressing opinions.

In messaging services such as chatting services, various forms of expression, such as emoticons and emoticons, are used to express users' emotions.

An emoticon is a symbol used in a chat service to express the emotions of service users. It is a compound term of emotion and icon. Early emoticons were created as a simple combination of various symbols and letters on the keyboard. , Recently, it has developed into a form that expresses the user's emotions through various types of character images.

In addition, unlike the emoticon method based on character combinations, emoji is a method of outputting pictograms embedded in the font form of the platform system, and is exemplified by a method in which an emotional expression image corresponding to the input emoji code is output.

As service users' demand for these emojis and emoticons increases, it has become common for message services to designate and provide various types of image items expressing opinions to service users.

However, for existing opinion expression image items such as emojis and emoticons, the designer must randomly create images that fit each opinion expression, match the created image to the opinion expression and store it on the platform system, and use the platform system where the user is stored. There is an inconvenience in that you can only enter messages by calling and selecting them on the screen.

In the current environment where the demand for emoji and emoticons is continuously increasing, the design cost increases in proportion to the number of creations, the system construction cost increases due to platform updates, and the number of unnecessary images increases when user inputs. It may take a long time or cause problems with multiple unwanted images appearing.

Due to these limitations and inconveniences, it is common for emojis and emoticons for each system OS or messenger platform to be updated on a large scale over a period of time. Especially in this environment, even if problems or complaints regarding specific images occur, Problems that cannot be reflected in real time also arise.

The present invention was developed to solve the problems described above, and can gradually produce natural expression images according to the accumulation of learning data through machine learning without additional design time or cost burden, and provides corresponding real-time feedback. The purpose is to provide a message processing method that includes a combined opinion expression image item that can provide emojis or emoticons that reflect maximized diversity and feedback.

A method according to an embodiment of the present invention for solving the above-described problem is a method of providing a message service, providing a message interface to a user terminal, processing messages transmitted and received through the message interface, and using the interface. providing one or more image items expressing opinions that can be included in the message to the user terminal; storing the one or more pseudo-expressive image items; Classifying and managing one or more opinion expression image items stored in the database according to element information; and processing machine learning of the pseudo-expression image item database based on the classification management information of the visual item management unit to generate a pseudo-expression image item learning model, and using the learning model to create a first pseudo-expression image item stored in the database. and generating a combined image item by selecting and combining the second pseudo-expressive image items.

In addition, the method according to an embodiment of the present invention for solving the above-mentioned problems can be implemented with a program for executing the method on a computer and a recording medium on which the program is recorded.

According to an embodiment of the present invention, combined opinion expression images are generated based on machine learning processing using element classification information and images of opinion expression image items as input values, and a message service using the combination opinion expression image items is provided. can do.

Accordingly, according to an embodiment of the present invention, natural expression images can be produced by accumulating learning data through machine learning without additional design time or cost burden, and corresponding real-time feedback can be reflected, thereby diversifying. A message processing method including a combined opinion expression image item that can provide personalized emojis or emoticons can be provided.

1 is a diagram schematically showing the entire system according to an embodiment of the present invention.
Figure 2 is a block diagram illustrating in more detail a service providing device according to an embodiment of the present invention.
Figure 3 is a block diagram illustrating in more detail a combined image item generator according to an embodiment of the present invention.
Figure 4 is a flowchart illustrating a method for creating a combined image item according to an embodiment of the present invention.
Figure 5 is an example of creating a combined image item according to an embodiment of the present invention.
6 to 7 are diagrams for explaining a message interface provided according to an embodiment of the present invention.

The following merely illustrates the principles of the invention. Therefore, those skilled in the art will be able to invent various devices that embody the principles of the present invention and are included in the spirit and scope of the present invention, although not explicitly described or shown herein. In addition, it is understood that all conditional terms and embodiments listed herein are, in principle, expressly intended only for the purpose of ensuring that the concept of the invention is understood, and are not limited to the embodiments and conditions specifically listed as such. It has to be.

Additionally, it is to be understood that any detailed description reciting principles, aspects, and embodiments of the invention, as well as specific embodiments, is intended to encompass structural and functional equivalents thereof. In addition, these equivalents should be understood to include not only currently known equivalents but also equivalents developed in the future, that is, all elements invented to perform the same function regardless of structure.

Accordingly, for example, the block diagrams herein should be understood as representing a conceptual view of an example circuit embodying the principles of the invention. Similarly, all flow diagrams, state transition diagrams, pseudo-code, etc. are understood to represent various processes that can be substantially represented on a computer-readable medium and are performed by a computer or processor, whether or not the computer or processor is explicitly shown. It has to be.

Additionally, the clear use of terms such as processor, control, or similar concepts should not be construed as exclusively referring to hardware capable of executing software, and should not be construed as referring exclusively to hardware capable of executing software, including without limitation digital signal processor (DSP) hardware and ROM for storing software. It should be understood as implicitly including ROM, RAM, and non-volatile memory. Other hardware for public use may also be included.

The above-described purpose, features and advantages will become clearer through the following detailed description in conjunction with the accompanying drawings, and accordingly, those skilled in the art will be able to easily implement the technical idea of the present invention. There will be. Additionally, in describing the present invention, if it is determined that a detailed description of known technologies related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description will be omitted.

Hereinafter, a preferred embodiment according to the present invention will be described in detail with reference to the attached drawings.

1 is a diagram schematically showing the entire system according to an embodiment of the present invention.

Referring to FIG. 1, a system according to an embodiment of the present invention may include a plurality of user terminals 100 and a service providing device 200.

More specifically, the user terminals 100 and the service providing device 200 may be connected wired or wirelessly through a network, and each user terminal 100 and the service providing device 200 may be connected to the Internet for mutual communication between networks. Data can be transmitted and received through networks, LAN, WAN, PSTN (Public Switched Telephone Network), PSDN (Public Switched Data Network), cable TV network, WIFI, mobile communication network, and other wireless communication networks. Additionally, each user terminal 100 and the service providing device 200 may include respective communication modules for communicating using a protocol corresponding to each communication network.

In addition, the user terminal 100 described in this specification includes mobile phones, smart phones, laptop computers, digital broadcasting terminals, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Players), navigation, etc. However, the present invention is not limited to this and may be a variety of devices capable of user input and information display.

In such a system, the user terminal 100 is connected to the service providing device 200 and can receive a message interface service with other user terminals.

For this processing, the service providing device 200 provides a message interface to the user terminal 100, and transfers messages received through the message interface to other user terminals, or transmits messages received from other user terminals to the user terminal. Processing of forwarding to (100) can be performed.

For example, the service providing device 200 may be a chat server device, the user terminal 100 may be a portable terminal with a chat application installed, and the service providing device 200 may be used between each user terminal or between terminals. It provides a chat interface within the group created by, and relays and archives messages sent and received through the chat interface.

The message processed by the service providing device 200 may include various message contents, and the content may include data in an email, short, or long text message format, and the service providing device 200 may provide the message content. Data in the format of email, short text, or long text message can be transmitted through a mobile communication network or a dedicated Internet network.

Meanwhile, the service providing device 200 includes communication protocol conversion, file server, collect server, push gateway server, transfer server, account server, admin server, reception server, conversion server, messaging server, It can be implemented as a server device that performs overall roles such as push receiving server and push sending server functions.

Additionally, the communication networks to which the user terminal 100 and the service providing device 200 are connected may be different networks, and accordingly, the service providing device 200 may provide data conversion processing according to each network protocol. Additionally, the service providing device 200 may encrypt and manage all or part of the message information as needed.

In addition, the service providing device 200 according to an embodiment of the present invention can provide various additional information and functions processed according to the message service to the user terminal 100, provide service subscription and affiliate processing, or provide a financial server. By communicating with, billing processing to provide affiliated services with multiple business servers can be further performed.

In addition, a separate application may be installed in the user terminal 100, and it is a general portable terminal including a separate control unit (not shown), a storage unit (not shown), an output unit (not shown), and a communication unit (not shown). The configuration can be basically included.

Through such a system, when the service providing device 200 provides a relay service of message content transmitted and received through a message interface, the message interface may include one or more image items expressing opinions.

An image item expressing an opinion may include an image representing the user's emotion, mood, state, and other intention to express, and the image may be, for example, a pictogram emoji format or text emoticon that matches specific opinion expression information. It may consist of data in a format or a separately specified picture file format, or a file format that allows movement. Additionally, the opinion expression information may include emotion, mood, state, or other intention keywords and data representing additional explanations, and one image may be matched with one or more keywords and data representing additional explanations.

Accordingly, the service providing device 200 can fuse the opinion expression information and the image, store and manage the fused information as an opinion expression image item, and the service provision device 200 generates a list of opinion expression image items. This can be provided through a message interface output from the user terminal 100.

The user terminal 100 can input a message through the message interface, or select and input a desired opinion expression image item from the list of opinion expression image items, and a message containing the entered opinion expression image item may be sent to another user terminal. It can be delivered to or registered as a post in a group chat room.

In addition, the service providing device 200 may generate an image item learning model by performing machine learning processing corresponding to pre-stored existing opinion expression image items, and newly combined opinion expression image items based on the learning model. can be created.

Here, the learning model may be a model for deriving condition setting information for creating a combination of image items from existing pseudo-expression image items, and training is performed by processing pseudo-expression image items to learn the condition setting information derivation algorithm. Element classification information as information can be obtained, and machine learning processing using the element classification information as an input value can be performed.

Then, the service providing device 200 selects the first pseudo-expression image item and the second pseudo-expression image item from the pseudo-expression image database, and combines the elements of each pseudo-expression image item according to the learned condition setting information. It is possible to generate combined opinion expression images, store the generated combined opinion expression images, and provide a message service using the combined opinion expression image items.

Accordingly, according to an embodiment of the present invention, natural expression images can be produced by accumulating learning data through machine learning without additional design time or cost burden, and corresponding real-time feedback can be reflected, thereby diversifying. A message processing method and device for processing a message including a combined opinion expression image item capable of providing personalized emojis or emoticons can be provided.

Figure 2 is a block diagram illustrating in more detail a service providing device according to an embodiment of the present invention, and Figure 3 is a block diagram illustrating in more detail a combined image item generator according to an embodiment of the present invention.

2 and 3, the service providing device 200 includes a control unit 210, a communication unit 220, a message interface unit 225, a database 240, a visual item management unit 250, and an analysis unit 260. and a combination image item generator 230.

The control unit 210 controls the overall operation of each module, provides operation processing according to the service provision method according to an embodiment of the present invention, and may include one or more microprocessors for this purpose.

The communication unit 220 may include one or more modules that enable wired or wireless communication between the user terminal 100 or the network where the service providing device 200 is located. For example, the communication unit 220 may include at least one of a mobile communication module, a wired Internet module, a wireless Internet module, and a short-range communication module.

In addition, the message interface unit 225 can register user information in response to users who use the service according to an embodiment of the present invention, and relay, store, and manage transmitted and received messages. The user information may include user identification information for sending and receiving messages, and the user information may be newly registered, updated, and managed according to a user information registration or update request received from the user terminal 100. Additionally, all or part of user information may be encrypted and stored.

Meanwhile, the database 240 may store programs for the operation of each component, and may also store input/output data (e.g., pseudo-expression image items, combination image items, etc.) for learning processing. Separately classified databases can also be stored and managed.

The visual item management unit 250 may store and manage the image items expressing opinions stored through the database 240 by matching them according to their classification information. The classification information may include attribute information for each element of the image item, and the attribute information for each element may include, for example, information on the type of opinion expressed, information on the type of image, information on the number of uses by user, and feedback information. .

In addition, the visual item management unit 250 may receive the feedback information received from the user terminal 100 through the message interface unit 225 and apply it to the classification information of the opinion expression image item, and the feedback information may be For example, it may include at least one of element-specific evaluation information, report information, or usage rate information corresponding to the opinion expression image item.

In particular, when report information is received, the visual item management unit 250 can perform the process of modifying or deleting the image item expressing opinion corresponding to the report information, and such update of the database 240 is processed in real time, so that each User feedback corresponding to image items expressing opinions can be actively reflected.

Meanwhile, the visual item management unit 250 may provide classification information for generating a combination image of the combination image item creation unit 230, and store the combination image item generated by the combination image item creation unit 230 in the database 240. You can also manage it by registering a new one.

More specifically, referring to FIG. 3, the combination image item generator 230 includes an image item combination unit 231, a first machine learning processing unit 233, a feedback information weight application unit 235, and a second machine learning unit. It includes a processing unit 237, and an image item generation learning model 232 generated according to machine learning may be provided through the database 240.

The first machine learning processing unit 233 processes the first machine learning of the opinion expression image item database 240 based on the classification management information of the visual item management unit 250 to generate an initial image item learning model, The generated image item creation learning model 232 is processed to be stored in the database 240.

Here, the image item generation learning model may include a learning model for deriving a combination setting information matching algorithm for generating image items, and may include classification information of the image item as an input variable, and the output variable may be a pseudo expression. May include combination setting information for combination between image items. The combination setting information may include combination information between elements, image arrangement type information, image grid position information, image scale ratio information, semantic combination information, and keywords of the first and second expressive image items subject to combination. It may include at least one of combination information, preference information, and setting information.

The machine learning algorithm in the first machine learning processing unit 233 may use one or more processing methods, and further, apply input variables and target variables to a plurality of machine learning algorithms and a combination of a plurality of machine learning algorithms to pre-learn, , the optimal machine learning algorithm or the optimal combined machine learning algorithm can be determined by considering at least one of the accuracy of each algorithm according to the learning results, computation time to accuracy, performance ratio, and stability (fitness) of the model.

Machine learning algorithms processed in the first machine learning processing unit 233 include, for example, Ridge Regression, Logistic Regression, Generalized Linear Model, and Random Forest. , GradientBoosting Model, Neural Network, etc. can be used.

Ridge regression is a basic linear model and provides additional parameters to handle extreme values or outliers. Logistic regression analysis is a basic linear model and is effective when used when the target variable, that is, the object to be predicted, shows a binomial distribution. The generalized linear model is a basic linear model and is effective when used when the target variable shows a Poisson distribution. It is a random forest ensemble model made up of multiple decision trees. Each decision tree is developed individually considering the correlation between input variables and target variables and has more flexible characteristics compared to the upper linear model. The change boosting model is also called the Generalized Boosting Model or GBM. It is an ensemble model with similar tendencies to Random Forest, but the difference is that each decision tree is developed by considering the accuracy of the already created decision tree, so it is often a model with higher accuracy than Random Forest. It is also considered

A neural network is a very flexible model that can predict virtually any target and can encompass both linear and non-linear patterns by adjusting parameters, so parameter tuning needs to be detailed. In particular, a neural network is a statistical learning algorithm inspired by biological neural networks in machine learning and cognitive science, and deep learning methods can be implemented by configuring these neural networks in a complex manner.

For example, neural networks may include Fully Connected (FC), Convolutional Neural Network (CNN) optimized with Max Pooling and convolution, and Recurrent Neural Networks (RNN) useful for time series data.

When a neural network learning process is used, for example, the first machine learning processing unit 233 performs a process of generating a predicted value of a combined image item generated according to combination setting information from classification information of the image item database 240 and a predicted value. In order to minimize the difference between the actual classification information value and the actual classification information value, a learning process (backpropagation) can be performed to update the weight of the connection. To this end, the first machine learning processing unit 233 finds a weight that minimizes the difference between the actual value and the predicted value by repeating the process of partial differentiation of the cost function with respect to the weight and then gradually moving the weight in the gradient direction. As a method, gradient descent can be used to learn optimal combination setting information similar to reality.

Meanwhile, the first machine learning processing unit 233 may use the recently emerging GAN (Generative Adversarial Network) learning method to build an image generation model that generates not only combination setting information but also a composite image between image items. When GAN is used, the first machine learning processing unit 233 generates learning data for GAN training from image item classification information and image data, and is equipped with a generator (GENERATOR) and a discriminator (DISCRIMINATOR) based on the learning data. The discriminator determines the similarity of the image item data generated by the generator to the actual image item by exchanging mutual information between the generator and the discriminator, thereby processing the process to increase accuracy according to the amount of learning data, thereby processing the combined image. It can be processed to be properly created so that it is difficult to distinguish from an actual existing image item.

When the first machine learning is processed in this way, the image item combination unit 231 determines a first expression image item and a second expression image item to be combined, and from the first machine learned image item learning model, 1 Obtain combination setting information calculated corresponding to the opinion expression image item and the second opinion expression image item, and process the combination based on the combination setting information to generate the combination image item.

The image item combination unit 231 may obtain combination information between elements of the first and second expression image items from the combination setting information obtained from the learned image item generation learning model 232, According to the combination information between elements, the element information of the first expression image item and the second expression image item can be combined, and a combined image item according to the combination can be generated.

Here, the inter-element combination information includes at least one of image arrangement type information, image grid position information, image scale ratio information, semantic combination information, and keyword combination information of the first pseudo-expression image item and the second pseudo-expression image item. can do.

Additionally, the generated combined image item may be stored in the database and used recursively in the first machine learning processing of the first machine learning processing unit 233.

Accordingly, the generated combined image item may be stored in the database 240 and provided to the user terminal 100 through the message interface unit 225.

For example, the message interface unit 225 may provide a pseudo-expression image item list including the combination image item through the user terminal 100, and the combination image item selected from the pseudo-expression image item list may be added to the message. It can be processed to be included.

Meanwhile, the visual item management unit 250 may receive feedback information corresponding to the generated combined image item from the user terminal 100 and perform update processing on the database 240 corresponding to the feedback information.

The message interface unit 225 may request feedback information corresponding to the combined image item entered by the user from the user terminal 100, obtain feedback information as a response to the request, and transmit it to the visual item management unit 250. . Here, the feedback information may include at least one of evaluation information for each element of the combined image item, report information, and usage rate information.

Meanwhile, the feedback information weight application unit 235 may process weight tuning of the combination setting information corresponding to the feedback target combination image item based on at least one of element-specific evaluation information and usage rate information obtained from the feedback information. there is.

And, the second machine learning processing unit 237 uses the weight tuning information of the combination setting information corresponding to the feedback target combination image item and processes the second machine learning of the image item learning model using the adjusted weight parameter. By doing so, the image item creation learning model 232 can be updated.

Accordingly, the image item creation algorithm for users can be more accurately calibrated and learned, making it possible to create more useful image items, and the usage rate of the message service and the use rate of combined image items according to embodiments of the present invention can be increased. There is an effect.

Meanwhile, the analysis unit 260 analyzes messages transmitted and received between each user through the message interface unit 225, and provides image information, content information, settings information, additional description information, or information of a combined image item that can be provided in response to the message. Keywords, etc. can be extracted.

In addition, the analysis unit 260 may further perform detection of a specific expression corresponding to the message content, recommendation of an image item, provision of an image item, analysis of the usage rate of the image item, etc., using the information extracted according to the analysis. .

For example, the analysis unit 260 uses one or more of a number of natural language understanding techniques, such as morpheme analysis of a message, keyword recognition, speech act analysis, semantic analysis, or expression image item analysis, according to one or more messages and message ranges. By analyzing the message obtained from the message interface unit 225, a specific emotional expression phrase, which is a phrase expressing a specific emotion, is identified, and corresponding image information, content information, setting information, additional explanation information, or keywords are extracted. , it is possible to recommend or provide at least one matching image item. In addition, the analysis unit 260 performs analysis processing corresponding to the image item expressing the opinion, and provides corresponding image information, content information, setting information, Additional description information or keywords, etc. may be extracted, and other image items matching at least one may be recommended or provided. FIG. 4 is a flowchart illustrating a method of generating a combined image item according to an embodiment of the present invention. 5 is an example of creating a combined image item according to an embodiment of the present invention.

Referring to Figures 4 and 5, the combination image item generator 230 determines the first expression image and the second expression image item to be combined (S101).

Then, the combination image item generator 230 obtains combination setting information calculated corresponding to the first image and the second image item from the first machine learned image item generation learning model (S103).

Then, according to the combination setting information, the combination image item generator 230 determines the arrangement type between the first image and the second image (S105).

The placement type may include placement relationship information of the first image item and the second image item, for example, and may include at least one of a left-right placement type, a top-bottom placement type, a diagonal placement type, and a front-back placement type. The combination setting information may further include distance information between the first image and the second image, corresponding to the arrangement type. Accordingly, the placement type and distance between images may be predetermined.

Then, according to the arrangement type and the distance between images, the combined image item generator 230 determines the grid positions of the first image and the second image (S107).

For example, the combined image item generator 230 generates the You can decide.

For example, the grid may be a virtual grid in which the distance between lines is variable. For example, the location information may include location coordinates within a virtual grid created so that some of the areas surrounded by the lines constituting the virtual grid have different widths, and the distance between the lines may vary when creating the virtual grid. You can. This virtual grid is not displayed on the display, and may be created to correspond to the entire area or a partial area of the display.

When creating a combined image item, each location information can be recognized through, for example, a matrix index corresponding to a virtual grid. In other words, if the location of the first image is included in the area of the first column of the first row of the virtual grid, an index of (1, 1) can be recognized. Alternatively, an index that increases sequentially from left to right and from top to bottom may be used. In this case, the area of the first column of the first row of the virtual grid may have an index of '1', and the area of the first column of the second row may have an index of '2'. In this way, the index of the virtual grid can use a structure selected from various structures.

As such, in this embodiment, the position within each grid can be displayed using the index of a virtual grid where the distance between lines is variable, rather than using the entire coordinate system.

Thereafter, the combination image item generator 230 determines the scale ratio of the first image and the second image according to the combination setting information (S109).

The image scale ratio may include ratio information between the first image and the second image, such as 1:1, 1:3, 3:1, etc., and the size of each image in the combined image item is adjusted according to the scale ratio. It can be.

In addition, the combined image item creation unit 230 may generate size information and setting information for each image according to the determination of the position and scale ratio corresponding to each image in the combined image item, and store the setting information in the database 240. It can be.

According to this scale ratio and location information determination, the layout between image items can be determined using the grid, and the combined image item creation unit 230 can place each image item in unoccupied positions in the display grid. There are various processes that can be used in the deployment pattern and deployment rules.

For example, the combined image item generator 230 may identify a match between a candidate image element and a corresponding set of one or more grid positions based on a set of placement patterns and placement rules for the image items, and the placement The algorithm may determine unoccupied positions in the display grid. Positions in the display grid may be marked as occupied due to a particular area of the display grid being designated for a particular purpose.

Additionally, the combination image item generator 230 may change the placement origin for the image item based on a specific placement rule applied when placing the image item on the grid.

For example, a placement rule may specify placement according to a typical British layout, from left to right and top to bottom. However, different placement rules may be applied to specify placement according to the typical top-bottom, right-to-left Chinese text layout, and may be applied separately to the layers.

In addition, in arranging images, the combined image item generator 230 provides a directional path of arrangement (for example, the upper left corner of the display grid or the center point in the center of the display grid) with respect to the starting point in the display grid, which is the origin of the arrangement, and /or a grid-filling pattern (e.g., when the end of a row or column is reached, the pattern (1) jumps to the opposite end of the next row or column in a conventional text layout pattern, or (2) Batch processing may be performed based on one or more factors, such as the orientation of the displayable area, the priority of placement along a first axis relative to a second axis, or a U-turn to the next row or column in a zigzag pattern. there is. The order in which positions are filled in the display grid may be reevaluated after the image items are placed in the grid.

The combination image item generator 230 can generate a zoomed-in version or a zoomed-out version of each image according to scaling, and it can be determined where the corresponding image will be placed, and the placed image will be at the determined location. It can be laid out or layered on a grid.

For example, the combined image item generator 230 can process image conversion for scaling, which includes the creation of screenshots, creation of thumbnails, extraction of video frames, cropping of images, scaling of images, and one Include at least one of the following: conversion of an image from one format to another or file conversion (e.g., vector to raster conversion, conversion between Portable Networking Graphics (PNG), Joint Photographic Experts Group (JPEG), HEIF, HEVC, WebP, etc.) You can.

Then, the combined image item generator 230 combines the semantic expression information between the first image and the second image items (S111).

Meaning expression information includes, for example, keyword information that specifically indicates the mood, emotion, and state of each image item, such as 'happy', 'angry', and 'busy', as well as related text and images such as additional explanations and tags. It may be a variety of data information, and the combination image item generator 230 may simply combine the two data information or combine them in order according to the combination setting information to obtain the combined semantic expression information.

Thereafter, the combined image item generator 230 combines classification information between the first image and second image items (S113).

Classification information may include, for example, classification keyword information for classifying image items. Classification keyword information may include keyword information representing properties of image items such as 'mood', 'state', and 'object'.

Then, the combined image item generator 230 uses the pseudo-expression image information processed in steps S103 to S113 and the combined semantic expression information and classification information to generate a combined pseudo-expression image item (S115).

The generated combined pseudo-expression semantic item may be newly registered by the visual item management unit 250 and stored in the database 240.

Meanwhile, the message interface unit 225 provides the combined opinion expression image item through a message service (S117), and the visual item management unit 250 provides evaluation information and usage rate information feedback for each element corresponding to the combined opinion expression image item. Can be received (S121).

Accordingly, the combined image item generator 230 may perform combination setting information weight tuning of the learning model according to the feedback information through the feedback information weight application unit 235, and through the second machine learning processing unit 237. , the tuned combination setting information can be applied to perform a second machine learning corresponding to the image item generation learning model (S123).

This first machine learning and second machine learning-based learning processing is repeatedly processed according to image generation and accumulation of feedback information to build an image item generation learning model that sequentially and gradually generates useful image items. make it possible

Figure 5 illustrates such a combination image creation item. The first image and the second image item may be initial images included in the existing database 240, and one of the already created combination image items may be the first image. Alternatively, it may be determined as the second image.

As shown in Figure 5(A), when the first image is a 'face' image and the second image is a 'nose' image, according to the combination setting information, the 'nose' on the front is the 'face' image on the back. A combination image item may be created in a form that is reduced and arranged in the center. Additionally, semantic expression information may be generated by combining information such as 'nose/face'.

In addition, as shown in Figure 5(B), when the first image is a 'heart' image and the second image is also a 'heart' image, the matching combination setting information reduces one of them according to the diagonal arrangement. A combination image item in the form of arrangement may be created. Additionally, semantic expression information can be generated by combining them, such as 'heart/heart'.

And, as shown in Figure 5(C), as a combination image that has already been created, a second image combining two hearts and a first image representing a 'face' may be combined. In this case, a combination image item can be created in which the 'two hearts' on the front are reduced and placed on top of the 'face' image on the back. Semantic expression information can be created by combining things like 'heart/heart/face'.

For the above combined processing, the combined image item generator 230 may process the first image and the second image shown on the left side of FIG. 5 by layering them.

In addition, the combined image item creation unit 230 can perform various transformation processing including scaling factor, grid placement, front/back placement, and color change information corresponding to each layer, and by combining each transformed layer, Each combination image on the right side of Figure 5 can be created.

The combined images thus generated may be selected as candidates and/or ranked according to relevance to the user, user preference, or other factors. For example, combined images may be selected and/or ranked according to user rating information.

Additionally, the item generator 230 may determine attribute values between images according to user evaluation information corresponding to the combined image. Attribute values may include various values corresponding to images, for example, connection values between images. For example, the item creation unit 230 may determine a connection value indicating the strength of connection between images and determine a relevance score for the connection value, based on user evaluation information.

6 to 7 are diagrams for explaining a message interface provided according to an embodiment of the present invention.

As shown in FIGS. 6 and 7, the message interface unit 225 provides one or more combined expression image items in the message input window of the chat application of the user terminal 100, and provides a message according to the user's selection. It can be processed to be entered on the screen.

Referring to FIG. 6, when the message input into the message input window of the chatting application of the user terminal 100 is analyzed through the analysis unit 260, the message interface unit 225 according to an embodiment of the present invention displays the analyzed expression. A recommended item list including a combined opinion expression image item matching at least one image information, content information, setting information, additional description information, or keyword information corresponding to the information may be provided.

In addition, in order to provide the recommended item list, the message interface unit 225 selects the recommended items through a selection means including at least one of an auxiliary list window called on the right side of the message input window on the message application or an appropriate point within the message interface unit. A list of recommended items can be provided, and the selected combined image item can be entered through a message input window or automatically entered as content to be transmitted through a message application.

For example, if a chat message to be posted in the message input window by a plurality of service users using a messaging application contains a specific emotional expression phrase, the analysis unit 260 generates a combined expression of opinion corresponding to the specific emotional expression phrase. A recommended item list including image items is determined, and the determined recommended item list is called at a point that includes one or more of the auxiliary list window adjacent to the input window or an appropriate location within a specific message interface unit, and the image determined by user selection. Items can be entered into the message.

For this processing, the message interface unit 225 periodically provides combination image items in advance to the application of the user terminal 100, or connects to the database 240 in real time to provide combination image items updated in real time when a message is input. You can also process it.

In addition, the message interface unit 225 may process a message containing a combination image item to be transmitted when the message is entered into a chat application, and when a message containing a combination image item is transmitted, the user terminal 100 ), you can request feedback.

Here, combination guide information corresponding to the combination image item may be output together with the feedback request, and the user may use the second machine learning through rating information for the combination guide information and opinion input for feedback for each element. Feedback information can be input, and the message interface unit 225 can collect the rating information, feedback information, and usage rate information of the combined image item, and process it to be delivered to the visual item management unit 250.

Meanwhile, the method according to various embodiments of the present invention described above is implemented in the form of installation data to be executed on a terminal device, stored in various non-transitory computer readable media, and stored in each server or device. can be provided. Accordingly, the user terminal 100 can connect to the server or device and download the installation data.

A non-transitory readable medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short period of time, such as registers, caches, and memories. Specifically, the various applications or programs described above may be stored and provided on non-transitory readable media such as CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, etc.

In addition, although preferred embodiments of the present invention have been shown and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the invention pertains without departing from the gist of the present invention as claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be understood individually from the technical idea or perspective of the present invention.

Claims (3)

In a method of providing a message service from a message service providing device,
Providing a message interface from a message interface unit to a user terminal, processing messages transmitted and received through the message interface, and providing one or more image items expressing opinions that can be included in the message to the user terminal through the interface;
storing the one or more pseudo-expression image items in a pseudo-expression image item database;
Classifying and managing one or more opinion expression image items stored in the database according to element information in a visual item management unit;
The combined image item generator processes machine learning of the pseudo-expressive image item database based on the classification management information of the visual item management unit to generate a pseudo-expressive image item learning model, and uses the learning model to create a pseudo-expressive image item learning model. Generating a combined image item by selecting and combining a first pseudo-expression image item and a second pseudo-expression image item;
Processing the first machine learning of the opinion expression image item database based on the classification management information of the visual item manager in a first machine learning processing unit to generate an initial image item learning model;
The feedback information weight application unit processes weight tuning of the combination setting information corresponding to the feedback target combination image item based on at least one of element-specific evaluation information and usage rate information obtained from the feedback information corresponding to the combination image item. step; and
Comprising the step of processing a second machine learning of the image item learning model in a second machine learning processing unit using the weight tuning information of the combination image item and combination setting information corresponding to the feedback target combination image item.
How to provide messaging services. The method of claim 1, wherein the step of generating the combined image item comprises:
Further comprising determining at least one of size information, keyword information, and setting information corresponding to each of the first pseudo-expression image item and the second pseudo-expression image item of the combined image item and storing it in the database.
How to provide messaging services. According to paragraph 1,
The analysis unit analyzes messages or image items transmitted and received between each user through the message interface unit, and extracts at least one of image information, content information, setting information, additional description information, or keywords of a combined image item that can be provided to the user. Containing more steps to
How to provide messaging services.