KR102155768B1 - Method for providing question and answer data set recommendation service using adpative learning from evoloving data stream for shopping mall - Google Patents

Abstract

Provided is a shopping mall question and answer recommendation service providing method using a question and answer data set evolving through learning, which comprises the steps of: receiving product data describing a product from a seller terminal; pre-processing text included in the product data, generating a word of the smallest unit after analyzing a morpheme through part-of-speech tagging, and analyzing the generated word; performing text analysis modeled by using a classifier after forming the analyzed text into a matrix; and extracting a previously stored question and answer data set based on the analyzed text based on similarity and recommending the question and answer data set to the seller terminal.

Description

Shopping mall Q&A recommendation service using Q&A data set evolving through learning {METHOD FOR PROVIDING QUESTION AND ANSWER DATA SET RECOMMENDATION SERVICE USING ADPATIVE LEARNING FROM EVOLOVING DATA STREAM FOR SHOPPING MALL}

The present invention relates to a method for providing a shopping mall Q&A recommendation service using a Q&A data set evolving through learning, and a method for automatically recommending a Q&A set of products by analyzing text or images of a product when a product is uploaded. to provide.

Chatbot refers to an interactive system or service that operates in the language people use in everyday life, and major IT companies have already started fierce platform competition with chatbots as the next major industry. The fashion field, which is sensitive to change, is showing a movement to use chatbots as personalized customer contact channels without missing this trend, and the online commerce field is also very interested in combining fashion and chatbots. However, most chatbots currently do not meet the expectations of users, which means that chatbots require qualitative verification rather than quantitative expansion. Fashion chatbots that provide personalized recommendation services to each customer are more essential to a high level of quality.In most shopping malls, not only chatbots, but also in Q&A, customers' questions are not answered in many cases.

At this time, when providing a question and answer, a method of providing a service based on natural language has been researched and developed. In this regard, Korean Patent Application Publication No. 2019-0059084 (published on May 30, 2019), which is a prior art, describes natural language text. Parsing and generating a verification question and verification answer, the verification question from the question answer generation device is analyzed according to the natural language processing algorithm to recognize the correct answer type, and the correct answer corresponding to the correct answer type recognized in the knowledge base Configuration that verifies the correct answer from the Q&A device based on the generated configuration and the correct answer for verification, and if the correct answer is an incorrect answer, collects information including the correct answer for verification from the web, and updates the knowledge base with the collected information Is disclosed.

However, if the above-described Q&A model is used, the answer can be automatically answered, but the answer that is not loaded in the database can not be answered, so customers using it complain of frustration and call ARS to wait for a call to a counselor. In addition, when a shopping mall operator uploads a product to the shopping mall, detailed information about the product is uploaded, but based on the information imbalance or asymmetry between the customer and the supplier, the business operator does not know what the customer does not know, so the customer wants to answer the question. Most of the information is not posted, and when a customer asks a question, they forget to register for Q&A or register for Q&A each time and repeat the same answer over and over again.Therefore, there was a problem that the operator also increased unnecessary workload. .

According to an embodiment of the present invention, when a shopping mall seller uploads a product to the platform, it recommends the Q&A data set required for the uploaded product, so that the shopping mall seller does not have to write question-and-answer data, and the customer You do not have to wait for a contact with an agent because you cannot find the answer to the question, and the seller itself provides the input for learning so that the quality and quantity of Q&A can be continuously increased by adding parts that are insufficient or need to be added. Since the recommendation model is self-evolving and retrained, the Q&A database can be enriched as it is used. Furthermore, when a chatbot is introduced, the connection to the customer center can be reduced because the number of queries that can be answered by the chatbot increases. It is possible to provide a method of providing a shopping mall Q&A recommendation service using a Q&A data set evolving through learning, which can lower the work load of the customer and increase the customer's emotional quality because customers can receive answers immediately. However, the technical problem to be achieved by the present embodiment is not limited to the technical problem as described above, and other technical problems may exist.

As a technical means for achieving the above-described technical problem, an embodiment of the present invention includes the step of receiving product data describing a product from a seller terminal, pre-processing the text included in the product data, and performing part-of-speech tagging. After morpheme analysis, the smallest unit of word is generated, the generated word is analyzed, the analyzed text is formed into a matrix and then modeled using a classifier is performed, and a pre-stored based on the analyzed text And extracting the Q&A data set based on the similarity and recommending it to the seller terminal.

In another embodiment of the present invention, an upload unit that receives product data describing a product from a seller terminal, performs pre-processing of text included in the product data, and generates a word of a minimum unit after morpheme analysis through POS tagging, Based on the similarity, the analysis unit that analyzes the generated word, the execution unit that performs the text analysis modeled using the classifier after formalizing the analyzed text into a matrix, and the previously stored Q&A data set based on the analyzed text. It includes a recommendation unit that is extracted and recommended to the seller terminal.

Another embodiment of the present invention is an upload unit that receives product data describing a product, pre-processes text included in the product data, generates a word of the smallest unit after morpheme analysis through POS tagging, and generates An analysis unit that analyzes word phrases, an execution unit that performs text analysis modeled using a classifier after formalizing the analyzed text into a matrix, and a seller by extracting a previously stored Q&A data set based on the analyzed text based on similarity A Q&A recommendation service providing server including a recommendation unit recommended to a terminal, and product data describing the product are uploaded, and the Q&A data set recommended from the Q&A recommendation service providing server is received and set as a Q&A format of the shopping mall platform. Or a seller terminal for editing the received Q&A data set.

According to any one of the above-described problem solving means of the present invention, when a shopping mall operator uploads a product to the platform, it recommends a Q&A data set required for the uploaded product, so that the shopping mall operator does not have to create question-and-answer data individually. , Customers do not have to wait for a call to an agent because they cannot find the Q&A they want, and the business operator can continuously increase the quality and quantity of Q&A by adding insufficient or additional parts in the data set. It can provide input for learning, and the recommendation model is retrained while evolving itself, so the more you use it, the more you can enrich the database of Q&A. In addition, when you introduce a chatbot, the number of queries that can be answered by the chatbot increases. By reducing the number of connections, the operator's workload can be lowered, and the customer's emotional quality can be improved as the customer can receive an answer right away.

1 is a view for explaining a system for providing a Q&A recommendation service for a shopping mall using a Q&A data set evolving through learning according to an embodiment of the present invention.
FIG. 2 is a block diagram illustrating a Q&A recommendation service providing server included in the system of FIG. 1.
3 is a view for explaining an embodiment in which a shopping mall Q&A recommendation service using a Q&A data set evolving through learning according to an embodiment of the present invention is implemented.
FIG. 4 is a diagram illustrating a process of transmitting and receiving data between components included in a system for providing a Q&A recommendation service using a Q&A data set evolving through learning of Fig. 1 according to an embodiment of the present invention.
5 is an operation flowchart illustrating a method of providing a shopping mall Q&A recommendation service using a Q&A data set evolving through learning according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily implement the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and similar reference numerals are assigned to similar parts throughout the specification.

Throughout the specification, when a part is said to be "connected" to another part, this includes not only "directly connected" but also "electrically connected" with another element interposed therebetween. . In addition, when a part "includes" a certain component, it means that other components may be further included, and one or more other features, not excluding other components, unless specifically stated to the contrary. It is to be understood that it does not preclude the presence or addition of any number, step, action, component, part, or combination thereof.

The terms "about", "substantially" and the like, as used throughout the specification, are used in or close to the numerical value when manufacturing and material tolerances specific to the stated meaning are presented, and are used in the sense of the present invention. To assist, accurate or absolute figures are used to prevent unfair use of the stated disclosure by unscrupulous infringers. As used throughout the specification of the present invention, the term "step (to)" or "step of" does not mean "step for".

In the present specification, the term "unit" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. Further, one unit may be realized using two or more hardware, or two or more units may be realized using one hardware.

In this specification, some of the operations or functions described as being performed by the terminal, device, or device may be performed instead in a server connected to the terminal, device, or device. Likewise, some of the operations or functions described as being performed by the server may also be performed by a terminal, device, or device connected to the server.

In this specification, some of the operations or functions described as mapping or matching with the terminal means mapping or matching the unique number of the terminal or the identification information of the individual, which is the identification information of the terminal. Can be interpreted as.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating a system for providing a Q&A recommendation service for a shopping mall using a Q&A data set evolving through learning according to an embodiment of the present invention. Referring to FIG. 1, a shopping mall Q&A recommendation service providing system 1 using a Q&A data set evolving through learning includes at least one seller terminal 100, and a Q&A recommendation service providing server 300. can do. However, since the shopping mall Q&A recommendation service providing system 1 using the Q&A data set evolving through learning of Fig. 1 is only an embodiment of the present invention, the present invention is limitedly interpreted through Fig. 1 no.

In this case, each component of FIG. 1 is generally connected through a network 200. For example, as shown in FIG. 1, at least one seller terminal 100 may be connected to a Q&A recommendation service providing server 300 through a network 200. In addition, the Q&A recommendation service providing server 300 may be connected to at least one seller terminal 100 through the network 200.

Here, the network refers to a connection structure in which information exchange is possible between respective nodes such as a plurality of terminals and servers, and examples of such networks include RF, 3rd Generation Partnership Project (3GPP) network, and Long Term (LTE). Evolution) network, 5GPP (5th Generation Partnership Project) network, WIMAX (World Interoperability for Microwave Access) network, Internet, LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network) , Personal Area Network (PAN), Bluetooth (Bluetooth) network, NFC network, satellite broadcasting network, analog broadcasting network, Digital Multimedia Broadcasting (DMB) network, and the like, but are not limited thereto.

In the following, the term “at least one” is defined as a term including the singular number and the plural number, and even if the term “at least one” does not exist, each component may exist in the singular or plural, and may mean the singular or plural. It will be self-evident. In addition, it will be possible to change according to the embodiment that each component is provided in a singular or plural.

The at least one seller terminal 100 may be a terminal that uploads product data using a web page, an app page, a program, or an application related to a shopping mall Q&A recommendation service using a Q&A data set evolving through learning. At this time, the at least one seller terminal 100 may be a terminal that receives a set of recommended Q&A based on the uploaded product data. In addition, the at least one seller terminal 100 may be a terminal for inserting the received Q&A set into a page where product data is uploaded as it is, or adding, deleting, or modifying a Q&A set.

Here, the at least one seller terminal 100 may be implemented as a computer capable of accessing a remote server or terminal through a network. Here, the computer may include, for example, a navigation system, a notebook equipped with a web browser, a desktop, a laptop, and the like. At this time, the at least one seller terminal 100 may be implemented as a terminal capable of accessing a remote server or terminal through a network. At least one vendor terminal 100 is, for example, a wireless communication device with guaranteed portability and mobility, and includes navigation, personal communication system (PCS), global system for mobile communications (GSM), personal digital cellular (PDC), PHS(Personal Handyphone System), PDA(Personal Digital Assistant), IMT(International Mobile Telecommunication)-2000, CDMA(Code Division Multiple Access)-2000, W-CDMA(W-Code Division Multiple Access), Wibro(Wireless Broadband Internet) ) All types of handheld-based wireless communication devices such as terminals, smartphones, smartpads, and tablet PCs may be included.

The Q&A recommendation service providing server 300 may be a server that provides a shopping mall Q&A recommendation service web page, an app page, a program, or an application using a Q&A data set evolving through learning. In addition, the Q&A recommendation service providing server 300 may be a server that receives product data uploaded from the seller terminal 100. In addition, the Q&A recommendation service providing server 300 may be a server that analyzes product data uploaded from the seller terminal 100, and extracts a Q&A data set corresponding to the analysis result and recommends it to the seller terminal 100. It can be a server that does. To this end, the Q&A recommendation service providing server 300 is a server that analyzes product data using an organically interconnected text analysis engine, an OCR (Optical Character Recognition) text engine, a similar image search engine, and a category analysis engine. I can. In addition, the Q&A recommendation service providing server 300 uses a text analysis engine when text is included in the product data, and an OCR text engine when text is included in the product data but exists in the form of image-processed text. A server that recommends a question-and-answer data set by using a similar image search engine if there is no text or image text in the product data, and extracting a category by product name if there is no text or image Can be In addition, the Q&A recommendation service providing server 300 learns using the product uploaded from the seller terminal 100 and the edited Q&A data set when the Q&A data set recommended by the seller terminal 100 is edited. It may be a server that evolves to become a more accurate and rich Q&A data set when the Q&A data set is recommended later by retraining the model.

Here, the Q&A recommendation service providing server 300 may be implemented as a computer that can access a remote server or terminal through a network. Here, the computer may include, for example, a navigation system, a notebook equipped with a web browser, a desktop, a laptop, and the like.

2 is a block diagram illustrating a Q&A recommendation service providing server included in the system of Fig. 1, and Fig. 3 is a shopping mall Q&A using a Q&A data set evolving through learning according to an embodiment of the present invention. A diagram for describing an embodiment in which a recommendation service is implemented.

2, the Q&A recommendation service providing server 300 includes an upload unit 310, an analysis unit 320, an execution unit 330, a recommendation unit 340, an OCR engine unit 350, and a similar image. A unit 360, a category analysis unit 370, and an automatic evolution unit 380 may be included.

A shopping mall using a question-and-answer data set evolving through learning in at least one seller terminal 100 by a Q&A recommendation service providing server 300 or another server (not shown) interlocked with each other according to an embodiment of the present invention When transmitting a Q&A recommendation service application, program, app page, web page, etc., at least one seller terminal 100 uses a Q&A data set evolving through learning, a shopping mall Q&A recommendation service application, program, and app page. , Web pages, etc. can be installed or opened. In addition, a service program may be driven in at least one seller terminal 100 using a script executed in a web browser. Here, the web browser is a program that enables you to use the web (WWW: world wide web) service, and refers to a program that receives and displays hypertext described in HTML (hyper text mark-up language). For example, Netscape , Explorer, chrome, etc. In addition, the application means an application on the terminal, and includes, for example, an app that is executed on a mobile terminal (smartphone).

Referring to FIG. 2, a Q&A recommendation service providing server 300 according to an embodiment of the present invention recommends a Q&A data set by driving the above-described four engines to be mutually linked. At this time, the text analysis engine, OCR text engine, similar image search engine, and category analysis engine will be described below, and those commonly applied are not described repeatedly.

<Text Analysis Engine>

Product data describing a product may be uploaded from the seller terminal 100. In this case, the product data may be text or an image, and the text may be text included in the image or text such as detailed description included separately from the image. In addition, the image may be an image photographing the exterior of the product or an image including text as described above.

The analysis unit 320 may pre-process the text included in the product data, generate a word of the smallest unit after morpheme analysis through part-of-speech tagging, and analyze the generated word. Here, when analyzing a word, for example, a TF-IDF (Term Frequency Inverse Document Frequency) algorithm may be used. At this time, when performing the preprocessing in the analysis unit 320, it is possible to perform space error correction, spelling error correction, stop word removal, etc. In the space error correction model, after deleting all spaces from the text included in the product data, machine learning You can use the sequential labeling method to determine the space to write using the model. The sequential labeling method determines whether there is a space between the syllables and the syllables from the beginning to the end of the sentence and attaches a cover (0 or 1 in the figure).To determine the cover, n syllables before and after the current position are used as features. , As a machine learning method, CRFs (Conditional Random Fields) or Structural SVM (Support Vector Machine), which are known to reflect transition information well in both directions (from the beginning to the current position and from the last to the current position) can be used. However, it is not limited thereto. The spelling error correction model can mainly use dictionary or statistical information.If the spelling error correction using a dictionary is not found in the corpus or does not exist in a general dictionary, it can be assumed that it is a spelling error, and it is assumed as a spelling error. The resulting character string may be replaced with a dictionary word that is close to the corresponding word by using an edit-distance or metaphone algorithm. Since this dictionary-based method has to construct all candidate misspelling words in advance, probability models based on statistical information can also be used. For example, a noise channel model, which is a method of correcting spelling errors, can be used, and the noise channel model is a type of generation model that finds words that maximize the probability of generating input noise (spelling errors). It is calculated based on the edit distance in syllables (or alphabets) between the and error words. For example, it may be calculated using the probability that the error word'acress' is generated from the correct word'actress' (= the probability that'c' is deleted from'ct' or'ct' is replaced with'c'). .

At this time, Korean is a typical agglutinative language or affixing language in which research and endings are developed. Unlike English, morphemes rather than words play an important role in sentiment analysis or sentence structure analysis. A morpheme is defined as the smallest unit of speech with a meaning, and a word consists of one or more morphemes. For example, if'pretty' is a word, the morphemes for this are'pretty (= adjective)' and'go (= concatenated ending)'. Reflecting the importance of such morphemes, in text analysis targeting Korean, it is common to use morphemes as basic units rather than words, and morphemes vectors (not word vectors mainly used in English texts as input in text analysis) morpheme vector) can be used, but it does not exclude word vectors. A morpheme vector can be said to be a vector expression for a morpheme, and can be derived by applying a basic word vector derivation mechanism to a sentence expressed in morpheme units. In this case, the word vector is a word converted into a vector of tens to hundreds of dimensions of real number elements. The word vectors converted in this way contain more than just numerical values. In fact, words with similar semantic and syntactic properties are expressed as vectors with close Euclidean distance or cosine similarity distance in vector space. For example, the Word2Vec model internally learns the vector representation of words using a neural network structure called CBOW (Continuous Bag-Of-Words) or Skip-Gram. CBOW predicts the target word from context words, for example, in the sentence “This lipstick is beautiful in (_) and has a good sustainability.”, it corresponds to (_) from context words that precede and follow (_). It is a method of predicting target words such as'color'. The Skip-Gram structure inversely predicts context words from the target word.

For example, when there is a sentence "This lipstick has a beautiful color and has good lasting power", if you use words separated by word as input to CBOW, a word vector such as'Lipstick is' or'Color degree' is used. You will get However, in this way,'Lipstick을','Lipstick이','Color','Color', and'Color is' are all recognized as different words, which inefficiently increases the number of words to consider. By the way, if this sentence is morphologically analyzed, it will be slightly different depending on the selected morpheme analyzer or the degree of segmentation, but, “Lee (= spectral sentence) lipstick (= general noun) silver (= general noun) color (= general noun) also (= auxiliary) pretty ( =adjective) high (=connection ending) persistence (= general noun) degree (= auxiliary) good (= adjective) c (= ending ending) .(=period)”. From this, a sentence expression in morpheme units such as “this lipstick has a beautiful color and good lasting power” can be obtained, and by inputting this into CBOW, a morpheme vector in morpheme units can be obtained. Here, the morpheme vector may be used as an input of a CNN (Convolutional Neural Network) deep learning model.

CNN is a typical feedforward neural network, also called a multi-layer perceptron, in which units of adjacent layers are fully-connected. In addition to this fully-connected layer, CNN has a convolution layer and a pooling layer in which only specific units of an adjacent layer are connected. In the convolution layer, a convolution operation that calculates the sum after multiplying the corresponding elements element-wise while sequentially moving a filter in the form of a weight matrix smaller in size than the image in the weight region of the image expressed in the form of a weight matrix. This happens. In addition, the pooling layer performs an operation of sequentially extracting representative values such as a maximum value or an average value for each specific size area with respect to values obtained by convolution. The convolutional layer and the pooling layer may be paired with each other and repeated several times, or a structure in which the pooling layer comes after only the convolutional layer is repeated several times is possible. Useful features are hierarchically extracted from the input image through such an operation, and the extracted features are used to classify input data into target classes through one or more fully-connected layers. When CNN is applied to a weight matrix representing a sentence as a combination of word vectors, local n-gram features can be extracted according to the window size of a filter. And based on these qualities, it is possible to extract higher-level qualities again. The convolution operation applies a filter to a word window to extract a new feature, and the filter is applied to all possible windows of the sentence to create a feature map. The pooling operation obtains a global maximum value for this feature map or a local representative value for each specific size. If several filters with different window sizes are applied, features of various features can be extracted, and the features extracted in this way are connected to the pre-combination layer and used for class classification.

In addition, the analysis unit 320 may perform a process of dividing the document into words, word embedding, and TF-IDF weighting steps after space error, spelling error, stop word removal, and the like are performed. First, CNN requires input data in a two-dimensional form. Accordingly, in order to satisfy this, the analysis unit 320 may perform word embedding of dividing the document into words and numbering each word. At this time, as the embedding method, a method of sequentially numbering the words according to the order in which they appear, or a word-level model that uses words as a basic unit or a character-level model that processes individual letters as basic units. May be. In addition, the analysis unit 320 obtains a TF-IDF weight value to indicate how important the embedded words are in the corresponding document, attaches the weight value to an array of embedded words, and attaches the weight value to an input value of one convolutional neural network. You can create and use it. Here, TF-IDF is a weight used in information retrieval and text mining, and is a statistical value indicating how important a word is in a specific document when there is a document group consisting of several documents. Word frequency (Term Frequency) is a value indicating how often a specific word appears in a document, and Inverse Document Frequency refers to a word that appears frequently in a document that is not many in other documents. The value of TF-IDF is used as the product of word frequency and reverse document frequency. For example, the word leather bag does not appear well in the one-piece Q&A data set, so the IDF value increases and can be a keyword for the document at the same time. Conversely, in the fashion category, the IDF value decreases because leather bags appear on many shopping mall sites. TF can be calculated using the total frequency of words in the document by default, but since the value can be continuously increased when the number of words increases, TF values are defined as 0 and 1 only by the occurrence of words using Boolean Frequency. can do. Alternatively, the size of the document can be resolved and the actual frequency can be reflected by taking a log of the TF value using a logarithmically scaled frequency. Finally, by using the Augmented Frequency, the maximum scale may not exceed 1 in a manner that indicates the relative frequency of words according to the document length. As a result, TF-IDF increases as the frequency of the word is high in a specific document and the number of documents containing the word in the entire document decreases. This allows you to filter out words that appear frequently in all documents.

The execution unit 330 may perform text analysis modeled using a classifier after forming the analyzed text into a matrix. In this case, a vector space model may be used. The vector space model is called a word space model in the field of text mining, and is an algebraic model representing a given text document as a vector of words. The vector space model can be used for information filtering, information search, indexing, and similarity ranking. If a word is included in a document, the word has a non-zero vector value. There may be various methods for calculating this value called the word weight, but in an embodiment of the present invention, the TF-IDF method described above is used. In this case, the classifier may be a document classification model based on a Convolution Neural Network (CNN), but is not limited thereto. CNN, as described above, is a kind of artificial intelligence deep neural network, has a structure suitable for learning two-dimensional data, can be trained through a back propagation algorithm, and is included in a pre-built database. It can be used to search the question-and-answer data set using the analyzed text and make recommendations to sellers based on similarity.

The recommendation unit 340 may extract a previously stored Q&A data set based on the analyzed text based on the similarity and recommend it to the seller terminal 100. For example, if the product detail text for "Women's One Piece" was analyzed, and as a result of the analysis, three Q&A sets previously stored in "Women's One Piece" were extracted, and that the similarity was 90%, 80%, and 70%, respectively, Q&A sets can be sorted and recommended in the order of high similarity.

<OCR text engine>

The OCR engine unit 350 uploads product data describing a product from the seller terminal 100 in the upload unit 310, and then when there is no text in the product data and only a product image exists, the product image is OCR. (Optical Character Recognition) Can be converted into text by applying a text-based image analysis model. In most shopping mall pages, at least a few to dozens of product images are uploaded on one page, and only text is included in each product image, and text is rarely described separately. For example, 0061 (product number)-jasmine dress (brand name)-FREE (size)-beige (color) is simply uploaded as an image along with the product image as a text in the form of an image on the thumbnail. , Lining, washing method, and material are often included in the imaged product image. Accordingly, when the product image + product description text are all imaged, the OCR engine unit 350 first performs an operation for reading the text from the image file.

OCR stands for optical character recognition and is a method of recognizing optically processed characters. Unlike online recognition, which recognizes characters expressed on a computer, optical recognition can be performed offline after recording or printing is complete. Hand-printed or printed characters can all be recognized, but accuracy depends on the quality of the entered document. It depends directly. By digitizing an analog document using an optical scanner, the text area can be extracted through the process of dividing the text area into each symbol, and the pre-processing process can proceed to remove the noise of the symbol to facilitate text extraction, After the text is extracted, each test is compared and the words and numbers of the original text are reconstructed. At this time, the OCR text-based image analysis model largely performs a process of text detection and text recognition. The former can use a deep learning CTPN (Connectionist Text Proposal Network) model that finds the location of text from an image. Here, the CTPN can be largely composed of a CNN model for distinguishing between text and non-text in an image, and a recurrent neural network (RNN) model that specifies the text estimation region, and the region estimated as text is divided into a plurality of pixels as a vertical segmentation plane Through the split CNN model, it is possible to determine whether each split plane is close to the text, and check whether the split plane on both sides is connected text through the RNN. In an embodiment of the present invention, a structure of VGG16, Bidirectional Long Short-Term Memory (BLSTM), and FC may be used.

VCC16 is a program developed by Oxford University. The input is an RGB image, and the structure includes 8 to 16 convolutional layers and 3 fully-connected layers. It is not limited. At this time, VCC16 is used to extract features of an image, and BLSTM (Bidirectional Long Short-Term Memory) is an algorithm used to classify and predict time series data as a type of cyclic neural network (RNN). At this time, the BLSTM can perform the task by connecting two LSTM outputs that process a forward state sequence from left to right and a reverse state sequence from right to left. One-way LSTM is a context from a past time instance. While only information is considered, BLSTM can learn using both past and future contextual information transmitted in the forward and reverse directions. Here, LSTM is a kind of recurrent neural network with a structure to solve the gradient loss problem that occurs during learning in the RNN method, and maintains the potential long-term memory dependence. Thus, to classify, process, and predict time series signals, LSTMs can learn from history. The LSTM remembers the previous state and special memory cells that have their own connection to the cyclic concealment layer to maintain their state over time, and three gates are used to control the flow of information to the inputs and outputs of each layer. It can have structures (input gate, forget gate and output gate), and an LSTM with such a cyclic output layer can capture the contextual information of the input text together. In addition, a result of learning the temporal relationship of time series data in the BLSTM can be obtained through the above-described fully connected layer.

The latter, text recognition, may use a Convolution Recurrent Neural Network (CRNN) model that extracts text images as text. At this time, the CRNN is a deep learning structure that combines CNN and RNN, and can be used when extracting data where both spatial and temporal information are important, such as video. For example, after extracting feature information of a product image uploaded to a shopping mall page through a CNN, the extracted value may be used as an input of an RNN to predict and extract a text sequence of data. The CRNN according to an embodiment of the present invention may be composed of a convolution layer, a recurrent layer, and a transcription layer, and after extracting a feature sequence from an image input from the convolution layer, , In the circular layer, a value is predicted for each frame in the feature sequence, and the sequence of predicted values is rewritten as a correct answer label through the transfer layer to finally extract the text. And, unlike CTPN (Connectionist Text Proposal Network), since CRNN learns based on the image of the text area, not the whole image, when the text in the image is extracted, an entity name recognition process is required to classify it by field existing in the product image. However, BLSTM, CNNs, and CRF (Conditional Random Field) models can be further used, and learning can be performed based on characters and corresponding entity names.

Named Entity refers to a word that can identify an entity from a previously predefined set of other entities with similar properties, or a phrase that can no longer be resolved within a sentence. The entity name recognition process refers to the process of identifying entity names expressed in documents, and is a process that must precede the process of extracting information through the relationship between entity names. In the product page, it is common for the product number, product name, size, and color to be listed with predefined attributes at a certain location. Recognition of entity names is classified as a kind of sequence labeling problem for information in a sentence. By learning which attribute has an entity name in a location, attribute information of each product can be obtained from a clothing or fashion page where dozens of products are registered on one page. If the CRF-based entity name recognition method of a linear linear chain is used, the performance of entity name recognition can be improved by maximizing the Log-Likelihood value, which is a conditional probability for string information. At this time, the deep learning technique that shows good performance in the continuous labeling problem, LSTM (Long-Short Term Unit), is used in both directions, and a BLSTM-CRF model in the form of attaching a conditional random field is constructed to recognize the entity name. You can also do it. In entity name recognition, entity name dictionary information including entity name information for each entity type is used as a major feature. Based on the BLSTM model mentioned in the text-based engine, the mapping information for the entity name dictionary can be configured in a binary vector format and merged into the input vector.

<Similar image search engine>

The similar image unit 360, after receiving product data describing the product from the seller terminal 100 in the upload unit 310, does not have text in the product data, and only product images, among the previously stored database. At least one feature of a product image is extracted using at least one model learned by deep learning, the extracted at least one feature is expressed in a vector space, and a similarity distance is calculated. A similar image similar to a product image may be searched using a calculating algorithm or a clustering algorithm, and the similar image unit 360 may extract a text previously mapped to the searched similar image. Here, for deep learning, ImageNet can be used, transfer learning can be used to extract features of an image, and DensNet, Resnet50, VGG16, Inceptionv3, Xception, etc. can be used. Here, the algorithm for calculating the similarity distance may be a K-nearest Neighbor algorithm, and the clustering algorithm may be a K-means clustering algorithm.

At this time, transfer learning is a method that uses deep learning as a feature extractor and learns another model with only the extracted features. When creating a new model using an existing model, it is a method of speeding up learning and improving prediction. With this completed model (Pre-Training Model), transfer learning is a method of learning by giving fine adjustments to the target learning, and this re-learning process of a neural network is called fine-tuning, and the actual CNN is constructed. In most cases, training is not performed from the beginning (random initialization), and large datasets such as ImageNet can be used. In summary, transfer learning is a learning method that overcomes the limitations of insufficient training data and efficiently extracts feature maps by initializing with values learned from a large amount of image data instead of random initialization of the convolutional layer.

The K-nearest neighbor algorithm means that the nearest neighbors are used. This method is a very intuitive way to classify unlabeled samples according to the similarity between samples in the training data set. In other words, if a sample without a label is given, it finds the sample with the closest label in the training data set and assigns it to the class with the most frequency in the subset. The K-nearest neighbor algorithm is simple because it only needs a constant, a sample of a labeled training data set, and a distance measure. At this time, the subset of data in the same class represented by each data in the classification problem can be calculated as a Euclidean distance, the middle of the distance having the largest distance value in the same class and the smallest distance value among other classes. You can determine the distance by value.

<Category Analysis Engine>

The category analysis is based on the premise that there may be a case in which the seller does not provide text or images in the detailed content of the product in the shopping mall and there is no image of the product. In this case, the point is that the product name (product name) used in the shopping mall itself can extract the category of the product, and if the category can be extracted, the Q&A data set mapped to the corresponding category can be extracted. To this end, the category analysis unit 370 uploads product data describing the product from the seller terminal 100 in the upload unit 310, and then the text and image do not exist in the product data, and the product name indicating the product. If only exists, preprocessing including removal of the stop-word included in the product name can be performed. In addition, the category analysis unit 370 performs one-hot encoding on the text included in the product name on which the pre-processing has been performed, based on the syllable, and learns the text included in the product name changed into a vector by machine learning. A category prediction may be performed through a classification model based on a text input as data and input as training data, and a text stored by pre-mapped to the predicted category may be extracted. Here, the classification model may be a deep neural network classification model.

In this case, the deep neural network (DNN) is an artificial neural network (ANN) composed of a plurality of hidden layers between an input layer and an output layer. DNN can model a complex non-linear relationship like a general artificial neural network, and a neural network operation method may include a feed-forward network (FFN), backpropagation, and an RNN. For example, data includes numeric data and text data or categorical data. Since machine learning or deep learning algorithms can only understand numerical data, they transform the data into a form that machines can understand, while categorical data transforms it into a one-hot encoding form. One-hot encoding means that only one data is changed to 1 and the rest are filled with 0. For example, assuming that apples, pears, and persimmons are included in the fruit category, each fruit is apple and pear. Create a column with, persimmon, and mark only the fruits as 1, and mark the remaining fruits as 0. For example, assuming that entity name recognition is performed from a product name, there are many words that are difficult to say that it contains knowledge or meaning among the extracted entity names, and the dimension varies in the vectorization process due to the difference in the number of entity names extracted for each category. There is. Therefore, if you set a constraint that matches the dimension by limiting to the high-order number of entity names with high frequency of occurrence for each category, as a result, entity names extracted from a number of categories can be vectorized through one-hot encoding, and input for learning. The data is complete.

At this time, when test data is input to the trained DNN model, the behavior of neurons can be divided into two categories based on the result value output from the neurons of the neural network. The first is when the result value falls within the expected range and is said to be located in the main case area. The second is when the result value falls within the unexpected range, and it is said to be located in the corner case area. Here, the neuron of the DNN has a structure in which a result value is derived through an activation function using input data, weight, and bias. In the case of a trained DNN model, the weight and bias are fixed, and only the input data can affect the neuron's result value, and it is said that the neuron is activated when the neuron reacts by the input data, that is, when the result value changes. When activated on data, categories can be predicted. In other words, if it belongs to the section where the result value of the neuron is divided by the input data, it can be considered that the section is activated, so that the category can be predicted. Here, when neurons are not activated, that is, corner cases are excluded.

The four engines described above are organically connected while performing a complementary role, and regardless of what kind of product data the seller uploads, even if he just writes the product name, it can recommend appropriate Q&A, and the seller does not answer one by one. There is no need, and consumers do not have to try to find the answers they want from product inquiries and answers listed in a list type.

The automatic evolution unit 380 extracts a pre-stored Q&A data set based on the text analyzed by the recommendation unit 340 and recommends it to the seller terminal 100, and then makes a query from the seller terminal 100. When the response data is added to the previously stored Q&A data set or the previously stored Q&A data is transformed or deleted, the added, modified, or deleted data can be set as input data for retraining the text-based model. In addition, the automatic evolution unit 380 may periodically or real-time retrain by inputting the input data into the text-based model.

Hereinafter, the operation process according to the configuration of the Q&A recommendation service providing server of FIG. 2 will be described in detail with reference to FIG. 3 as an example. However, it will be apparent that the embodiment is only any one of various embodiments of the present invention, and is not limited thereto.

Referring to FIG. 3, (a) when product data is uploaded from the seller terminal 100, the Q&A recommendation service providing server 300 (b) analyzes text or images included in the product data, and is stored in advance. Databaseized Q&A is searched and recommended to the seller terminal 100 in the order of similarity. At this time, the seller terminal 100 may use it as it is, or may edit (add, delete, or modify) according to the product it sells. In this case, when editing occurs in the seller terminal 100, the product data analysis result and the edited contents of the seller terminal 100 are re-learned, thereby providing a richer Q&A recommendation list. As shown in (c), it is possible to build a database that automatically evolves by accumulating and learning not only the fashion field, but also the Q&A data set for each category or field, and the quality and quantity of the database will increase spontaneously with the direct participation of the user, the seller. I can.

For information that is not explained about the method of providing a shopping mall Q&A recommendation service using the Q&A data set evolving through the learning of Figs. 2 and 3, a shopping mall using a Q&A data set that evolves through learning through Fig. Since the Q&A recommendation service providing method is the same as the description or can be easily inferred from the description, the following description will be omitted.

FIG. 4 is a diagram illustrating a process of transmitting and receiving data between components included in a system for providing a Q&A recommendation service using a Q&A data set evolving through learning of Fig. 1 according to an embodiment of the present invention. Hereinafter, an example of a process in which data is transmitted/received between each component will be described with reference to FIG. 4, but the present application is not limitedly interpreted as such an embodiment, and is illustrated in FIG. 4 according to various embodiments described above. It is apparent to those skilled in the art that the process of transmitting and receiving data may be changed.

Referring to FIG. 4, when the Q&A recommendation service providing server 300 receives product data from the seller terminal 100 (S4100), it checks whether text is included (S4200), and if there is no text, the product image It is checked whether is present (S4300), and if the text exists in the product image (S4400), text is extracted from the image using the OCR image text analysis model (S4710). And, if the product image exists but the text does not exist in the form of an image, the Q&A recommendation service providing server 300 searches for the most similar image among the previously established images using the similar image search model, Is detected (S4600, S4720). In addition, the Q&A recommendation service providing server 300 analyzes the product name using category analysis when there is no product text and no product image, and thus no information to recommend Q&A exists (S4700).

On the other hand, if the text is defined as containing text in step S4200 or the product image exists in step S4300 and text is extracted, proceed to step S4800 to perform preprocessing (S4800), and after performing POS tagging and morpheme analysis (S4810). ), using the text analysis algorithm (S4820), it results in a step of checking whether a Q&A dataset mapped to any one of text, image, and category exists (S4940). In addition, if the data set is extracted from the text that is not imaged or only the product name exists, the process proceeds to a similar step to S4840. If the Q&A recommendation service providing server 300 does not find the Q&A data set mapped to the text, or if the similarity does not satisfy a preset reference value, request new data from the seller or a manager terminal (not shown) The information is notified and the insufficient information is filled in (S4850), and the newly entered value is input as learning data and used as data for re-learning (S4851). At this time, if the S4850 fails to recommend, the seller has to write a Q&A from start to finish, so even if the similarity does not satisfy the reference value, it is not excluded to provide it to the seller.

On the other hand, when the Q&A recommendation service providing server 300 adds, deletes, or transforms the Q&A data set recommended from the seller terminal 100 (S4910), the database is retrained by inputting it as learning data. It can be enriched and further evolved by itself (S4920).

The order between the above-described steps S4100 to S4920 is only an example and is not limited thereto. That is, the order of the above-described steps (S4100 to S4920) may be mutually changed, and some of the steps may be executed or deleted at the same time.

The matters that are not described about the method of providing a shopping mall Q&A recommendation service using the Q&A data set evolving through the learning of FIG. 4 are previously described in Figs. 1 to 3, a shopping mall using a Q&A data set that evolves through learning. Since the Q&A recommendation service providing method is the same as the description or can be easily inferred from the description, the following description will be omitted.

5 is a flowchart illustrating a method of providing a shopping mall Q&A recommendation service using a Q&A data set evolving through learning according to an embodiment of the present invention. 5, the Q&A recommendation service providing server receives product data describing a product from a seller terminal (S5100), performs pre-processing of the text included in the product data, and performs a morpheme analysis through part-of-speech tagging. A unit word is generated, and the generated word is analyzed (S5200).

The Q&A recommendation service providing server forms the analyzed text into a matrix, then performs text analysis modeled using a classifier (S5300), and extracts a previously stored Q&A data set based on the analyzed text based on similarity. It is recommended to the seller terminal (S5400).

The matters that are not described about the method of providing a shopping mall Q&A recommendation service using the Q&A data set evolving through the learning of FIG. 5 are described in the shopping mall using the Q&A data set evolving through learning through FIGS. 1 to 4 above. Since the Q&A recommendation service providing method is the same as the description or can be easily inferred from the description, the following description will be omitted.

The method for providing a shopping mall Q&A recommendation service using a Q&A data set evolving through learning according to an embodiment described with reference to FIG. 5 includes instructions executable by a computer such as an application or program module executed by a computer. It can also be implemented in the form of a recording medium. Computer-readable media can be any available media that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media. Further, the computer-readable medium may include all computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

The method for providing a shopping mall Q&A recommendation service using a Q&A data set evolving through learning according to an embodiment of the present invention described above includes an application basically installed in a terminal (this is included in a platform or operating system basically installed in the terminal). It can be executed by an application (that is, a program) installed directly on the master terminal through an application providing server such as an application store server, an application, or a web server related to the service. have. In this sense, the method for providing a shopping mall Q&A recommendation service using a Q&A data set evolving through learning according to an embodiment of the present invention described above is an application (i.e., a program) installed in a terminal or directly installed by a user. And recorded on a computer-readable recording medium such as a terminal.

The above description of the present invention is for illustrative purposes only, and those of ordinary skill in the art to which the present invention pertains can understand that it is possible to easily transform it into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not limiting. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as being distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the claims to be described later rather than the detailed description, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. do.

Claims (7)

In the Q&A recommendation service provision method executed in the Q&A recommendation service provision server,
Receiving product data describing the product from the seller terminal;
Pre-processing the text included in the product data, analyzing the morpheme through part-of-speech tagging, generating a word of a minimum unit, and analyzing the generated word;
Performing text analysis modeling using a classifier after forming the analyzed text into a matrix; And
Including; extracting a previously stored Q&A data set based on the analyzed text based on similarity and recommending it to the seller terminal; Including,
After the step of uploading product data describing a product from the seller terminal, when there is no text in the product data and only a product image exists, an OCR (Optical Character Recognition) text-based image analysis model is applied to the product image. And converting the text into text; the OCR text-based image analysis model further comprises: text detection for finding a location of text from an image and text recognition for extracting a text image as text Including,
After the step of uploading product data describing a product from the seller terminal, when there is no text in the product data and only a product image exists, the product image using at least one model of deep learning from a previously stored database Extracting at least one feature of; Expressing the extracted at least one feature in a vector space and searching for a similar image similar to the product image using an algorithm or a clustering algorithm for calculating a similarity distance; The method of providing a shopping mall Q&A recommendation service using a Q&A data set evolving through learning, further comprising: extracting a text pre-mapped to the searched similar image.
delete delete The method of claim 1,
After the step of uploading product data describing a product from the seller terminal,
Performing pre-processing including removing a stop-word included in the product name when text and image are not present in the product data and only a product name indicating the product exists;
Performing one-hot encoding of the text included in the pre-processed product name based on the syllable, and inputting the text included in the product name changed into a vector as machine learning training data;
Performing category prediction through a classification model based on the text input as the training data;
Extracting previously mapped and stored text to the predicted category;
The method of providing a shopping mall Q&A recommendation service using a Q&A data set evolving through learning.
The method of claim 1,
After the step of extracting a previously stored Q&A data set based on the analyzed text based on similarity and recommending it to the seller terminal,
Input data for retraining a text-based model from the added, modified or deleted data when the seller terminal adds Q&A data to the previously stored Q&A data set or transforms or deletes the previously stored Q&A data Setting to;
Inputting the input data into the text-based model and retraining periodically or in real time;
The method of providing a shopping mall Q&A recommendation service using a Q&A data set evolving through learning.
delete delete

Images