KR102436549B1 - Method and apparatus for automatically generating training dataset for faq and chatbot based on natural language processing using deep learning - Google Patents

Abstract

In the present specification, a method, in which a Question and Answer (Q&A) generating device generates learning data for frequently asked questions (FAQ) and a chatbot, may include the steps of: collecting and preprocessing document data for automatically generating Q&A data, wherein the document data includes text files related to business regulations; automatically generating, based on the preprocessed document data, the Q&A data by using an artificial intelligence model based on natural language processing, wherein the preprocessed document data includes contents for automatically generating the Q&A data, and the Q&A data includes questions and answers in pairs; and based on the Q&A data, generating learning data for 1) the FAQ and 2) the chatbot. According to the present invention, key answers are automatically extracted and corresponding questions are automatically created.

Description

Method and device for automatically generating learning data for FAQ and chatbot based on natural language processing using deep learning

This specification provides a method for implementing a pipeline and service of an artificial intelligence model for automatically generating learning data for frequently asked questions (FAQs) and chatbots based on natural language processing using deep learning and a device for the same it's about

In order for service providers to create and update machine learning and deep learning-based FAQs, or to provide chatbot and chatbot builder services, a large amount of labeling is done to understand the natural language text data entered by the user. of learning data. For example, through data accumulated in a local or cloud database, a process for constructing high-quality learning data manually by a person is required.

Labeling such learning data is not an easy task unless the service provider has an annotation team. In particular, the process of building chatbot learning data based on raw data from the database or the company's work rulebook is time-consuming and costly because the annotator must manually input the chatbot user's expected questions and corresponding answers. However, it is not efficient in terms of manpower input.

An object of the present specification is to provide a method and an apparatus for automatically extracting key answers from previously prepared documents such as business rules and FAQs uploaded to a server by a service provider and automatically generating a corresponding question.

The technical problems to be achieved by the present specification are not limited to the technical problems mentioned above, and other technical problems not mentioned are clear to those of ordinary skill in the art to which this specification belongs from the detailed description of the following specification will be able to be understood

An aspect of the present specification is a method for automatically generating Q & A (Question and Answer) device to generate learning data for frequently asked questions (FAQ) and chatbot, document data for automatically generating Q & A data is collected and pre-processing; wherein the document data includes a text file related to business rules; Based on the pre-processed document data, automatically generating the Q&A data using a natural language processing-based artificial intelligence model; wherein the pre-processed document data includes content for automatically generating the Q&A data. wherein the Q&A data includes a question and an answer in pairs; Based on the Q&A data, 1) generating the FAQ and 2) learning data for the chatbot; may include.

In addition, when the Q&A data is edited, re-learning the AI model using the edited Q&A data; may further include.

In addition, the collecting and pre-processing of the document data may include: classifying a file extension of the document data; parsing the text of the document data based on the file extension; and processing the text in units of sentences. may include.

In addition, the step of automatically generating the Q&A data may include tokenizing the pre-processed document data; extracting a summary sentence from the tokenized document data; generating the question based on the summary sentence; and generating the Q&A data by using the summary sentence as the answer.

Also, extracting the summary sentence may use one or more windows.

Also, the summary sentence may be based on an extractive text summarization technique of extracting, combining, and summarizing key sentences or words from the preprocessed document data.

Another aspect of the present specification is a memory containing a natural language processing-based artificial intelligence model in an automatic Q & A (Question and Answer) generation device for generating learning data for FAQ (Frequently asked questions) and chatbot ; a communication unit for communicating with an external electronic device; and a processor for functionally controlling the memory and the communication unit. Including, the processor collects and pre-processes document data for automatically generating Q&A data, the document data includes a text file related to business rules, and based on the pre-processed document data, the Using an artificial intelligence model, the Q&A data is automatically generated, the pre-processed document data includes contents for automatically generating the Q&A data, and the Q&A data includes a question and an answer in pairs, Based on the Q&A data, 1) the FAQ and 2) learning data for the chatbot may be generated.

According to the embodiment of the present specification, the time and cost of building learning data by automatically extracting key answers from existing documents such as work rules and FAQs uploaded by the service provider to the server and automatically generating corresponding questions , it can increase productivity and work efficiency in terms of manpower input.

The effects obtainable in the present specification are not limited to the above-mentioned effects, and other effects not mentioned will be clearly understood by those of ordinary skill in the art to which this specification belongs from the description below. .

1 is a block diagram illustrating an electronic device related to the present specification.
2 is a block diagram of an AI device according to an embodiment of the present specification.
3 is an example of a text-to-text framework applicable to the present specification.
4 is an embodiment of an automatic Q&A generating apparatus that can be applied to the present specification.
5 and 6 are examples of collection and pre-processing of document data applicable to the present specification.
7 is an example of automatic generation of Q&A data to which this specification can be applied.
8 is an example of modification and/or addition of Q&A data to which this specification may be applied.
9 is an example of artificial intelligence model re-learning to which this specification can be applied.
10 is an example of FAQ data and/or chatbot learning data to which this specification can be applied.
BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are included as a part of the detailed description to help the understanding of the present specification, provide embodiments of the present specification, and together with the detailed description, explain the technical features of the present specification.

Hereinafter, the embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings, but the same or similar components are assigned the same reference numbers regardless of reference numerals, and redundant description thereof will be omitted. The suffixes "module" and "part" for components used in the following description are given or mixed in consideration of only the ease of writing the specification, and do not have distinct meanings or roles by themselves. In addition, in describing the embodiments disclosed in the present specification, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in the present specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in the present specification, and the technical spirit disclosed in this specification is not limited by the accompanying drawings, and all changes included in the spirit and scope of the present specification , should be understood to include equivalents or substitutes.

Terms including ordinal numbers such as first, second, etc. may be used to describe various elements, but the elements are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

When a component is referred to as being “connected” or “connected” to another component, it may be directly connected or connected to the other component, but it is understood that other components may exist in between. it should be On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the other element does not exist in the middle.

The singular expression includes the plural expression unless the context clearly dictates otherwise.

In the present application, terms such as “comprises” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It is to be understood that this does not preclude the possibility of the presence or addition of numbers, steps, operations, components, parts, or combinations thereof.

1 is a block diagram illustrating an electronic device related to the present specification.

The electronic device 100 includes a wireless communication unit 110 , an input unit 120 , a sensing unit 140 , an output unit 150 , an interface unit 160 , a memory 170 , a control unit 180 , and a power supply unit 190 . ) and the like. The components shown in FIG. 1 are not essential for implementing the electronic device, and thus the electronic device described herein may have more or fewer components than those listed above.

More specifically, the wireless communication unit 110 among the components, between the electronic device 100 and the wireless communication system, between the electronic device 100 and another electronic device 100, or the electronic device 100 and an external server It may include one or more modules that enable wireless communication between them. Also, the wireless communication unit 110 may include one or more modules for connecting the electronic device 100 to one or more networks.

The wireless communication unit 110 may include at least one of a broadcast reception module 111 , a mobile communication module 112 , a wireless Internet module 113 , a short-range communication module 114 , and a location information module 115 . .

The input unit 120 includes a camera 121 or an image input unit for inputting an image signal, a microphone 122 or an audio input unit for inputting an audio signal, and a user input unit 123 for receiving information from a user, for example, , a touch key, a push key, etc.). The voice data or image data collected by the input unit 120 may be analyzed and processed as a user's control command.

The sensing unit 140 may include one or more sensors for sensing at least one of information within the electronic device, surrounding environment information surrounding the electronic device, and user information. For example, the sensing unit 140 may include a proximity sensor 141, an illumination sensor 142, an illumination sensor, a touch sensor, an acceleration sensor, a magnetic sensor, and gravity. Sensor (G-sensor), gyroscope sensor, motion sensor, RGB sensor, infrared sensor (IR sensor: infrared sensor), fingerprint sensor (finger scan sensor), ultrasonic sensor , optical sensors (eg, cameras (see 121)), microphones (see 122), battery gauges, environmental sensors (eg, barometers, hygrometers, thermometers, radiation sensors, It may include at least one of a thermal sensor, a gas sensor, etc.) and a chemical sensor (eg, an electronic nose, a healthcare sensor, a biometric sensor, etc.). Meanwhile, the electronic device disclosed in the present specification may combine and utilize information sensed by at least two or more of these sensors.

The output unit 150 is for generating an output related to visual, auditory or tactile sense, and includes at least one of a display unit 151 , a sound output unit 152 , a haptip module 153 , and an optical output unit 154 . can do. The display unit 151 may implement a touch screen by forming a layer structure with the touch sensor or being formed integrally with the touch sensor. Such a touch screen may function as the user input unit 123 providing an input interface between the electronic device 100 and the user, and may provide an output interface between the electronic device 100 and the user.

The interface unit 160 serves as a passage with various types of external devices connected to the electronic device 100 . Such an interface unit 160, a wired / wireless headset port (port), an external charger port (port), a wired / wireless data port (port), a memory card (memory card) port, for connecting a device equipped with an identification module It may include at least one of a port, an audio I/O (Input/Output) port, a video I/O (Input/Output) port, and an earphone port. In response to the connection of the external device to the interface unit 160 , the electronic device 100 may perform appropriate control related to the connected external device.

In addition, the memory 170 stores data supporting various functions of the electronic device 100 . The memory 170 may store a plurality of application programs (or applications) driven in the electronic device 100 , data for operation of the electronic device 100 , and commands. At least some of these application programs may be downloaded from an external server through wireless communication. In addition, at least some of these application programs may exist on the electronic device 100 from the time of shipment for basic functions (eg, incoming calls, outgoing functions, message reception, and outgoing functions) of the electronic device 100 . Meanwhile, the application program may be stored in the memory 170 , installed on the electronic device 100 , and driven to perform an operation (or function) of the electronic device by the controller 180 .

In addition to the operation related to the application program, the controller 180 generally controls the overall operation of the electronic device 100 . The controller 180 may provide or process appropriate information or functions to the user by processing signals, data, information, etc. input or output through the above-described components or by driving an application program stored in the memory 170 .

Also, the controller 180 may control at least some of the components discussed with reference to FIG. 1 in order to drive an application program stored in the memory 170 . Furthermore, in order to drive the application program, the controller 180 may operate at least two or more of the components included in the electronic device 100 in combination with each other.

The power supply unit 190 receives external power and internal power under the control of the control unit 180 to supply power to each component included in the electronic device 100 . The power supply unit 190 includes a battery, and the battery may be a built-in battery or a replaceable battery.

At least some of the respective components may operate cooperatively to implement an operation, control, or control method of an electronic device according to various embodiments described below. In addition, the operation, control, or control method of the electronic device may be implemented on the electronic device by driving at least one application program stored in the memory 170 .

In the present specification, the apparatus and terminal for automatically generating Q&A may include the electronic device 100 .

2 is a block diagram of an AI device according to an embodiment of the present specification.

The AI device 20 may include an electronic device including an AI module capable of performing AI processing, or a server including the AI module. Also, the AI device 20 may be included as a component of at least a part of the electronic device 100 shown in FIG. 1 to perform at least a part of AI processing together.

The AI device 20 may include an AI processor 21 , a memory 25 and/or a communication unit 27 .

The AI device 20 is a computing device capable of learning a neural network, and may be implemented in various electronic devices such as a server, a desktop PC, a notebook PC, and a tablet PC.

The AI processor 21 may learn the neural network using a program stored in the memory 25 . In particular, the AI processor 21 may generate an artificial intelligence model for natural language processing on the memory 25, and use the data that may be collected from the automatic Q&A generation device to train the artificial intelligence model. .

Such an AI model may include a T5 model in the form of a “Text-to-Text Transfer Transformer”.

3 is an example of a text-to-text framework applicable to the present specification.

Referring to FIG. 3 , a diagram of a text-to-text framework including an artificial intelligence model (eg, a T5 model) is illustrated. The Q&A automatic generation device inputs tasks that can be considered, including translation, question, answer, and classification, into the AI model as text, and the AI model generates a target text based on the inputted texts. can be trained to

The AI model may include not only the T5 model, but also other models (eg, BART, GPT-2, etc.) that utilize a decoder of Transformer that can generate text or have autoregressive characteristics.

Referring back to FIG. 2 , the AI processor 21 performing the function as described above may be a general-purpose processor (eg, CPU), but an AI-only processor (eg, GPU, graphics for artificial intelligence learning) processing unit).

The memory 25 may store various programs and data necessary for the operation of the AI device 20 . The memory 25 may be implemented as a non-volatile memory, a volatile memory, a flash-memory, a hard disk drive (HDD), or a solid state drive (SDD). The memory 25 is accessed by the AI processor 21 , and reading/writing/modification/deletion/update of data by the AI processor 21 may be performed. Also, the memory 25 may store a neural network model (eg, a deep learning model) generated through a learning algorithm for data classification/recognition according to an embodiment of the present specification.

Meanwhile, the AI processor 21 may include a data learning unit that learns a neural network for data classification/recognition. For example, the data learning unit may learn the deep learning model by acquiring learning data to be used for learning, and applying the acquired learning data to the deep learning model.

The communication unit 27 may transmit the AI processing result by the AI processor 21 to an external electronic device.

Here, the external electronic device may include other terminals and servers.

On the other hand, although the AI device 20 shown in FIG. 2 has been functionally divided into the AI processor 21, the memory 25, the communication unit 27, and the like, the above-described components are integrated into one module and the AI module Alternatively, it may be called an artificial intelligence (AI) model.

4 is an embodiment of an automatic Q&A generating apparatus that can be applied to the present specification.

Referring to FIG. 4 , the automatic Q&A generating apparatus may automatically generate Q&A data, and generate FAQ data and learning data for the chatbot.

S4010: Q&A automatic generation device collects and pre-processes document data for automatic Q&A generation.

5 and 6 are examples of collection and pre-processing of document data applicable to the present specification.

Referring to FIG. 5 , the automatic Q&A generation apparatus uploads document data for automatic Q&A generation to the server (S5010). For example, the document data for automatic Q&A generation may include a service provider-related business regulation text file, a consultation knowledge document text file, and a FAQ text file. The automatic Q&A generation device can upload document data for automatic Q&A data generation to a server for managing document data.

Referring to FIG. 6 , the apparatus for automatically generating Q&A may display a guide screen for uploading document data to the user through the terminal. The user may check a list of document data to be uploaded through the guide screen, select document data, and may upload document data to the server by inputting the document upload button 610 .

The server may manage the uploaded document data using a local and/or cloud database. For example, a unique ID may be generated for document data.

Referring back to FIG. 5 , the automatic Q&A generating apparatus classifies file extensions of document data ( S5020 ). For example, the apparatus for automatically generating Q&A may select document data for automatic generation of Q&A data according to a file extension (eg, docx, hwp, pdf, txt) of the document data.

Referring to FIG. 6 , the apparatus for automatically generating Q&A may receive a toggle button 620 for automatically generating Q&A from a user through a terminal, and classify file extensions of document data.

Referring back to FIG. 5 , the automatic Q&A generating apparatus parses the text of the document data based on the file extension ( S5030 ). For example, the apparatus for automatically generating Q&A may use an artificial intelligence model to parse text of document data to extract text representing content required for automatic generation of Q&A. Alternatively, the apparatus for automatically generating Q&A may parse the text of the document data according to a preset rule to extract text representing content required for automatic generation of the Q&A data.

The automatic Q&A generating apparatus pre-processes the text in units of sentences (S5040). For example, the automatic Q&A generator may divide text into sentence units through regular expressions, and then convert it into JSON format having the form of a key-value pair.

The automatic Q&A generation device stores the pre-processed document data (S5050). For example, a Q&A auto-generator may store JSON data in a database contained in memory. In more detail, the automatic Q&A generator may store the document's unique ID as a key.

Table 1 below is an example of JSON data.

{
document_id: unique_hash_value,
document_name: Upload document name,
content: text extracted from the document
}

S4020: Referring back to FIG. 4 , the automatic Q&A generation apparatus automatically generates Q&A data using an artificial intelligence model based on the pre-processed document data. 7 is an example of automatic generation of Q&A data to which this specification can be applied.

Referring to FIG. 7 , the apparatus for automatically generating Q&A may use the artificial intelligence model 700 to automatically generate Q&A data using pre-processed document data as an input value.

1. The automatic Q&A generation device inputs preprocessed document data into the artificial intelligence model 700 . For example, the content of the preprocessed document data may be tokenized through a BERT (Bidirectional Encoder Representations from Transformers) tokenizer. In the field of natural language processing, tokenization may mean breaking a given text into character units or meaningful strings.

2. The automatic Q&A generator can divide the tokenized vector into n groups in order using n windows. A windowing technique that can be used here may mean dividing an input vector into n pieces according to the size of the window. For example, the automatic Q&A generating apparatus may extract m summary sentences from the pre-processed document data. Using the windowing technique, the automatic Q&A generator may extract m sentences from each window instead of extracting m sentences from a long document. Through this, since the automatic Q&A generation apparatus can extract a total of m*n summary sentences, more information can be extracted from the preprocessed document data.

3. The automatic Q&A generation apparatus can extract m summary sentences from each window. For example, extraction of summary sentences may be divided into extractive text summarization and abstract text summarization according to a document summary method. In more detail, the extraction summary is a method of extracting key sentences or words from a document and combining them to summarize, and the generation summary is a method of summarizing the original text by generating a new sentence by reflecting the core context. Extractive summaries may be used in this specification, as abstract sentences will answer questions to be generated later. For example, the artificial intelligence model 700 used for the extraction summary may include BertSumExt, which transforms the Transformer-based BERT to fit the extraction summary task. However, it goes without saying that other artificial intelligence models that can be used for extraction summarization can be used in addition to the BertSumExt above.

4. The automatic Q&A generation apparatus may generate a question from the summary sentence by using the question generation network. For example, since the T5 model uses both the encoder and the decoder of the transformer, it is more suitable for solving the text generation problem than the BERT using only the encoder of the transformer. In addition, since the T5 model was designed to solve all natural language processing (NLP) tasks in a text-to-text format by integrating them, it is also the same for downstream tasks such as machine translation, Q&A, and generation summary through fine-tuning. can be used Accordingly, fine-tuning can also be applied to the AI model in the present specification. In addition, since both Bert and T5 are pre-trained with a large corpus, excellent performance can be achieved even when fine-tuning is performed with a small amount of data. For example, the automatic Q&A generation apparatus may tokenize summary sentences using the tokenizer of the T5 model, input them to the question generation network, and return them as output values in the form of strings. In this case, the returned value may be a question-type sentence with the corresponding summary sentence as the correct answer. In the present specification, T5 is used for this purpose, but other models with autoregressive characteristics or utilizing a decoder of Transformer capable of generating sentences may be used (eg, BART, GPT-2, etc.).

5. The Q&A automatic generation device can store the generated question-answer pair data in a database. For example, this question-answer pair data may be given a unique ID for identifying it.

6. Q&A automatic generation device can output question-answer pair data as Q&A data.

S4030: Referring back to FIG. 4 , the automatically generated Q&A device modifies and/or adds automatically generated Q&A data. For example, the Q&A automatic generation device may be connected to the terminal and display Q&A data to the user through an application for modifying and/or adding Q&A data, and for modifying and/or adding Q&A data from the user You can input text.

8 is an example of modification and/or addition of Q&A data to which this specification may be applied.

Referring to FIG. 8 , the apparatus for automatically generating Q&A may display a screen for correcting and/or adding Q&A data through the terminal. A screen for modifying and/or adding Q&A data may include a question search panel 810 , an answer display panel 820 , and an editing panel 830 .

The question search panel 810 displays question information of Q&A data. For example, a user may retrieve a question from Q&A data in a database. When the user selects a specific question from the question list displayed on the question search panel 810 , the corresponding question may be highlighted. If the user's desired question is not found, the user may add a question and, through the editing panel 830 , may add a corresponding answer.

The answer display panel 820 displays answer information of Q&A data corresponding to the selected specific question. The apparatus for automatically generating Q&A may display an answer sentence corresponding to the selected specific question on the answer display panel 820 through the terminal. Sentences representing the corresponding question-answer may be highlighted.

When the answer sentence displayed on the answer display panel 820 is selected by the user, the editing panel 830 displays a question corresponding to the answer sentence. Edit panel 830 may also display IDs and answers associated with the question. The user can check the answer sentence and edit it.

S4040: Referring back to FIG. 4 , the automatic Q&A generation apparatus retrains the artificial intelligence model by using the corrected and/or added Q&A data.

9 is an example of artificial intelligence model re-learning to which this specification can be applied.

Referring to FIG. 9 , the apparatus for automatically generating Q&A may display a screen for re-learning of the artificial intelligence model to the user through the terminal. For example, the user may select Q&A data that can be used for re-learning of the AI model, input a model learning button, and instruct the re-learning of the AI model.

S4050: Again, referring to FIG. 4 , the automatic Q&A generating device generates FAQ data and/or learning data of the chatbot for answering the user's question. The automatic Q&A generation device may generate FAQ data and/or chatbot learning data using the corrected and/or added Q&A data.

10 is an example of FAQ data and/or chatbot learning data to which this specification can be applied.

Referring to FIG. 10 , the apparatus for automatically generating Q&A may display Q&A data to the user through the terminal. Users can search for questions and delete Q&A data through the Q&A menu. Through this, the automatic Q&A generation device may generate optimized FAQ data and/or chatbot learning data.

In building and updating general machine learning and deep learning-based FAQs or providing chatbot and chatbot builder services, the artificial intelligence model must receive labeled high-quality learning data. For this, work productivity and efficiency are reduced because the user has to manually input the information.

In addition, the labor cost burden is high as manpower is input for the manual labeling of the learning data, but the automatic Q&A generation apparatus of the present specification can reduce the labor cost.

The above-described specification can be implemented as computer-readable code on a medium in which a program is recorded. The computer-readable medium includes all types of recording devices in which data readable by a computer system is stored. Examples of computer-readable media include Hard Disk Drive (HDD), Solid State Disk (SSD), Silicon Disk Drive (SDD), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. There is also a carrier wave (eg, transmission over the Internet) that includes implementation in the form of. Accordingly, the above detailed description should not be construed as restrictive in all respects but as exemplary. The scope of this specification should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of this specification are included in the scope of this specification.

In addition, although the above description has been focused on services and embodiments, these are merely examples and do not limit the present specification, and those of ordinary skill in the art to which this specification pertains will not deviate from the essential characteristics of the present service and embodiments. It can be seen that various modifications and applications not exemplified above are possible. For example, each component specifically shown in the embodiments may be implemented by modification. And differences related to such modifications and applications should be construed as being included in the scope of the present specification defined in the appended claims.

Claims (7)

In a method for generating learning data for a Q&A (Question and Answer) automatic generation device for frequently asked questions (FAQ) and chatbots,
collecting and pre-processing document data for automatically generating Q&A data;
the document data includes a text file related to business rules;
automatically generating the Q&A data using an artificial intelligence model based on natural language processing based on the pre-processed document data;
The pre-processed document data includes content for automatically generating the Q&A data,
the Q&A data includes questions and answers in pairs;
Based on the Q&A data, 1) generating the FAQ and 2) learning data for the chatbot;
including,
The step of collecting and pre-processing the document data is
classifying a file extension of the document data;
parsing the text of the document data based on the file extension; and
processing the text in units of sentences;
includes,
The step of automatically generating the Q&A data is
tokenizing the preprocessed document data;
extracting a summary sentence from the tokenized document data;
generating the question based on the summary sentence; and
generating the Q&A data using the summary sentence as the answer;
Containing, a production method.
According to claim 1,
re-learning the AI model using the edited Q&A data when the Q&A data is edited;
Further comprising a, generating method.
delete delete According to claim 1,
The step of extracting the summary sentence is
A method of generating, using one or more windows.
6. The method of claim 5,
The summary sentence is
A method for generating, based on an extractive text summarization technique of extracting, combining, and summarizing key sentences or words from the preprocessed document data.
In the Q&A (Question and Answer) automatic generation device for generating learning data for FAQ (Frequently asked questions) and chatbot,
memory containing artificial intelligence models based on natural language processing;
a communication unit for communicating with an external electronic device; and
a processor for functionally controlling the memory and the communication unit;
including,
the processor
To collect and pre-process document data for automatically generating Q&A data, classify a file extension of the document data, parse the text of the document data based on the file extension, and convert the text into a sentence processed as a unit,
The document data includes a text file related to business regulations,
To automatically generate the Q&A data based on the pre-processed document data, using the artificial intelligence model, tokenize the pre-processed document data, and extract a summary sentence from the tokenized document data; generating a question based on the summary sentence, and generating the Q&A data by using the summary sentence as an answer;
The pre-processed document data includes content for automatically generating the Q&A data,
the Q&A data includes the question and the answer in pairs,
Based on the Q&A data, 1) the FAQ and 2) the Q&A automatic generating device for generating learning data for the chatbot.

Images