CN115309888B - Method and device for generating chart abstract and training method and device for generating model - Google Patents

Method and device for generating chart abstract and training method and device for generating model Download PDF

Info

Publication number
CN115309888B
CN115309888B CN202211037048.0A CN202211037048A CN115309888B CN 115309888 B CN115309888 B CN 115309888B CN 202211037048 A CN202211037048 A CN 202211037048A CN 115309888 B CN115309888 B CN 115309888B
Authority
CN
China
Prior art keywords
information
target
chart
data
type
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202211037048.0A
Other languages
Chinese (zh)
Other versions
CN115309888A (en
Inventor
周景博
杜明轩
李宇
窦德景
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Baidu Netcom Science and Technology Co Ltd
Original Assignee
Beijing Baidu Netcom Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Baidu Netcom Science and Technology Co Ltd filed Critical Beijing Baidu Netcom Science and Technology Co Ltd
Priority to CN202211037048.0A priority Critical patent/CN115309888B/en
Publication of CN115309888A publication Critical patent/CN115309888A/en
Application granted granted Critical
Publication of CN115309888B publication Critical patent/CN115309888B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/34Browsing; Visualisation therefor
    • G06F16/345Summarisation for human users
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Biophysics (AREA)
  • Evolutionary Computation (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Machine Translation (AREA)

Abstract

The disclosure provides a method, a device, equipment and a medium for generating a chart abstract, which relate to the field of artificial intelligence, in particular to the technical fields of deep learning, natural language processing, knowledge graph and the like. The specific implementation scheme of the chart abstract generation method is as follows: extracting data from the target chart by adopting an operation function to obtain target data and attribute information of the target data; fusing the target data, the attribute information and the type information of the operation function to obtain key information of a target chart; carrying out coding treatment on the key information to obtain coding characteristics; and decoding the coding features to obtain the abstract text of the target chart.

Description

Method and device for generating chart abstract and training method and device for generating model
Technical Field
The disclosure relates to the field of artificial intelligence, in particular to the field of deep learning and knowledge maps, and particularly relates to a method for generating a chart abstract and a training method, device, equipment and medium for generating a model.
Background
With the development of computer technology and network technology, deep learning technology is widely used in a plurality of fields. For example, deep learning techniques may be employed to generate summary text for a chart from data in the chart. If the abstract text is directly generated according to the data in the chart, the problems of less information expressed by the abstract text and inaccurate abstract text exist.
Disclosure of Invention
The disclosure aims to provide a method for generating a chart abstract and a training method, device, equipment and medium for generating a model, so as to improve the expression capability of abstract text generated for a chart.
According to one aspect of the present disclosure, there is provided a method for generating a graph summary, including: extracting data from the target chart by adopting an operation function to obtain target data and attribute information of the target data; extracting data from the target chart by adopting an operation function to obtain target data and attribute information of the target data; carrying out coding treatment on the key information to obtain coding characteristics; and decoding the coding features to obtain the abstract text of the target chart.
According to another aspect of the present disclosure, there is provided a training method of generating a model, wherein the generating model includes an encoder and a decoder; the training method comprises the following steps: extracting data from a target chart included in the chart sample by adopting an operation function to obtain target data and attribute information of the target data; wherein the chart sample also comprises the actual abstract text of the target chart; fusing the target data, the attribute information and the type information of the operation function to obtain key information of a target chart; adopting an encoder to encode the key information to obtain encoding characteristics; adopting a decoder to decode the coding features to obtain a predicted abstract text of the target chart; and training the generated model according to the difference between the predicted abstract text and the actual abstract text.
According to another aspect of the present disclosure, there is provided a generating apparatus of a graph summary, including: the data extraction module is used for extracting data from the target chart by adopting an operation function to obtain target data and attribute information of the target data; the information fusion module is used for fusing the target data, the attribute information and the type information of the operation function to obtain key information of the target chart; the information coding module is used for coding the key information to obtain coding characteristics; and the text generation module is used for decoding the coding features to obtain the abstract text of the target chart.
According to another aspect of the present disclosure, there is provided a training apparatus that generates a model, wherein the generated model includes an encoder and a decoder; the training device comprises: the data extraction module is used for extracting data from a target chart included in the chart sample by adopting an operation function to obtain target data and attribute information of the target data; wherein the chart sample also comprises the actual abstract text of the target chart; the information fusion module is used for fusing the target data, the attribute information and the type information of the operation function to obtain key information of the target chart; the information coding module is used for coding the key information by adopting an encoder to obtain coding characteristics; the text generation module is used for decoding the coding features by adopting a decoder to obtain a predicted abstract text of the target chart; and the model training module is used for training the generated model according to the difference between the predicted abstract text and the actual abstract text.
According to another aspect of the present disclosure, there is provided an electronic device including: at least one processor; and a memory communicatively coupled to the at least one processor; the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of generating the graph summary and/or the method of training the generation model provided by the present disclosure.
According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method of generating a graph summary and/or the training method of generating a model provided by the present disclosure.
According to another aspect of the present disclosure, there is provided a computer program product comprising computer programs/instructions which, when executed by a processor, implement the method of generating a graph summary and/or the training method of generating a model provided by the present disclosure.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.
Drawings
The drawings are for a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:
FIG. 1 is a schematic diagram of an application scenario of a method for generating a graph summary and a training method and apparatus for generating a model according to an embodiment of the disclosure;
FIG. 2 is a flow diagram of a method of generating a chart summary in accordance with an embodiment of the present disclosure;
FIG. 3 is a schematic diagram of deriving key information for a target chart according to an embodiment of the disclosure;
FIG. 4 is a schematic diagram of summary text for generating a chart according to an embodiment of the disclosure;
FIG. 5 is a flow diagram of a training method of generating a model according to an embodiment of the present disclosure;
FIG. 6 is a block diagram of a diagram summary generation apparatus according to an embodiment of the disclosure;
FIG. 7 is a block diagram of a training apparatus that generates a model according to an embodiment of the present disclosure; and
FIG. 8 is a block diagram of an electronic device for implementing a method of generating a graph summary and/or a training method of generating a model in accordance with an embodiment of the present disclosure.
Detailed Description
Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
The generation technology of the chart abstract refers to: given Chart data, text describing the Chart data is generated, which technique may be referred to simply as Chart-to-Text. Wherein the chart data may be embodied in tabular form.
For example, in generating text describing chart data, data may be extracted from the chart data using a predefined arithmetic function, and then the extracted data is input into a generation model together with the original record data of the form, generating digest text. In the method of this embodiment, the generated model cannot learn the reasoning information according to the original record data and the extracted data of the table, so that the reader of the abstract text cannot be helped to quickly learn the hidden information in the chart data, and cannot be helped to comprehensively understand the meaning of the chart data.
Based on the method, the device and the method for generating the chart abstract and training the generated model are provided. An application scenario of the method and apparatus provided by the present disclosure will be described below with reference to fig. 1.
Fig. 1 is an application scenario schematic diagram of a method for generating a chart abstract and a training method and device for generating a model according to an embodiment of the disclosure.
As shown in fig. 1, the application scenario 100 of this embodiment may include an electronic device 110, and the electronic device 110 may be various electronic devices with processing functions, including but not limited to a smart phone, a tablet computer, a laptop computer, a desktop computer, a server, and the like.
The electronic device 110 may, for example, process the chart 120 and may specifically extract data from the chart 120. Subsequently, the electronic device 110 may generate key information for the graph 120 based on the extracted data, corresponding row and column information for the data in the graph 120, the type of arithmetic function employed when extracting the data, and the like. The summary text 130 of the chart 120 may then be generated from the key information.
In one embodiment, the electronic device 110 may employ the pre-trained generative model 140 to encode and decode the key information, and may obtain the summary text 130 based on the decoding result.
In an embodiment, as shown in fig. 1, the application scenario 100 may further include a server 150, where the server 150 may be, for example, a background management server supporting the operation of a client application in the electronic device 110. Electronic device 110 may be communicatively coupled to server 150 via a network, which may include wired or wireless communication links.
For example, the server 150 may train the generation model based on the graph with the summary text and send the trained generation model 140 to the electronic device 110 in response to a request by the electronic device 110 to generate the summary text 130 using the generation model by the electronic device 110.
In an embodiment, the electronic device 110 may send the data extracted from the chart 120 or the generated key information to the server 150, for example, and the server 150 generates the summary text 130 of the chart 120. Alternatively, the electronic device 110 may send the chart 120 directly to the server 150, the server 150 extracts data from the chart 120, generates key information from the data, and employs the generation model 140 to encode and decode the key information to generate the summary text 130 of the chart 120.
It should be noted that, the training method of the generating model provided in the present disclosure may be executed by the server 150. Accordingly, the training apparatus for generating a model provided by the present disclosure may be provided in the server 150. The method for generating the chart abstract provided by the disclosure can be executed by the electronic device 110, can be executed by the server 150, can also be executed by the electronic device 110 in part, and can be executed by the server 150 in part. Accordingly, the generating device of the chart abstract provided by the disclosure may be disposed in the electronic device 110, may be disposed in the server 150, or may be a part of modules disposed in the electronic device 110, and a part of modules disposed in the server 150.
It should be understood that the number and type of electronic devices 110 and servers 150 in fig. 1 are merely illustrative. There may be any number and type of electronic devices 110 and servers 150 as desired for implementation.
The method for generating the chart abstract provided by the present disclosure will be described in detail below with reference to fig. 2 to 4.
Fig. 2 is a flow diagram of a method of generating a chart summary according to an embodiment of the disclosure.
As shown in fig. 2, the method 200 of generating a graph summary of this embodiment may include operations S210 to S240.
In operation S210, data is extracted from the target graph by using the operation function, and target data and attribute information of the target data are obtained.
According to an embodiment of the present disclosure, the operation function may include, for example, a function that extracts at least one of the following data: maximum, minimum, next maximum, next minimum, median, mode, data that is the latest in time, data that is the earliest in time, etc., which is not limiting to the present disclosure.
According to embodiments of the present disclosure, the target chart may be any table, which may include a number of columns therein, which may include an Index column Index, a time column, and/or an Index column, etc. The index column may include values of any index at different time points. For example, any index may be a statistical index such as an annual average production amount or a delivery amount, or any other index, which is not limited in the present disclosure.
In this embodiment, the attribute data of the target data may include, for example, a column name of a column in which the target data is located, other data than the target data in a row in which the target data is located, a column name of the other data, an index value of a row in which the target data is located, and the like. The attribute information may be any information for reflecting the position of the target data in the target graph, or any data having a mapping relation with the target data in the target graph, for example, and the attribute information is any information extracted from the target graph according to the target data, which is not limited in the present disclosure.
It is understood that when the target graph includes a plurality of index columns, operation S210 may also extract data from a specified column of the plurality of index columns using an operation function. The designated column may be determined according to an input column name or the like, which is not limited in this disclosure.
In operation S220, the target data, the attribute information, and the type information of the operation function are fused to obtain key information of the target graph.
The type of the operation function may be a type of data extracted by the operation function, for example, may include: the maximum value type, minimum value type, next-maximum value type, next-minimum value type, median type, mode type, type of data latest in time, type of data earliest in time, and the like, which is not limited by the present disclosure.
The embodiment can combine the target data, the attribute information and the type information of the operation function into a piece of multi-element group information, and the multi-element group information is used as key information of the target chart.
In operation S230, the key information is encoded to obtain an encoding feature.
According to embodiments of the present disclosure, an encoder may be employed to encode the critical information to obtain the encoded signature. The encoder may be an encoder constructed based on a depth neural network, for example, an encoder constructed based on a sequence network, including an encoder in a transducer architecture, an encoder constructed based on a long-term and short-term memory network, and the like. It will be appreciated that the encoder may be of any configuration, for example, the encoder may be formed of a convolutional layer, as this disclosure is not limited in this respect. This embodiment may input key information into a pre-trained encoder, which outputs the encoded features.
In an embodiment, the following formula (1) may be used to perform the encoding process on the key information Z, to obtain the encoding feature h:
h=w [ Z ] +b formula (1).
Wherein, W, b is the network parameter in the encoder, and the value of the network parameter is obtained through training.
In operation S240, the encoding features are decoded to obtain the digest text of the target chart.
According to embodiments of the present disclosure, a decoder may be employed to decode the encoded features to obtain the summary text. The decoder may be a decoder constructed based on a deep neural network, for example, a decoder constructed based on a sequence network, including a decoder in a transducer architecture, or a decoder constructed based on a long-term and short-term memory network, etc. It will be appreciated that the decoder may be any structure of decoder, for example, the decoder may be a decoder constructed based on an attention mechanism, which is not limited in this disclosure. The embodiment can input the coding characteristics into a pre-trained decoder, and the decoder outputs a plurality of words forming the abstract text in a sequence form, and the abstract text can be obtained by combining the words.
When the abstract text of the target chart is generated, the key information according to the embodiment of the disclosure not only comprises the data extracted according to the operation function, but also comprises the type information of the operation function, the attribute information of the extracted data and the like, so that the content of the key information can be enhanced, reasoning information can be provided in the encoding and decoding process of the generated abstract text, and the quality and the expression capability of the generated abstract text are improved.
The manner of obtaining the key information in operation S220 described above will be further expanded and defined below.
According to embodiments of the present disclosure, at least two types of operation functions may be employed to extract data from a target graph and obtain one key information for each type of operation function. Specifically, for each of at least two types of operation functions, data may be extracted from the target graph by using the each function, so as to obtain target data and attribute information of the target data. And then, merging the target data extracted by adopting each function, the attribute information of the target data and the function type of each function to obtain key information aiming at each function.
The embodiment may input at least two pieces of key information for at least two types of arithmetic functions into the encoder in the form of an information sequence, resulting in a coded feature sequence. For example, it is set that the coding feature obtained by coding key information for the ith operation function of the operation functions of at least two types is h i The encoding characteristic h of the encoder output can be expressed as
Figure SMS_1
Where k is the number of at least two types of operational functions.
In an embodiment, the decoder may generate the digest text, for example, based on a probability function in the following equation (2) when performing the decoding process on the encoded signature sequence:
Figure SMS_2
where y is the word sequence of the abstract text, y t For the t-th word in the word sequence, |y| is the word in the word sequenceTotal number, y <t The word which the decoder has outputted before outputting the t word can also be expressed as the word which is positioned before the position of the t word in the word sequence. p (y|h) is the conditional probability of generating y when the coding feature is h.
According to the embodiment, the data are extracted by adopting at least two types of operation functions, so that the extracted data and the key information obtained by fusion are richer, richer reasoning information can be provided for the generation of the abstract text, and the quality and the expression capability of the generated abstract text can be improved.
In an embodiment, the target data, the attribute information, and the type information of the operation function may be fused according to a predetermined information template. Wherein, a plurality of element positions can be set for at least one information of the target data, the attribute information and the type information of the operation function in the predetermined information template, so that the at least one information is filled in different positions in the plurality of element positions according to the requirements of different scenes. Therefore, the applicability of the principle of obtaining key information through fusion can be improved, and the robustness of the method for generating the chart abstract provided by the disclosure is improved.
Specifically, when obtaining the key information, the target data, the attribute information, and the type information of the operation function may be combined into the multi-group information according to a predetermined information template, and the multi-group information may be used as the key information of the target graph. It will be appreciated that where the arithmetic function comprises at least two types of functions, the embodiment may obtain a plurality of sets of information for each type of arithmetic function. This embodiment will be described in detail below in conjunction with fig. 3.
Fig. 3 is a schematic diagram of deriving key information for a target chart according to an embodiment of the disclosure.
As shown in fig. 3, in this embodiment 300, when the data is extracted from the target graph 310 using the maximum value type arithmetic function arg_max () 320, the column to which the data is directed may be the index column "XXX", and the result of the extraction is the maximum value "41691" in the index column "XXX", and the target data 330 is "41691". The attribute information of the target data 330 may include a column name "XXX" of a column in which the target data is located, a year 2019 having a mapping relationship with the target data, and an index value "1" of a row in which the target data is located. The attribute information and the target data 330 may be represented by, for example, a six-tuple 340 (1, y, XXX,41691, none, 2019), where "1" in the six-tuple 340 is an index value, "y, XXX" represents a column name of a column in which the target data is located, and it is understood that a relationship between "y" and "XXX" may be understood as a relationship between a key (value) and a value (value). Where "None" and "2019" are two different types of attribute values of attributes having a mapping relationship with the target data. For example, in the target graph 310, the attribute value having a mapping relation with the target data 330 is year "2019", and year "2019" is a data type. If the year column in the target graph 310 is replaced by the entity column, the attribute value having a mapping relationship with the target data 330 should be an entity, such as entity a, and "None" in the six-tuple 340 is replaced by "entity a" and "2019" in the six-tuple 340 is replaced by "None". It will be appreciated that the above information having a mapping relationship with the target data is used as any attribute in the attribute information, and two elements are set for any attribute, so that the data form of the six-tuple 340 can be used to represent the target data and attribute information of multiple types of charts. It can be appreciated that, according to actual requirements, three or more elements may be further set for any attribute, so as to further extend the application scenario of the six-tuple 340 in the form of data.
As shown in fig. 3, the type of the operation function 320 in this embodiment may be represented by a binary group 350, and if the operation function 320 is of a maximum value type and the data targeted by the operation function 320 is a numerical value in the "XXX" column, i.e., the type of the data targeted by the operation function 320 is of a numerical value type, the binary group 350 may be represented as (max, none); if the operation function 320 is of the minimum type, the tuple 350 is changed to (min, none). If the operation function 320 is of the maximum value type and the data for which the operation function 320 is aimed is the "year" column, the purpose of the operation function 320 is to obtain the latest "XXX" value, and the maximum value refers to the latest time in the "year" column, i.e. the type of the data for which the operation function 320 is aimed is the time type, the binary set may be expressed as (none, last); if the operation function 320 is of the minimum type, the doublet is changed to (none, oldest). It will be appreciated that by providing two elements for the type of data to which the operation function is directed, the data form of the tuple 350 can be used to represent the type of operation function that extracts data from columns of different types of data. It can be appreciated that, according to actual requirements, three or more elements may be further set for the type of data targeted by the operation function, so as to further extend the application scenario of the data form of the binary group 350.
In this embodiment 300, the predetermined information template may be, for example, an octave information template in which the first six-bit element corresponds to the six-tuple 340 and the second two-bit element corresponds to the two-tuple 350, and this embodiment may be implemented by combining the six-tuple 340 and the two-tuple 350 into an octave (1, y, xxx,41691, none,2019, max, none) 360, and using the octave 360 as one piece of key information of the target chart.
It is understood that at least two elements may be set for only any attribute, or may be set for only the type of data for which the operation function is directed, which is not limited by the present disclosure. And it is understood that the above-described binary, six-tuple and eight-tuple in the embodiment 300 are merely examples to facilitate understanding of the present disclosure, which is not limited thereto, and any number of tuples may be provided according to actual needs.
The principles of generating summary text for a chart will be further expanded and defined below.
Fig. 4 is a schematic diagram of summary text for generating a chart according to an embodiment of the disclosure.
According to the embodiment of the disclosure, when the abstract text of the chart is generated, besides the target data extracted from the target chart, the attribute information of the target data and the type of the operation function called during extraction, for example, the title of the target chart can be considered, so that richer information is provided for generating the abstract text, and the quality and the expression capability of the generated abstract text are further improved.
As shown in fig. 4, in this embodiment 400, after obtaining the key information 410 of the target chart by the method described above, the key information 410 may be encoded by, for example, an encoder 420 to obtain an encoded feature 430. At the same time, this embodiment may also obtain embedded features 450 of the title information 440 of the target chart. It will be appreciated that the embedded feature 450 may be output by the text embedding layer by inputting the title information 440 into the text embedding layer. Alternatively, the embedded feature 450 may be pre-generated and stored in a memory space, from which the embodiment may obtain the embedded feature 450 of the title information 440 when performing the method of generating a chart summary.
After obtaining the embedded feature 450 and the encoded feature 430, the embodiment 400 may first fuse the two features to obtain a fused feature 460. For example, fusion of the two features may be achieved by stitching the embedded feature 450 and the encoded feature 430. Alternatively, the encoding feature 430 is set to the sequence of features described above
Figure SMS_3
The embedded feature is denoted as h title The fusion feature may for example also be a feature sequence, denoted +.>
Figure SMS_4
Wherein (1)>
Figure SMS_5
Can be expressed by the following formula (3):
Figure SMS_6
Wherein,,
Figure SMS_7
indicating that will be h i Each element of (a) and h title The elements at the corresponding positions in (a) are added.
After obtaining the fused feature 460, the embodiment may input the fused feature 460 into the decoder 470, and output the summary text 480 of the target chart by the decoder 470.
In order to facilitate implementation of the graph summary generation method, the present disclosure further provides a training method for generating a model, which will be described in detail below with reference to fig. 5.
Fig. 5 is a flow diagram of a training method of generating a model according to an embodiment of the present disclosure.
As shown in fig. 5, the training method 500 of the generative model of this embodiment may include operations S510 to S550. Wherein the generative model may comprise an encoder and a decoder.
In operation S510, data is extracted from the target graph included in the graph sample by using the operation function, so as to obtain the target data and attribute information of the target data.
Wherein the chart sample also includes the actual summary text of the target chart. The implementation principle of operation S510 is similar to that of operation S210, and will not be described here again.
In operation S520, the target data, the attribute information, and the type information of the operation function are fused to obtain key information of the target graph. The implementation principle of the operation S520 is similar to that of the operation S220, and will not be described herein.
In operation S530, the key information is encoded using an encoder to obtain an encoding characteristic. The implementation principle of the operation S530 is similar to that of the operation S230, and will not be described herein.
In operation S540, the encoding features are decoded by a decoder to obtain the predicted digest text of the target graph. The implementation principle of the operation S540 is similar to that of the operation S240, and will not be described herein.
In operation S550, the generated model is trained according to the difference between the predicted digest text and the actual digest text.
According to embodiments of the present disclosure, the value of the cross entropy loss function may be employed, for example, to represent the difference between the predicted digest text and the actual digest text. The embodiment may aim to minimize this difference, adjusting network parameters in the generated model, thereby enabling training of the generated model. The network parameters may include, for example, W and b, etc., as described above.
Based on the method for generating the chart abstract provided by the disclosure, the disclosure also provides a device for generating the chart abstract, and the device will be described in detail with reference to fig. 6.
Fig. 6 is a block diagram of a diagram summary generation apparatus according to an embodiment of the present disclosure.
As shown in fig. 6, the apparatus 600 for generating a chart summary of this embodiment may include a data extraction module 610, an information fusion module 620, an information encoding module 630, and a text generation module 640.
The data extraction module 610 is configured to extract data from the target graph by using an operation function, so as to obtain target data and attribute information of the target data. In an embodiment, the data extraction module 610 may be configured to perform the operation S210 described above, which is not described herein.
The information fusion module 620 is configured to fuse the target data, the attribute information, and the type information of the operation function to obtain key information of the target chart. In an embodiment, the information fusion module 620 may be configured to perform the operation S220 described above, which is not described herein.
The information encoding module 630 is configured to encode the key information to obtain an encoding feature. In an embodiment, the information encoding module 630 may be configured to perform the operation S230 described above, which is not described herein.
The text generation module 640 is configured to perform decoding processing on the encoded features to obtain the abstract text of the target chart. In an embodiment, the text generating module 640 may be configured to perform the operation S240 described above, which is not described herein.
According to an embodiment of the present disclosure, the operational function includes at least two types of functions. The information fusion module 620 may be configured to: for each of at least two types of functions: and fusing the target data extracted by each function, the attribute information of the target data and the type information of each function to obtain key information aiming at each function. Wherein the encoding process of the key information is performed by using an encoder based on a sequential network configuration, and the decoding process of the encoded features is performed by using a decoder based on a sequential network configuration.
The apparatus 600 may further include an embedded feature acquisition module for acquiring embedded features of title information of the target chart according to an embodiment of the present disclosure. The text generation module 640 may include a fusion submodule and a decode submodule. And the fusion submodule is used for fusing the embedded features and the coding features of the header information to obtain fusion features. And the decoding submodule is used for decoding the fusion characteristics to obtain the abstract text of the target chart.
According to an embodiment of the present disclosure, the information fusion module 620 may be configured to: and combining the target data, the attribute information and the type information of the operation function into multi-element group information according to a preset information template to obtain the key information of the target chart. Wherein, the multi-element group information comprises at least two elements aiming at any attribute in the attribute information so as to respectively correspond to at least two types of attribute values of any attribute; and/or, the multi-element group information comprises at least two elements aiming at the operation function so as to respectively correspond to at least two types of data aiming at the operation function.
According to an embodiment of the present disclosure, the at least two types of attribute values of any of the above attributes include: a value type, an entity type; the at least two types of data for which the arithmetic function is directed include: numerical type, time type.
According to an embodiment of the present disclosure, the operational function includes a function of at least one of the following types: minimum type, next-minimum type, maximum type, and next-maximum type.
Based on the training method for generating the model provided by the present disclosure, the present disclosure further provides a training device for generating the model, and the device will be described in detail below with reference to fig. 7.
Fig. 7 is a block diagram of a training apparatus that generates a model according to an embodiment of the present disclosure.
As shown in fig. 7, the training apparatus 700 for generating a model of this embodiment may include a data extraction module 710, an information fusion module 720, an information encoding module 730, a text generation module 740, and a model training module 750. Wherein the generative model may comprise an encoder and a decoder.
The data extraction module 710 may be configured to extract data from a target graph included in the graph sample by using an operation function, so as to obtain target data and attribute information of the target data. Wherein the chart sample also includes the actual summary text of the target chart. In an embodiment, the data extraction module 710 may be configured to perform the operation S510 described above, which is not described herein.
The information fusion module 720 may be configured to fuse the target data, the attribute information, and the type information of the operation function to obtain key information of the target chart. In an embodiment, the information fusion module 720 may be configured to perform the operation S520 described above, which is not described herein.
The information encoding module 730 may be configured to encode the key information by using an encoder to obtain the encoding feature. In an embodiment, the information encoding module 730 may be configured to perform the operation S530 described above, which is not described herein.
The text generation module 740 may be configured to decode the encoded feature with a decoder to obtain the predicted digest text of the target chart. In an embodiment, the text generation module 740 may be configured to perform the operation S540 described above, which is not described herein.
Model training module 750 may be used to train a generated model based on differences between predicted summary text and actual summary text. In an embodiment, the model training module 750 may be used to perform the operation S550 described above, which is not described herein.
In the technical scheme of the disclosure, the related processes of collecting, storing, using, processing, transmitting, providing, disclosing and applying personal information of the user all conform to the regulations of related laws and regulations, necessary security measures are adopted, and the public welcome is not violated. In the technical scheme of the disclosure, the authorization or consent of the user is obtained before the personal information of the user is obtained or acquired.
According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.
FIG. 8 illustrates a schematic block diagram of an example electronic device 800 that may be used to implement the method of generating a graph summary and/or the method of training a generation model of embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 8, the apparatus 800 includes a computing unit 801 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 802 or a computer program loaded from a storage unit 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other by a bus 804. An input/output (I/O) interface 805 is also connected to the bus 804.
Various components in device 800 are connected to I/O interface 805, including: an input unit 806 such as a keyboard, mouse, etc.; an output unit 807 such as various types of displays, speakers, and the like; a storage unit 808, such as a magnetic disk, optical disk, etc.; and a communication unit 809, such as a network card, modem, wireless communication transceiver, or the like. The communication unit 809 allows the device 800 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.
The computing unit 801 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 801 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as a method of generating a graph summary and/or a training method of generating a model. For example, in some embodiments, the method of generating the chart summary and/or the method of training the generated model may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program may be loaded and/or installed onto device 800 via ROM 802 and/or communication unit 809. When the computer program is loaded into RAM 803 and executed by computing unit 801, one or more steps of the chart summary generation method and/or the model generation training method described above may be performed. Alternatively, in other embodiments, computing unit 801 may be configured to perform the method of generating the graph summary and/or the training method of generating the model in any other suitable manner (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, also referred to as a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so as to solve the defects of large management difficulty and weak service expansibility in the traditional physical hosts and VPS service ("virtual privateserver" or simply "VPS"). The server may also be a server of a distributed system or a server that incorporates a blockchain.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel or sequentially or in a different order, provided that the desired results of the technical solutions of the present disclosure are achieved, and are not limited herein.
The above detailed description should not be taken as limiting the scope of the present disclosure. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present disclosure are intended to be included within the scope of the present disclosure.

Claims (17)

1. A method of generating a chart summary, comprising:
extracting data from a target chart by adopting an operation function to obtain target data and attribute information of the target data;
fusing the target data, the attribute information and the type information of the operation function to obtain key information of the target chart;
coding the key information to obtain coding characteristics; and
decoding the coding feature to obtain abstract text of the target chart,
The fusing the target data, the attribute information and the type information of the operation function to obtain key information of the target chart includes:
and combining the target data, the attribute data and the type information of the operation function into multi-element group information to obtain the key information of the target chart.
2. The method of claim 1, wherein the operational function comprises at least two types of functions; the fusing the target data, the attribute information and the type information of the operation function to obtain key information of the target chart includes:
for each of at least two types of functions: fusing the target data extracted by each function, the attribute information of the target data and the type information of each function to obtain key information aiming at each function,
wherein the encoding process of the key information is performed using an encoder based on a sequential network configuration, and the decoding process of the encoded features is performed using a decoder based on a sequential network configuration.
3. The method of claim 1, further comprising:
Acquiring embedded features of title information of the target chart;
wherein, the decoding the encoding feature to obtain the chart abstract of the target chart includes:
fusing the embedded features of the title information and the coding features to obtain fused features; and
and decoding the fusion characteristics to obtain the abstract text of the target chart.
4. The method of claim 1, wherein the fusing the target data, the attribute information, and the type information of the operation function to obtain key information of the target graph includes:
combining the target data, the attribute information and the type information of the operation function into multi-element group information according to a preset information template to obtain key information of the target chart,
wherein the multi-group information comprises at least two elements aiming at any attribute in the attribute information so as to respectively correspond to at least two types of attribute values of any attribute; and/or the multi-element group information comprises at least two elements aiming at the operation function so as to respectively correspond to at least two types of data aiming at the operation function.
5. The method of claim 4, wherein the at least two types of attribute values for any one attribute comprise: a value type, an entity type; the at least two types of data for which the arithmetic function is directed include: numerical type, time type.
6. The method of claim 1, wherein the operational function comprises a function of at least one of the following types: minimum type, next-minimum type, maximum type, and next-maximum type.
7. A training method of a generative model, wherein the generative model comprises an encoder and a decoder; the method comprises the following steps:
extracting data from a target chart included in a chart sample by adopting an operation function to obtain target data and attribute information of the target data; wherein the chart sample further comprises an actual abstract text of the target chart;
fusing the target data, the attribute information and the type information of the operation function to obtain key information of the target chart;
the encoder is adopted to encode the key information, so as to obtain encoding characteristics;
adopting the decoder to decode the coding features to obtain a prediction abstract text of the target chart; and
Training the generation model according to the difference between the predicted abstract text and the actual abstract text,
the fusing the target data, the attribute information and the type information of the operation function to obtain key information of the target chart includes:
and combining the target data, the attribute data and the type information of the operation function into multi-element group information to obtain the key information of the target chart.
8. A graph summary generation apparatus, comprising:
the data extraction module is used for extracting data from the target chart by adopting an operation function to obtain target data and attribute information of the target data;
the information fusion module is used for fusing the target data, the attribute information and the type information of the operation function to obtain key information of the target chart;
the information coding module is used for coding the key information to obtain coding characteristics; and
a text generation module for decoding the coding feature to obtain the abstract text of the target chart,
the information fusion module is specifically configured to: and combining the target data, the attribute data and the type information of the operation function into multi-element group information to obtain the key information of the target chart.
9. The apparatus of claim 8, wherein the operational function comprises at least two types of functions; the information fusion module is used for:
for each of at least two types of functions: fusing the target data extracted by each function, the attribute information of the target data and the type information of each function to obtain key information aiming at each function,
wherein the encoding process of the key information is performed using an encoder based on a sequential network configuration, and the decoding process of the encoded features is performed using a decoder based on a sequential network configuration.
10. The apparatus of claim 8, further comprising:
the embedded feature acquisition module is used for acquiring the embedded features of the title information of the target chart;
wherein, the text generation module includes:
the fusion sub-module is used for fusing the embedded features of the title information and the coding features to obtain fusion features; and
and the decoding sub-module is used for decoding the fusion characteristics to obtain the abstract text of the target chart.
11. The apparatus of claim 8, wherein the information fusion module is to:
Combining the target data, the attribute information and the type information of the operation function into multi-element group information according to a preset information template to obtain key information of the target chart,
wherein the multi-group information comprises at least two elements aiming at any attribute in the attribute information so as to respectively correspond to at least two types of attribute values of any attribute; and/or the multi-element group information comprises at least two elements aiming at the operation function so as to respectively correspond to at least two types of data aiming at the operation function.
12. The apparatus of claim 11, wherein the at least two types of attribute values for any one attribute comprise: a value type, an entity type; the at least two types of data for which the arithmetic function is directed include: numerical type, time type.
13. The apparatus of claim 8, wherein the operational function comprises a function of at least one of the following types: minimum type, next-minimum type, maximum type, and next-maximum type.
14. A training apparatus that generates a model, wherein the generated model includes an encoder and a decoder; the device comprises:
The data extraction module is used for extracting data from a target chart included in the chart sample by adopting an operation function to obtain target data and attribute information of the target data; wherein the chart sample further comprises an actual abstract text of the target chart;
the information fusion module is used for fusing the target data, the attribute information and the type information of the operation function to obtain key information of the target chart;
the information coding module is used for coding the key information by adopting the coder to obtain coding characteristics;
the text generation module is used for decoding the coding features by adopting the decoder to obtain a predicted abstract text of the target chart; and
a model training module for training the generated model according to the difference between the predicted abstract text and the actual abstract text,
the information fusion module is specifically configured to: and combining the target data, the attribute data and the type information of the operation function into multi-element group information to obtain the key information of the target chart.
15. An electronic device, comprising:
at least one processor; and
A memory communicatively coupled to the at least one processor; wherein,,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 7.
16. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-7.
17. A computer program product comprising computer programs/instructions stored on at least one of a readable storage medium and an electronic device, which when executed by a processor, implement the steps of the method according to any one of claims 1 to 7.
CN202211037048.0A 2022-08-26 2022-08-26 Method and device for generating chart abstract and training method and device for generating model Active CN115309888B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211037048.0A CN115309888B (en) 2022-08-26 2022-08-26 Method and device for generating chart abstract and training method and device for generating model

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211037048.0A CN115309888B (en) 2022-08-26 2022-08-26 Method and device for generating chart abstract and training method and device for generating model

Publications (2)

Publication Number Publication Date
CN115309888A CN115309888A (en) 2022-11-08
CN115309888B true CN115309888B (en) 2023-05-30

Family

ID=83864523

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211037048.0A Active CN115309888B (en) 2022-08-26 2022-08-26 Method and device for generating chart abstract and training method and device for generating model

Country Status (1)

Country Link
CN (1) CN115309888B (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109508400A (en) * 2018-10-09 2019-03-22 中国科学院自动化研究所 Picture and text abstraction generating method
CN110704606A (en) * 2019-08-19 2020-01-17 中国科学院信息工程研究所 Generation type abstract generation method based on image-text fusion
CN111460135A (en) * 2020-03-31 2020-07-28 北京百度网讯科技有限公司 Method and device for generating text abstract
CN114020900A (en) * 2021-11-16 2022-02-08 桂林电子科技大学 Chart English abstract generation method based on fusion space position attention mechanism

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9607039B2 (en) * 2013-07-18 2017-03-28 International Business Machines Corporation Subject-matter analysis of tabular data
CN104657375B (en) * 2013-11-20 2018-01-26 中国科学院深圳先进技术研究院 A kind of picture and text subject description method, apparatus and system
CN111753050A (en) * 2020-06-18 2020-10-09 微软技术许可有限责任公司 Topic map based comment generation
CN113434664A (en) * 2021-06-30 2021-09-24 平安科技(深圳)有限公司 Text abstract generation method, device, medium and electronic equipment
CN113961685A (en) * 2021-07-13 2022-01-21 北京金山数字娱乐科技有限公司 Information extraction method and device
CN113806522A (en) * 2021-09-18 2021-12-17 北京百度网讯科技有限公司 Abstract generation method, device, equipment and storage medium
CN114936279A (en) * 2022-03-07 2022-08-23 重庆中烟工业有限责任公司 Unstructured chart data analysis method for collaborative manufacturing enterprise
CN114818627A (en) * 2022-05-25 2022-07-29 中国建设银行股份有限公司 Form information extraction method, device, equipment and medium

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109508400A (en) * 2018-10-09 2019-03-22 中国科学院自动化研究所 Picture and text abstraction generating method
CN110704606A (en) * 2019-08-19 2020-01-17 中国科学院信息工程研究所 Generation type abstract generation method based on image-text fusion
CN111460135A (en) * 2020-03-31 2020-07-28 北京百度网讯科技有限公司 Method and device for generating text abstract
CN114020900A (en) * 2021-11-16 2022-02-08 桂林电子科技大学 Chart English abstract generation method based on fusion space position attention mechanism

Also Published As

Publication number Publication date
CN115309888A (en) 2022-11-08

Similar Documents

Publication Publication Date Title
CN113313022B (en) Training method of character recognition model and method for recognizing characters in image
CN112597753A (en) Text error correction processing method and device, electronic equipment and storage medium
JP7346788B2 (en) Speech recognition model training methods, devices, equipment, and storage media
CN113792854A (en) Model training and word stock establishing method, device, equipment and storage medium
CN112861548B (en) Training method, device, equipment and storage medium for natural language generation and model
JP7384943B2 (en) Training method for character generation model, character generation method, device, equipment and medium
KR20220034080A (en) Training method for circulary generating network model, method and apparatus for establishing word library, electronic device, recording medium and computer program
JP7337203B2 (en) Character generation model training method, character generation method, apparatus and equipment
CN113011420A (en) Character recognition method, model training method, related device and electronic equipment
CN114724168A (en) Training method of deep learning model, text recognition method, text recognition device and text recognition equipment
CN113656613A (en) Method for training image-text retrieval model, multi-mode image retrieval method and device
CN113689868B (en) Training method and device of voice conversion model, electronic equipment and medium
CN114863182A (en) Image classification method, and training method and device of image classification model
CN114861637A (en) Method and device for generating spelling error correction model and method and device for spelling error correction
CN113641829A (en) Method and device for training neural network of graph and complementing knowledge graph
CN115309888B (en) Method and device for generating chart abstract and training method and device for generating model
CN115565177B (en) Character recognition model training, character recognition method, device, equipment and medium
CN113963358B (en) Text recognition model training method, text recognition device and electronic equipment
CN115357710A (en) Training method and device for table description text generation model and electronic equipment
CN115860003A (en) Semantic role analysis method and device, electronic equipment and storage medium
CN114841175A (en) Machine translation method, device, equipment and storage medium
CN114254028A (en) Event attribute extraction method and device, electronic equipment and storage medium
CN113204616A (en) Method and device for training text extraction model and extracting text
CN113553413A (en) Dialog state generation method and device, electronic equipment and storage medium
CN113553863B (en) Text generation method, device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant