CN111191430B - Automatic table building method and device, computer equipment and storage medium - Google Patents
Automatic table building method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN111191430B CN111191430B CN201911371969.9A CN201911371969A CN111191430B CN 111191430 B CN111191430 B CN 111191430B CN 201911371969 A CN201911371969 A CN 201911371969A CN 111191430 B CN111191430 B CN 111191430B
- Authority
- CN
- China
- Prior art keywords
- root
- matching result
- characters
- matching
- word
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application relates to the technical field of database table building, in particular to an automatic table building method, an automatic table building device, computer equipment and a storage medium, wherein the method comprises the following steps: acquiring historical tabulation data, extracting data with the occurrence frequency larger than a preset frequency threshold value from the historical tabulation data as a root word, and sequencing all the root words to generate a root word library; receiving table building information input by a user, extracting keywords in the table building information, and performing positive direction matching on the keywords and roots in a root bank to obtain a first matching result; performing inverse direction matching on the keywords and the roots in the root bank to obtain a second matching result; and comparing the first matching result with the second matching result, and then establishing a table. The matching results are compared in a forward matching mode and a reverse matching mode, so that word segmentation can be accurately obtained to the maximum extent, and the requirements of a user on the table can be quickly and accurately met.
Description
Technical Field
The present application relates to the field of database table creation technologies, and in particular, to an automatic table creation method and apparatus, a computer device, and a storage medium.
Background
In a database, a TABLE (TABLE) is an object used for storing data in the database, is a set of structured data, and is the basis of the entire database system. In an operating system, creating a table means that in storage management, the system will build a mapping table for each process.
At present, when a table is built, a table building assistant is usually adopted as a table building tool, the table building assistant is a submodule embedded in a metadata platform, the purpose is to standardize data table building, and the sql required by a user is matched by taking a basic root of root management as a support.
However, the existing table creation assistant cannot perform word segmentation accurately when creating a table, so that the requirements of a user on the table cannot be met quickly and accurately.
Disclosure of Invention
Based on this, aiming at the problem that the existing table building assistant cannot accurately perform word segmentation when building a table, so that the requirement of a user on the table cannot be quickly and accurately met, the automatic table building method, the automatic table building device, the computer equipment and the storage medium are provided.
An automatic table building method, comprising the steps of:
acquiring past table building data, extracting data with the occurrence frequency larger than a preset frequency threshold value from the past table building data as a root, and sequencing all the roots to generate a root bank;
receiving table building information input by a user, extracting keywords in the table building information, and performing positive direction matching on the keywords and roots in the root bank to obtain a first matching result;
performing inverse direction matching on the keywords and the roots in the root bank to obtain a second matching result;
giving different weights to each element in the first matching result, extracting a root word in the first matching result, and multiplying the root word in the first matching result by the corresponding weight to obtain a first score;
giving different weights to each element in the second matching result, extracting a root word in the second matching result, and multiplying the root word in the second matching result by the corresponding weight to obtain a second score;
and if the first score is consistent with the second score, the first matching result or the second matching result is used as a word segmentation scheme for building a table, otherwise, the matching result with a low score in the first score and the second score is used as the word segmentation scheme for building the table.
In one possible embodiment, the obtaining the past table building data, extracting data with occurrence times greater than a preset time threshold from the past table building data as a root, and sorting all the roots to generate a root bank includes:
acquiring a plurality of historical tabular building data, and respectively cleaning the historical tabular building data;
sorting the data after data cleaning according to the occurrence times of preset words, and taking the preset word with the largest occurrence time as a root word;
and summarizing all the root words, sorting the root words according to the occurrence times, and generating the root word library.
In one possible embodiment, the receiving table building information input by a user, extracting a keyword in the table building information, and performing positive direction matching on the keyword and a root in the root bank to obtain a first matching result, includes:
receiving a table building instruction input by a user, and acquiring table building information required by the user from the table building instruction, wherein the table building information comprises a table field and a table name;
extracting key characters in the table fields, and splicing the key characters and the table names to obtain the key words;
acquiring the number of characters of each root in the root bank, acquiring the number of the keywords characters by taking the maximum value N of the number of the characters as a first number of characters, and taking the number of the keywords characters as a second number of characters;
if the second number of characters is smaller than the first number of characters, matching along the positive direction of the root bank by taking the keywords as query conditions, and extracting the root with the first number of characters equal to the second number of characters as the first matching result;
if the second number of characters is not less than the first number of characters, intercepting the first N characters in the keywords as query conditions, matching along the positive direction of the root bank, and extracting the root of which the first number of characters is equal to the second number of characters as the first matching result.
In one possible embodiment, the obtaining a second matching result after performing inverse direction matching on the keyword and the root word in the root word library includes:
if the second number of characters is smaller than the first number of characters, matching along the reverse direction of the root bank by taking the keywords as query conditions, and extracting the root of which the first number of characters is equal to the second number of characters as a second matching result;
if the second number of characters is not less than the first number of characters, intercepting the last N characters in the keywords as query conditions, matching along the reverse direction of the root bank, and extracting the root with the first number of characters equal to the second number of characters as the first matching result.
In one possible embodiment, the giving different weights to the elements in the first matching result includes:
dividing elements in the first matching result into single words, word roots or non-word roots according to word attributes;
obtaining the word segmentation quantity of the root word according to the table building information input by the user;
and giving different weights to the single words, the number of the participles and the non-root words.
In one possible embodiment, after the performing the table-building with the first matching result or the second matching result as the word segmentation scheme, the method further includes:
respectively acquiring a first table main key for building a table according to the first matching result and a second table main key for building a table according to the second matching result;
and comparing the lengths of the first table main key and the second table main key, if the lengths are consistent, marking the table as successful table building, otherwise, re-scoring the first matching result and the second matching result until the table building is successful.
An automatic table building device comprises the following modules:
the root library module is used for acquiring the historical tabulation data, extracting data with the occurrence frequency larger than a preset frequency threshold value from the historical tabulation data as roots, and sequencing all the roots to generate a root library;
the first matching result module is used for receiving table building information input by a user, extracting keywords in the table building information, and performing positive direction matching on the keywords and the root words in the root word library to obtain a first matching result;
the second matching result module is used for obtaining a second matching result after the keyword is subjected to inverse direction matching with the root word in the root word library;
the first scoring module is configured to give different weights to each element in the first matching result, extract a root word in the first matching result, and multiply the root word in the first matching result and the corresponding weight to obtain a first score;
the second scoring module is configured to give different weights to each element in the second matching result, extract a root word in the second matching result, and multiply the root word in the second matching result and the corresponding weight to obtain a second score;
and the table building module is set to build a table by taking the first matching result or the second matching result as a word segmentation scheme if the first score is consistent with the second score, or else, build a table by taking the matching result with the lower score in the first score and the second score as the word segmentation scheme.
In one possible embodiment, the first matching result generating module is further configured to:
if the second number of characters is smaller than the first number of characters, matching along the reverse direction of the root bank by taking the keywords as query conditions, and extracting the root of which the first number of characters is equal to the second number of characters as a second matching result;
if the second number of characters is not less than the first number of characters, intercepting the last N characters in the keywords as query conditions, matching along the reverse direction of the root bank, and extracting the root with the first number of characters equal to the second number of characters as the first matching result.
A computer device comprising a memory and a processor, the memory having stored therein computer readable instructions which, when executed by the processor, cause the processor to perform the steps of the above-described automatic table building method.
A storage medium having stored thereon computer-readable instructions which, when executed by one or more processors, cause the one or more processors to perform the steps of the above-described automatic table creation method.
Compared with the existing mechanism, the method carries out accurate word segmentation through the forward matching and reverse matching modes, and comprises the following specific steps: acquiring historical tabulation data, extracting data with the occurrence frequency larger than a preset frequency threshold value from the historical tabulation data as a root word, and sequencing all the root words to generate a root word library; receiving table building information input by a user, extracting keywords in the table building information, and performing positive direction matching on the keywords and roots in the root bank to obtain a first matching result; performing inverse direction matching on the keywords and the roots in the root bank to obtain a second matching result; giving different weights to each element in the first matching result, extracting a root word in the first matching result, and multiplying the root word in the first matching result by the corresponding weight to obtain a first score; giving different weights to each element in the second matching result, extracting a root word in the second matching result, and multiplying the root word in the second matching result by the corresponding weight to obtain a second score; and if the first score is consistent with the second score, the first matching result or the second matching result is used as a word segmentation scheme for building a table, otherwise, the matching result with a low score in the first score and the second score is used as the word segmentation scheme for building the table. The method and the device can accurately obtain the word segmentation to a greater extent, and quickly and accurately meet the requirements of the user on the table.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the application.
FIG. 1 is a general flow diagram of an automatic table creation method in one embodiment of the present application;
FIG. 2 is a schematic diagram illustrating a root-of-word repository generation process in an automatic table creation method according to an embodiment of the present application;
FIG. 3 is a diagram illustrating a first matching result generation process in an automatic table creation method according to an embodiment of the present application;
FIG. 4 is a block diagram of an automatic table creation device in one embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
Fig. 1 is an overall flowchart of an automatic table building method according to an embodiment of the present application, and the automatic table building method includes the following steps:
s1, obtaining historical tabulation data, extracting data with the occurrence frequency larger than a preset frequency threshold value from the historical tabulation data to serve as a root word, and sequencing all the root words to generate a root word library;
specifically, the source of the calendar data may be data stored in an internal database, or data from the internet. If the data of the past table establishment is data from the internal database, a plurality of tables which are once established can be called from the internal database in a key word query mode. If the data source is the internet, the related table building data can be crawled from a cloud server in the internet by adopting technologies such as web crawlers and the like.
The time threshold in this step may be determined according to different types of tables, and common table types include DBD, HEAP, ISAM, MERGE, myIAS, innoDB, and the like. The number threshold for different types of tables is different, e.g., 10 for InNODB, 5 for HEAP, etc.
S2, receiving table building information input by a user, extracting keywords in the table building information, and performing positive direction matching on the keywords and roots in the root bank to obtain a first matching result;
specifically, the keywords mainly refer to table names in the building table, table types, and characteristic characters appearing in fields of the table, such as numerical values, special symbols, and the like. And performing positive direction matching, namely performing matching once from the left end starting point to the right end final point of the streaming data of the root bank. And if the roots in the root bank are not arranged by adopting the streaming data, converting the data in the root bank into a streaming data arrangement mode.
S3, performing inverse direction matching on the keywords and the roots in the root bank to obtain a second matching result;
specifically, when performing inverse direction matching, the matching method adopted is consistent with the positive direction matching, and the difference is only the direction difference, i.e., one is from left to right, and the other is from right to left.
S4, giving different weights to each element in the first matching result, extracting a root word in the first matching result, and multiplying the root word in the first matching result by the corresponding weight to obtain a first score;
specifically, the first matching result mainly includes three elements, which are roots, words or non-roots. When weighting is performed, the number of participles required for the root word needs to be used as a parameter. That is, the weight that can be given according to the actual scene is 0.3 for a single word, 0.4 for the number of participles, and 0.3 for the non-root word. In the evaluation, it is possible to employ: scoring is performed by a formula of 0.3 × n (number of participles in the scheme) + 0.4+ non-root 0.3.
S5, giving different weights to each element in the second matching result, extracting a root word in the second matching result, and multiplying the root word in the second matching result by the corresponding weight to obtain a second score;
wherein, the weight statistical mode in the step is consistent with the mode in the step 4.
S6, if the first score is consistent with the second score, the first matching result or the second matching result is used as a word segmentation scheme to build a table, otherwise, the matching result with the low score in the first score and the second score is used as the word segmentation scheme to build the table.
Specifically, when the word segmentation scheme is selected, the matching result with a low score is selected as the word segmentation priority scheme, that is, the matching degree with a low score is high. Generally, the situation that the scores of two matching results are consistent does not occur, if the scores are consistent, the situation is firstly checked, and if any error is reported, any matching result is taken as a word segmentation scheme to be established.
In the embodiment, the matching results are compared in a forward matching mode and a reverse matching mode, so that the word segmentation can be accurately obtained to a greater extent, and the requirement of a user on the table can be quickly and accurately met.
Fig. 2 is a schematic diagram of a root bank generation process in an automatic table building method in an embodiment of the present application, where as shown in the drawing, the S1 obtains the data of the past table building, extracts data with a frequency of occurrence greater than a preset frequency threshold from the data of the past table building as a root, sorts all the roots to generate a root bank, and includes:
s11, obtaining a plurality of historical tabular data, and respectively cleaning the historical tabular data;
the Data cleaning method mainly cleans four Data in the table building Data, namely a missing value, an abnormal value (outlier), a Duplicate removal process (Duplicate Data) and noise Data, and mainly cleans the abnormal value in the table building Data in the invention, wherein the abnormal value cleaning method mainly comprises the following steps: 1, completing simple statistical analysis in the EDA, and only using a descriptor method of pandas to realize the simple statistical analysis, and finding whether unreasonable values, namely abnormal values exist or not through data set descriptive statistics; 2.
principle- -outlier detection based on Normal distribution if data obeys normal distribution
In principle, the outlier is oneThe group measurements were deviated from the mean by a value of more than 3 times the standard deviation. If the data obeys normal distribution, the distance average
The probability of occurrence of a value other than that is
Belonging to very individual small probability events. If the data does not follow a normal distribution, it can also be described in terms of how many times the standard deviation is away from the mean. 3. Model-based detection: firstly, establishing a data model, wherein the abnormity is objects which cannot be perfectly fitted by the same model; if the model is a collection of clusters, then an anomaly is an object that does not significantly belong to any cluster; when using regression models, anomalies are objects that are relatively far from predicted values.
S12, sorting the data after data cleaning according to the occurrence times of preset words, and taking the preset word with the largest occurrence time as a root word;
specifically, the preset word is mainly some business terms, such as a table in the insurance field, and the business terms may be: premium, applicant and applicant, and the like, are terms relating to insurance. The root word is the basis of word segmentation in the table building process, and the table building data can be effectively classified according to different root words.
And S13, summarizing all the roots, sorting according to the occurrence times from most to least, and generating the root bank.
When sorting is performed, sorting is performed by generally adopting the way that the occurrence times are high to low, namely, the root with the highest occurrence time is arranged at the top, for two or more roots with the same occurrence times, the initial letters or the stroke numbers of the roots are obtained, and the roots with the same occurrence times are sorted according to the initial letters or the stroke numbers.
According to the embodiment, the data of the past table building data are cleaned, so that the effectiveness of the root word in the root word library is guaranteed.
Fig. 3 is a schematic diagram illustrating a first matching result generation process in an automatic table creation method in an embodiment of the present application, where as shown in the drawing, the step S2 of receiving table creation information input by a user, extracting a keyword in the table creation information, and performing positive direction matching between the keyword and a root in the root bank to obtain a first matching result includes:
s21, receiving a table building instruction input by a user, and acquiring table building information required by the user from the table building instruction, wherein the table building information comprises a table field and a table name;
specifically, when a user needs to build a table, the user can send a table building instruction through a table building operation interface generated in advance. The table building operation interface comprises a first input box, a first option and a second option; the first input box is used for a user to select fields, table names, table types and the like of the input table according to the type and the action of the established table; specifically, a field of the table refers to a column name used to store data; for example, a class has 3 students (xiao zhang, xiao Li, queen), if the class represents the name of the table, the name of the class is the field, the table has 3 fields, and then the names of the fields are respectively corresponding to the attributes, for example, if you give them several apples respectively, the number of apples and apples is equivalent to the attribute of the field, and the attribute of the field is equivalent to the attribute of the data stored in the field; the names of the tables can be defined by the developer according to the needs of the development project, and the types of the tables commonly used include page tables, segment tables, file allocation tables, tables in the file storage space, and the like.
S22, extracting key characters in the table fields, and splicing the key characters and the table names to obtain the key words;
specifically, the key characters mainly refer to the characters reflecting the core content of the table, such as the name of the student and the name of the class in the above example, and two splicing modes can be adopted when the key characters and the names of the table are spliced, namely, the key characters are in the front and the names of the table are in the back; or the name of the table is before and the key character is after.
S23, acquiring the number of characters of each root in the root bank, acquiring the number of the characters of the keywords by taking the maximum value N of the number of the characters as the first number of the characters, and taking the number of the characters of the keywords as the second number of the characters;
if the second number of characters is smaller than the first number of characters, matching along the positive direction of the root bank by taking the keywords as query conditions, and extracting the root with the first number of characters equal to the second number of characters as the first matching result;
if the second number of characters is not less than the first number of characters, intercepting the first N characters in the keywords as query conditions, matching along the positive direction of the root bank, and extracting the root of which the first number of characters is equal to the second number of characters as the first matching result.
Specifically, the longest root word number in the root word library is found to be N, and the input Chinese character word number is assumed to be L. If L is larger than N, intercepting the first N Chinese characters as the fields to be matched for root matching, otherwise, taking the whole Chinese character sequence as the fields to be matched for direct matching in a root library. If the word with the word number L exists in the root word, the matching is successful; if the word can not be found in the root word library, the matching is failed, the last word in the field to be matched is removed, the remaining Chinese character sequence is used as the field to be matched to perform matching processing again in the dictionary … …, and the matching is continued until the matching is successful, namely, a word is cut out, or until the length of the remaining word string is 1, namely, a single word is obtained.
In the embodiment, the first matching result of the forward matching is obtained through character number comparison, so that the accuracy of the comparison of the matching results is ensured.
In an embodiment, the step S3, after performing inverse direction matching on the keyword and the root word in the root word library, obtaining a second matching result, includes:
if the second number of characters is smaller than the first number of characters, matching along the reverse direction of the root bank by taking the keywords as query conditions, and extracting the root of which the first number of characters is equal to the second number of characters as a second matching result;
if the second number of characters is not less than the first number of characters, intercepting the last N characters in the keywords as query conditions, matching along the reverse direction of the root bank, and extracting the roots of which the first number of characters is equal to the second number of characters as the first matching results.
In one embodiment, said giving different weights to elements in said first matching result comprises:
dividing elements in the first matching result into single words, word roots or non-word roots according to word attributes;
obtaining the word segmentation quantity of the root word according to the table building information input by the user;
and giving different weights to the single words, the number of the participles and the non-root words.
In one embodiment, after the performing the table-building with the first matching result or the second matching result as a word segmentation scheme, the method further comprises:
respectively acquiring a first table main key for building a table according to the first matching result and a second table main key for building a table according to the second matching result;
and comparing the lengths of the first table main key and the second table main key, if the lengths are consistent, marking the table as successful table building, otherwise, re-scoring the first matching result and the second matching result until the table building is successful.
According to the embodiment, whether the scoring is consistent or not can be checked, and the error of the word segmentation scheme caused by parameter error can be effectively prevented through the verification of the table building main key, so that the error of table building is greatly avoided.
The technical features mentioned in any of the above corresponding embodiments or implementations are also applicable to the embodiment corresponding to fig. 4 in the present application, and the details of the subsequent similarities are not repeated.
The above description is directed to an automatic table building method, and the following description is directed to an apparatus for performing the automatic table building.
Fig. 4 is a block diagram of an automatic table creation apparatus, which can be applied to automatic table creation. The automatic table creation apparatus in the embodiment of the present application can implement the steps corresponding to the automatic table creation method executed in the embodiment corresponding to fig. 1. The function realized by the automatic table building device can be realized by hardware, and can also be realized by executing corresponding software by hardware. The hardware or software includes one or more modules corresponding to the above functions, which may be software and/or hardware.
In one embodiment, an automatic table building apparatus is provided, as shown in fig. 4, including the following modules:
the root library module 10 is configured to acquire the historical tabulation data, extract data with the occurrence frequency larger than a preset frequency threshold value from the historical tabulation data as roots, sort all the roots and generate a root library;
the first matching result module 20 is configured to receive table building information input by a user, extract keywords in the table building information, and perform positive direction matching on the keywords and roots in the root bank to obtain a first matching result;
a second matching result module 30, configured to perform inverse direction matching on the keyword and the root in the root bank to obtain a second matching result;
a first scoring module 40 configured to give different weights to each element in the first matching result, extract a root word in the first matching result, and multiply the root word in the first matching result with the corresponding weight to obtain a first score;
a second scoring module 50 configured to assign different weights to each element in the second matching result, extract a root word in the second matching result, and multiply the root word in the second matching result with the corresponding weight to obtain a second score;
and the table building module 60 is configured to build a table by using the first matching result or the second matching result as a word segmentation scheme if the first score is consistent with the second score, and otherwise, build a table by using a matching result with a low score in the first score and the second score as a word segmentation scheme.
In one embodiment, the first matching result module is further configured to:
if the second number of characters is smaller than the first number of characters, matching along the reverse direction of the root bank by taking the keywords as query conditions, and extracting the root of which the first number of characters is equal to the second number of characters as a second matching result;
if the second number of characters is not less than the first number of characters, intercepting the last N characters in the keywords as query conditions, matching along the reverse direction of the root bank, and extracting the roots of which the first number of characters is equal to the second number of characters as the first matching results.
In one embodiment, a computer device is provided, the computer device comprising a memory and a processor, the memory having stored therein computer-readable instructions, which, when executed by the processor, cause the processor to perform the steps of the automatic table building method in the above embodiments.
In one embodiment, a storage medium is provided that stores computer-readable instructions, which when executed by one or more processors, cause the one or more processors to perform the steps of the automatic table creation method in the above embodiments. Wherein the storage medium may be a non-volatile storage medium.
Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by associated hardware instructed by a program, which may be stored in a computer-readable storage medium, and the storage medium may include: read Only Memory (ROM), random Access Memory (RAM), magnetic or optical disks, and the like.
The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-described embodiments are merely illustrative of some embodiments of the present application, which are described in more detail and detail, but are not to be construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent application shall be subject to the appended claims.
Claims (9)
1. An automatic table building method, comprising:
acquiring historical tabulation data, extracting data with the occurrence frequency larger than a preset frequency threshold value from the historical tabulation data as a root word, and sequencing all the root words to generate a root word library;
receiving table building information input by a user, extracting keywords in the table building information, and performing positive direction matching on the keywords and roots in the root bank to obtain a first matching result;
performing inverse direction matching on the keywords and the roots in the root bank to obtain a second matching result;
dividing elements in the first matching result into single words, root words or non-root words according to the matching result of the root bank;
obtaining the word segmentation quantity of the root word according to the table building information input by the user;
respectively giving the weights multiplied by the number to the single words, the root participles and the non-root words, wherein the weights are determined based on the matching degree of the keywords and the root library;
extracting a root word in the first matching result, and multiplying the root word in the first matching result by the corresponding weight to obtain a first score;
giving different weights to each element in the second matching result, extracting a root word in the second matching result, and multiplying the root word in the second matching result by the corresponding weight to obtain a second score;
and if the first score is consistent with the second score, the first matching result or the second matching result is used as a word segmentation scheme for building a table, otherwise, the matching result with a low score in the first score and the second score is used as the word segmentation scheme for building the table.
2. The automatic table building method according to claim 1, wherein the obtaining of the past table building data, extracting data with occurrence frequency greater than a preset frequency threshold from the past table building data as a root word, and generating a root word library after sorting all the root words comprises:
acquiring a plurality of historical tabular building data, and respectively cleaning the historical tabular building data;
sorting the data after data cleaning according to the occurrence frequency of preset words, and taking the preset word with the most occurrence frequency as a root word;
and summarizing all roots, and sequencing according to the occurrence times to generate the root bank.
3. The automatic table building method of claim 1, wherein the receiving table building information input by a user, extracting keywords in the table building information, and performing positive direction matching between the keywords and the roots in the root bank to obtain a first matching result comprises:
receiving a table building instruction input by a user, and acquiring table building information required by the user from the table building instruction, wherein the table building information comprises a table field and a table name;
extracting key characters in the table fields, and splicing the key characters and the table names to obtain the key words;
acquiring the character number of each root in the root bank, acquiring the character number of the keyword by taking the maximum value N of the character number as a first character number, and taking the character number of the keyword as a second character number;
if the second number of characters is smaller than the first number of characters, matching along the positive direction of the root bank by taking the keywords as query conditions, and extracting the root with the first number of characters equal to the second number of characters as the first matching result;
if the second number of characters is not less than the first number of characters, intercepting the first N characters in the keywords as query conditions, matching along the positive direction of the root bank, and extracting the root of which the first number of characters is equal to the second number of characters as the first matching result.
4. The automatic table building method according to claim 3, wherein said obtaining a second matching result after performing inverse direction matching on said keyword and a root in said root bank comprises:
if the second number of characters is smaller than the first number of characters, matching along the reverse direction of the root bank by taking the keywords as query conditions, and extracting the root of which the first number of characters is equal to the second number of characters as a second matching result;
if the second number of characters is not less than the first number of characters, intercepting the last N characters in the keywords as query conditions, matching along the reverse direction of the root bank, and extracting the roots of which the first number of characters is equal to the second number of characters as the first matching results.
5. The automatic table creation method according to any one of claims 1 to 4, wherein after the table creation is performed with the first matching result or the second matching result as a word segmentation scheme, the method further comprises:
respectively acquiring a first table main key for building a table according to the first matching result and a second table main key for building a table according to the second matching result;
and comparing the lengths of the first table main key and the second table main key, if the lengths are consistent, marking the table building to be successful, otherwise, re-scoring the first matching result and the second matching result until the table building is successful.
6. An automatic table building device is characterized by comprising the following modules:
the root library module is used for acquiring the historical tabulation data, extracting data with the occurrence frequency larger than a preset frequency threshold value from the historical tabulation data as roots, and sequencing all the roots to generate a root library;
the first matching result module is used for receiving the table building information input by the user, extracting keywords in the table building information, and performing positive direction matching on the keywords and the roots in the root bank to obtain a first matching result;
the second matching result module is used for obtaining a second matching result after the keyword is subjected to inverse direction matching with the root word in the root word library;
the first grading module is used for dividing elements in the first matching result into single words, root words or non-root words according to the matching result of the root bank; obtaining the word segmentation quantity of the root word according to the table building information input by the user; respectively giving the weights multiplied by the number to the single words, the root participles and the non-root words, wherein the weights are determined based on the matching degree of the keywords and the root library; extracting a root word in the first matching result, and multiplying the root word in the first matching result by the corresponding weight to obtain a first score;
the second scoring module is configured to give different weights to each element in the second matching result, extract a root word in the second matching result, and multiply the root word in the second matching result and the corresponding weight to obtain a second score;
and the table building module is set to build a table by taking the first matching result or the second matching result as a word segmentation scheme if the first score is consistent with the second score, or else, build a table by taking the matching result with the lower score in the first score and the second score as the word segmentation scheme.
7. The automatic table building apparatus of claim 6, wherein the first matching result generating module is further configured to:
if the second number of characters is smaller than the first number of characters, matching along the reverse direction of the root bank by taking the keywords as query conditions, and extracting the root of which the first number of characters is equal to the second number of characters as a second matching result;
if the second number of characters is not less than the first number of characters, intercepting the last N characters in the keywords as query conditions, matching along the reverse direction of the root bank, and extracting the roots of which the first number of characters is equal to the second number of characters as the first matching results.
8. A computer device comprising a memory and a processor, the memory having stored therein computer-readable instructions, wherein the computer-readable instructions, when executed by the processor, cause the processor to perform the steps of the method of automatically creating tables according to any one of claims 1 to 5.
9. A storage medium having computer-readable instructions stored thereon which, when executed by one or more processors, cause the one or more processors to perform the steps of the method of automatically creating tables of any of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911371969.9A CN111191430B (en) | 2019-12-27 | 2019-12-27 | Automatic table building method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911371969.9A CN111191430B (en) | 2019-12-27 | 2019-12-27 | Automatic table building method and device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111191430A CN111191430A (en) | 2020-05-22 |
| CN111191430B true CN111191430B (en) | 2023-02-14 |
Family
ID=70707694
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911371969.9A Active CN111191430B (en) | 2019-12-27 | 2019-12-27 | Automatic table building method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111191430B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI779738B (en) * | 2021-07-22 | 2022-10-01 | 中華電信股份有限公司 | Search system and search method for keyword of log file |
| CN115982137B (en) * | 2023-03-17 | 2023-08-01 | 鲁班(北京)电子商务科技有限公司 | Data name and database table building generation method and system |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080077570A1 (en) * | 2004-10-25 | 2008-03-27 | Infovell, Inc. | Full Text Query and Search Systems and Method of Use |
| CN100444591C (en) * | 2006-08-18 | 2008-12-17 | 北京金山软件有限公司 | Method for acquiring front-page keyword and its application system |
| CN103473373A (en) * | 2013-09-29 | 2013-12-25 | 方正国际软件有限公司 | Threshold matching model-based similarity analysis system and threshold matching model-based similarity analysis method |
| CN105718473B (en) * | 2014-12-05 | 2019-01-25 | 成都复晓科技有限公司 | A kind of method of data modeling |
| CN106488050B (en) * | 2015-09-01 | 2019-06-07 | 阿里巴巴集团控股有限公司 | Interactive voice answering method and system |
| CN106951415A (en) * | 2017-04-01 | 2017-07-14 | 银联智策顾问（上海）有限公司 | A kind of name of firm searching method and device |
| CN108399161A (en) * | 2018-03-06 | 2018-08-14 | 平安科技（深圳）有限公司 | Advertising pictures identification method, electronic device and readable storage medium storing program for executing |
| CN108763341B (en) * | 2018-05-14 | 2023-07-25 | 中国平安人寿保险股份有限公司 | Electronic device, automatic table building method and storage medium |
-
2019
- 2019-12-27 CN CN201911371969.9A patent/CN111191430B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN111191430A (en) | 2020-05-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109885692B (en) | Knowledge data storage method, apparatus, computer device and storage medium | |
| CN109886294B (en) | Knowledge fusion method, apparatus, computer device and storage medium | |
| US8112421B2 (en) | Query selection for effectively learning ranking functions | |
| KR100688121B1 (en) | Method and apparatus for retrieving, accumulating, and sorting table-formatted data | |
| Menestrina et al. | Evaluating entity resolution results | |
| CN107862070B (en) | Online classroom discussion short text instant grouping method and system based on text clustering | |
| Herdagdelen et al. | Generalized syntactic and semantic models of query reformulation | |
| US8510312B1 (en) | Automatic metadata identification | |
| CN110795524B (en) | Main data mapping processing method and device, computer equipment and storage medium | |
| AU2014201516A1 (en) | Resolving similar entities from a transaction database | |
| CN109471889B (en) | Report accelerating method, system, computer equipment and storage medium | |
| WO2008106668A1 (en) | User query mining for advertising matching | |
| US8037069B2 (en) | Membership checking of digital text | |
| CN110659282B (en) | Data route construction method, device, computer equipment and storage medium | |
| CN111191430B (en) | Automatic table building method and device, computer equipment and storage medium | |
| KR20160100752A (en) | System and method for processing and analysing big data provding efficiently using columnar index data format | |
| CN110807102A (en) | Knowledge fusion method and device, computer equipment and storage medium | |
| CN109885641B (en) | Method and system for searching Chinese full text in database | |
| US20230205755A1 (en) | Methods and systems for improved search for data loss prevention | |
| US8548999B1 (en) | Query expansion | |
| CN110188196A (en) | A kind of text increment dimension reduction method based on random forest | |
| CN112395881B (en) | Material label construction method and device, readable storage medium and electronic equipment | |
| CN112948429B (en) | Data reporting method, device and equipment | |
| US20210182549A1 (en) | Natural Language Processing (NLP) Pipeline for Automated Attribute Extraction | |
| CN116450916A (en) | Information query method and device based on fixed-segment classification, electronic equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |