JP6586850B2 - Table reconstruction apparatus and method - Google Patents

Description

  The present invention relates to the field of information processing, and more specifically, to a table reconstruction apparatus and method.

  Along with the rapid development of the Internet, a large amount of table data appears on the Internet, and the table data includes abundant structured information. By digging out table data, a large amount of human knowledge can be obtained. However, table data generally has a complicated structure, duplicate fields, and one table includes a plurality of description objects and their attributes. Because some attributes can describe multiple description objects, to extract knowledge from a table, it is necessary to first restructure the table with a simple structure and a single description object. Can acquire knowledge. For example, the table of FIG. 1A includes two description objects “person” and “company”, and among these attributes, “name, nationality, degree, graduation school, founding company, contact method, e−” mail ”, and the attribute describing the company includes“ date of establishment, address, service, field, contact method ”, and the column of“ contact method ”is an attribute describing a person and an attribute describing a company But there is.

  Therefore, there is a need for an apparatus and method for reconfiguring a table so that the structure is simple and the description target is a single description target table having a single format.

  An object of the present invention is to provide an apparatus and a method for reconfiguring a table into a single description target table having a simple structure and a single description target.

  The outline of the present invention described below shows the present invention from several aspects and promotes a basic understanding thereof. It should be understood that this summary is not exhaustive. This summary is not intended to identify key points or important parts of the invention or to limit the scope of the invention. Its purpose is merely to present some concepts briefly for the purpose of further description.

  One main object of the present invention is to provide a table reconstruction device, the device comprising a column attribute determination unit configured to determine an attribute for each column in the table, and for each determined column. A description object determination unit configured to determine a description object in the table by linking an attribute with the vocabulary knowledge base; and determining an attribute set of each description object in the vocabulary knowledge base; A description target related attribute determination unit configured to calculate a first similarity between an attribute for each column and each attribute in the attribute set, and to determine an attribute related to the description target in the table; A table reconstruction unit configured to reconstruct the table using the description object and attributes associated with the description object;

  According to one aspect of the present invention, a table reconstruction method is provided, the method comprising: determining a column-by-column attribute in the table; and linking the determined column-by-column attribute with a lexical knowledge base. The step of determining the description target of the table and the attribute set of each description target in the lexical knowledge base are determined, and the first similarity between the attribute for each column in the table and each attribute in the attribute set is calculated. Then, a step of determining an attribute related to the description target in the table, and a step of reconfiguring the table using the description target and the attribute related to the description target are included.

  The embodiment of the present invention further provides a computer program for realizing the above method.

  The embodiment of the present invention further provides a computer program product in a medium format that can be read at least by a computer, and a computer program for realizing the above method is stored in the computer program product.

  The above and other advantages of the present invention will become more apparent by describing in detail preferred embodiments of the present invention with reference to the drawings.

In order that the above and other objects, features and advantages of the present invention may be more easily understood, embodiments of the present invention will be described below with reference to the drawings. The parts in the drawings are only intended to illustrate the principles of the invention. In the drawings, the same or similar technical features or parts are denoted by the same or similar reference numerals.
It is a figure which shows the example table containing several description object. It is a figure of the table of one single description object reconfigure | reconstructed from the table of FIG. 1A. It is a figure of the table of another single description object reconfigure | reconstructed from the table of FIG. 1A. FIG. 3 is a block diagram illustrating an exemplary configuration of an apparatus 200 for reconfiguring a table, in accordance with one embodiment of the present invention. FIG. 3 is a block diagram illustrating an exemplary configuration of a description target determination unit 204 in FIG. 2. 4 is a flowchart of a method 400 for reconstructing a table according to an embodiment of the present invention. FIG. 5 is a flowchart illustrating an exemplary process of step S404 of FIG. FIG. 2 is an exemplary structural diagram illustrating a computer device used to implement the table reconstruction device and method of the present invention.

  Embodiments of the present invention will be described below with reference to the drawings. Elements and features shown in one drawing or embodiment of the invention may be combined with elements and features shown in one or more other drawings or embodiments. For the sake of clarity, the display and description of parts and processes known to those skilled in the art, which are not related to the present invention, are omitted.

  The present invention provides a table reorganization apparatus and method capable of decomposing a table including a plurality of description objects (as shown in FIG. 1A) into a plurality of single description object tables (for example, as shown in FIGS. 1B and 1C). . As shown in FIGS. 1A to 1C, the table of FIG. 1A includes two description objects “person” and “company”, and among these attributes, “name, nationality, degree, graduation school, founding” There are "company, contact method, e-mail", and attributes describing the company include "established date, address, service, field, contact method", and "contact method" is an attribute that describes a person, It is also an attribute that describes the company. FIGS. 1B and 1C show two single description object tables reconfigured based on the two description objects “people” and “company” and their corresponding attributes in FIG. 1A, respectively.

  According to the apparatus and method of the present invention, first, the attribute corresponding to the column is determined based on the table column title and / or the example information in the table column, and the description target is determined based on the corresponding attribute of each column. Finally, the attribute column is determined, and finally, the original table is reconfigured into a single description target table based on the correspondence between the description target and the attribute column.

  Hereinafter, an apparatus for reconfiguring a table according to an embodiment of the present invention will be described in detail with reference to the drawings.

  FIG. 2 is a block diagram illustrating an exemplary configuration of an apparatus 200 for reconfiguring a table according to one embodiment of the present invention.

  As shown in FIG. 2, the table reconstruction device 200 includes a column attribute determination unit 202, a description target determination unit 204, a description target related attribute determination unit 206, and a table reconstruction unit 208.

  The column attribute determination unit 202 can determine the attribute for each column in the table.

  The description target determination unit 204 can determine the description target in the table by linking the determined attribute for each column with the vocabulary knowledge base.

  The description target related attribute determination unit 206 determines the attribute set of each description target in the vocabulary knowledge base, calculates the first similarity between the attribute for each column in the table and each attribute in the attribute set, It is possible to determine the attribute related to the description target in the table.

  The table restructuring unit 208 can reconfigure the table using the description object and the attributes related to the description object.

  Hereinafter, each unit included in the table reconstruction device 200 will be described in detail.

  Each column of the table indicates an attribute to be described. By confirming which attribute of the vocabulary knowledge base the column of the table corresponds to, the attribute corresponding to the column of the table can be determined. As an example, the vocabulary knowledge base can be a LOD (Linked Open Data, a kind of global knowledge base) knowledge base, and an example using the LOD knowledge base as a vocabulary knowledge base will be described below.

  In one exemplary embodiment of the present invention, two aspects of information, column title and column instance information, can be considered when determining the attributes corresponding to the columns of the table. If a column title exists in the table, the table column title is usually the first row of the table, and the corresponding attribute in the LOD knowledge base is obtained by linking the table column title with the attribute in the LOD knowledge base. Can do. When the corresponding LOD attribute is not found, the identification can be performed again by the identification method based on the column example. When there is no column title in the table, identification is performed using an identification method based on a direct column example.

  Hereinafter, an exemplary method for determining the attribute for each column in the table based on the column example information will be described.

  First, a column example having a specific pattern is identified using a regular expression. For example, postal code, telephone number (mobile number, fixed telephone number), network URL, IP address, E-mail, time, date, etc. It can be determined that the attributes in the sixth, seventh, and eighth columns of the table in FIG. 1A are the establishment date, the contact method, and e-mail.

  Next, the contents such as the name and address are identified. The personal name can be identified using a surname / dictionary dictionary. Matching is performed using an address knowledge base having a stage relationship with respect to the address, and the stage relationship of the address knowledge base is, from the top to the bottom, the state, province / autonomous region, ward prefecture, village, town, and the like. It is also possible to determine the address using a search engine, that is, when the column example is searched using the map search engine and the search result is returned from the map search engine, the attribute corresponding to the column example Is an address. It can be determined that the attributes corresponding to the first column and the ninth column in FIG. 1A are a name and an address, respectively.

  Finally, the column example corresponding to the column with unknown attribute is linked with the entity in the LOD knowledge base, and the attribute corresponding to the table column is determined using the knowledge in the LOD knowledge base. For example, by linking the fourth column in FIG. 1A with the LOD knowledge base, it can be seen that the attributes corresponding to “Tsinghua University” and “Stanford University” are both “university” based on the LOD knowledge base. By such a method, it can be determined that the attributes corresponding to the second, third, fourth, fifth, tenth and eleventh are nationality, degree, graduation school, founding company, service and field, respectively.

As an example, using global cooperative identification to identify the corresponding attribute of a table column, i.e., the identification result is not dependent on a particular column instance, but is the result of the cooperative identification of multiple column instances in that column. For example, there are n instances in a column, of which corresponds to m attributes p _a, if n-m number corresponds to the attribute p _b, eventually based on the type of said column to a predetermined condition Can be determined. If m> (n-m) and m / n is larger than a predetermined value, the attribute corresponding to said column is a _{p a.} If the predetermined condition is not satisfied, the column cannot be identified and the column may be ignored.

  Through the above process, the column attribute determination unit 202 can determine the attribute for each column in the table.

  Subsequently, the description target determination unit 204 can determine the description target in the table by linking the determined attribute for each column with the vocabulary knowledge base.

  FIG. 3 is a block diagram showing an exemplary configuration of the description target determination unit 204 of FIG.

  As shown in FIG. 3, the description target determination unit 204 includes an entity column number determination subunit 2042, a description target set determination subunit 2044, a description target condition probability determination subunit 2046, and a description target determination subunit 2048.

  In many cases, the description target is an entity, and the entity column number determination subunit 2042 finds a column including the entity based on the link result with the LOD knowledge base obtained by the column attribute determination unit 202, and includes the entity. Obtain the number of columns and write N.

If the attribute p _j is known, the probability P (s _i | p _j ) can be statistics from the LOD. Here, s _i is a description target variable, p _j is an attribute variable, and P (s _i | p _j ) is a probability that s _i can be described by p _j when a certain attribute p _j is known. means.

Assuming that the table T has c columns, one description target set S _j is obtained from the LOD for any of the attributes p _j (j = 1, 2,... C) for each column in the table, and the total It is possible to obtain c description object sets. Using the description target set determination subunit 2044, a description target set corresponding to the attribute for each column can be obtained.

Each description object s _{i in} each description object set has one conditional probability P (s _i | p _j ), and the set S is expressed as a set of <s _i , P (s _i | p _j )>. May be. The description target condition probability determination subunit 2046 can determine a condition probability that can describe each description target in the description target set corresponding to the attribute for each column based on the LOD knowledge base.

There is a possibility that one of the described subject s _i is described by a plurality of attributes, s _i is can appear in multiple descriptions target set, a plurality of conditions the probability that s _i is described by a different attribute P ( s _i | p ₁ ), P (s _i | p ₂ )...

  The description target determination subunit 2048 can determine the description target of the table T based on the condition probabilities of each description target in all description target sets.

As an example, the description target determination subunit 2048 calculates the average probability P (s _i ) of the conditional probabilities in the set of c description targets for each description target s _{i, and} sets P (s _i ) from the larger one to the smaller one. By arranging them in order, the previous N description objects can be used as the description object of the table T.

In another example, the description target determination subunit 2048 calculates the _sum P (s _i ) _sum of all condition probabilities in the set of c description objects for each description target s _{i, and} increases P (s _i ) _sum from the larger one. By arranging them in order from the smallest one, the previous N description objects can be used as the description object of the table T.

For each description target s _i determined by the description target determination unit 204, the description target related attribute determination unit 206 obtains the attribute set Ps _i of each description target s _i from the LOD, and each attribute p in the attribute set Ps _{i s} and similarity sim between attributes _{p t} of each row in the table _(p s, _{p t)} is calculated, the similarity sim _(p s, _{p t)} columns associated with recitations in the table on the basis of It can be confirmed. For example, when the similarity sim (p _s , p _t ) is larger than a predetermined threshold, it is considered that the attribute p _t and the description target s _i are related.

Those skilled in the art are familiar with the specific means or schemes that can be used to calculate the similarity between the attributes p _s and p _t and will not be described in detail here.

Calculated in an exemplary embodiment, description target related attribute determination unit 206 further the similarity sim _(v s, _{v t)} of the Illustrative _{v t} for each column in the value _{v s} and the table for each attribute _{p s} To do. Then, to calculate the weighted sum of the similarity sim _(p s, p _t) and the similarity _{sim (v s, v t)} .

^{_{sim = α * sim (p s}} , p t) + (1-α) sim (v s, v t), where, alpha is a predetermined coefficient.

Based on the weighted similarity sim, a column related to the description target in the table can be determined. When sim is larger than a predetermined threshold, the attribute _pt and the description target s _i are related.

In one exemplary embodiment, described target related attribute determination unit 206 further, when not found associated attributes DESCRIPTION subject s _i in the table column, the attribute in the attribute sets of the description is of an attribute in the table It can be determined that it is not related to any of them, and the description object s _i can be ignored.

  When the column description determination unit 202, the description target determination unit 204, and the description target related attribute determination unit 206 have already determined the table description target and the attribute related to the description target, the table reconstruction unit 208 determines the description target And the attribute related to the description object can be used to reconstruct the table.

  The table 200 shown in FIG. 1A is reconfigured by the apparatus 200 for reconfiguring the table according to an embodiment of the present invention, and the structure shown in FIGS. It can be a single table.

  As mentioned above, it is obvious that some processes or methods are also disclosed in the description of the apparatus for reconstructing the table according to the embodiment of the present invention. The following outlines these methods but does not duplicate some of the details already discussed in the preamble. Although these methods are disclosed in the overview of the apparatus for reconfiguring the table, it should be noted that these methods are not necessarily used by the above-described parts or executed by these parts. Should. For example, some or all of the embodiments of the apparatus for reconfiguring the table can be realized by hardware and / or firmware, and the computer can execute all the methods for reconfiguring the table to be discussed below. It can also be realized by a program, and these methods can use hardware and / or firmware of a device for reconfiguring the table.

  It should be noted here that the structure of the apparatus 200 for reconfiguring the table shown in FIG. 2-3 and the structure of its constituent units are merely examples, and those skilled in the art can change the structural block diagram shown in FIG. 2-3 as needed. Is possible.

  FIG. 4 is a flowchart illustrating a method 400 for reconstructing a table according to one embodiment of the present invention.

  First, in step S402, the attribute for each column in the table is determined.

  Specifically, when a table has a column title, the column title of the table is linked with an attribute in the vocabulary knowledge base to obtain a corresponding attribute for each column. If the table does not have a column title, one of the regular expression expression identification, the surname dictionary identification, the address knowledge base identification, the map search engine identification, and the linking to an entity in the vocabulary knowledge base Alternatively, attributes for each column in the table are determined using a plurality of identification methods.

  Subsequently, in step S404, the description target in the table is determined by linking the determined attribute for each column with the vocabulary knowledge base.

  FIG. 5 is a flowchart showing an exemplary process of step S404 in FIG. 4 (determining the description target in the table by linking the determined attribute for each column with the vocabulary knowledge base).

  As shown in FIG. 5, in step S4042, the number N of columns including the entity is obtained based on the attribute for each column determined in step S302.

  In step S4044, a description target set corresponding to the attribute for each column is obtained.

  Subsequently, in step S4046, based on the vocabulary knowledge base, the condition probability that each description object in the description object set corresponding to the attribute for each column can be described by the attribute is determined.

  Finally, in step S4048, the description target of the table can be determined based on the condition probability of each description target.

As an example, the step of determining the description object of the table based on the condition probability of each description object calculates the average probability P (s _i ) of the condition probabilities in the set of c description objects for each description object s _i , P (S _i ) may be arranged in order from the largest to the smallest to include the previous N description objects as the description objects of the table T.

In another example, the step of determining the description target of the table based on the condition probabilities of each description target is to calculate a total P (s _i ) _sum of all condition probabilities in the set of c description targets for each description target s _i. Then, P (s _i ) _sum may be arranged in order from the largest to the smallest, and the previous N description objects may be set as the description objects of the table T.

For each description object _{s i} which is determined in the step S404, in step S406, obtains the attribute set Ps _i DESCRIPTION subject _{s i} from LOD, attributes of each column in each attribute _{p s} and the table in the attribute set Ps _i similarity sim _(p s, p _t) and p _t is calculated, it is possible to determine the sequence associated with the recitations in the table based on the similarity _{sim (p s, p t)} . For example, when the similarity sim (p _s , p _t ) is larger than a predetermined threshold, it is considered that the attribute p _t and the description target s _i are related.

In one embodiment, further, it calculates the similarity sim _(v s, _{v t)} of the Illustrative _{v t} for each column in the value _{v s} and the table for each attribute _{p s.} Then, to calculate the weighted sum of the similarity sim _(p s, p _t) and the similarity _{sim (v s, v t)} .

^{_{sim = α * sim (p s}} , p t) + (1-α) sim (v s, v t), of which, α is a predetermined coefficient.

A column related to the description target in the table can be determined based on the weighted similarity sim. If sim is greater than a predetermined threshold value, _{p t} the recitations _{s i} is associated.

In an exemplary embodiment, if a related attribute of a description object s _i is not found from the table sequence, it is determined that an attribute in the attribute set of the description object is not related to any of the attributes in the table; The description object s _i may be ignored.

  Finally, in step S408, the table can be reconfigured based on the description target of the table determined in steps S402 to S406 and the attributes related to the description target.

  The table 400 shown in FIG. 1A is reconfigured by the method 400 for reconfiguring the table according to one embodiment of the present invention, and the structure as shown in FIGS. It can be a single table for single description.

  Details of each step of the method 400 for restructuring the table can be referred to the embodiment of the apparatus for restructuring the table of the present invention described with reference to FIGS.

  The table reconstructing apparatus and method provided by the present invention have a complicated structure, duplicate fields, reconstruct a table including a plurality of description objects and attributes, and have a simple structure and a single description object. Knowledge is efficiently acquired by using a single description target table.

  Although the basic principle of the present invention has been described above using specific embodiments, those skilled in the art will recognize all or any step or part of the method and apparatus of the present invention in any computer device (processor, memory). It can be understood that the present invention can be realized by hardware, firmware, software, or a combination thereof in a network of computer devices). This can be achieved by those skilled in the art after reading the description of the present invention and applying their programming basic skills.

  Therefore, the object of the present invention can be realized by executing one program or a set of programs in an arbitrary computer apparatus. The computer device may be a known general-purpose device. Therefore, the object of the present invention can be realized only by providing a program product including a program code for realizing the above method or apparatus. That is, such a program product also belongs to the present invention, and a storage medium storing such a program product also belongs to the present invention. Needless to say, the storage medium may be any known storage medium or any storage medium that may be developed in the future.

  When the embodiment of the present invention is realized by software and / or firmware, a program that configures the software from a storage medium or a network to a computer having a dedicated hardware structure, for example, the general-purpose computer 600 shown in FIG. When installed and various programs are installed in the computer, various functions and the like can be realized.

  In FIG. 6, a central processing unit (CPU) 601 performs various processes based on a program stored in a read-only memory (ROM) 602 or a program uploaded from a storage unit 608 to a random access memory (RAM) 603. Do. If necessary, the RAM 603 stores data necessary for the CPU 601 to execute various processes. The CPU 601, ROM 602, and RAM 603 are linked to each other by a bus 604. Input / output interface 605 is also linked to bus 604.

  The following parts link to the input / output interface 605: an input unit 606 (including a keyboard, a mouse, etc.), an output unit 607 (including a display such as a cathode ray tube (CRT) or a liquid crystal display (LCD), a speaker, etc.), A storage unit 608 (including a hard disk or the like), a communication unit 609 (including a network interface such as a LAN card, a modem, etc.) The communication unit 609 performs communication processing via a network such as the Internet. Can also be linked to the input / output interface 605. A removable medium 611, for example, a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is mounted on the drive 610, and a computer program is read out from the medium as necessary. Write It can be installed in the section 608.

  When the above-described series of processing is realized by software, a program constituting the software is installed from a network such as the Internet or a storage medium such as a removable medium 611.

  Those skilled in the art should understand that the storage medium is not limited to the removable medium 611 that stores the program and provides the program to the user apart from the apparatus as shown in FIG. Examples of the removable medium 611 include a magnetic disk (floppy disk (registered trademark)), an optical disk (including an optical disk read-only memory (CD-ROM) and a data multipurpose disk (DVD)), a magneto-optical disk (mini disk (MD ) (Registered trademark)) and a semiconductor memory. Alternatively, the storage medium may be a ROM 602, a hard disk included in the storage unit 608, etc., in which a program is stored and provided to a user together with a device including these.

  The present invention further provides a program including instructions readable by a machine. When the instructions are read and executed by the apparatus, the above-described method according to the embodiment of the present invention can be executed.

  Accordingly, a storage medium for storing a program of instructions that can be read by the machine is also included in the present invention. Storage media include, but are not limited to, floppy disks, optical disks, magneto-optical disks, memory cards, memory sticks, and the like.

  Those skilled in the art should understand that what has been described here is illustrative and the present invention is not limited thereto.

  In the present specification, expressions such as “first”, “second”, and “Nth” are used to clearly distinguish the features of the present invention in order to clearly describe the present invention. Therefore, these expressions do not have any limiting meaning.

  As an example, each step of the method and each component and / or unit of the device may be implemented by software, firmware, hardware, or a combination thereof and may be part of an associated device. Since those skilled in the art are familiar with the means and methods used to configure each component and unit of the above-described apparatus by software, firmware, hardware, or a combination thereof, they are omitted here.

  As an example, when realized by software or firmware, a program constituting the software is installed from a storage medium or a network to a computer having a dedicated hardware structure (for example, the general-purpose computer 600 shown in FIG. 6), When various programs are installed in the computer, various functions can be realized.

  In the specific embodiments of the present invention described above, the features described and / or illustrated by one embodiment may be applied to one or more other embodiments in the same or similar manner to other implementations. It can be combined with features of the form, or can replace features of other embodiments.

  As used herein, the term “include / include” means the presence of a feature, element, step or component, but the presence or addition of one or more other features, elements, steps or components. It is not excluded.

  Furthermore, the method of the present invention is not limited to the execution in time order described in the specification, and may be executed in parallel or individually according to other time orders. Accordingly, the execution order of the methods described herein does not limit the technical scope of the present invention.

  While the invention and its advantages have been described, various modifications, substitutions or transformations may be made within the spirit and scope of the invention as defined by the appended claims. In addition, the scope of the present invention is not limited to the specific embodiments of the processes, devices, means, methods and steps described in the specification. Those skilled in the art can, based on the present invention, carry out basically the same functions as those corresponding to these embodiments, or develop the present or future in which basically the same results can be obtained. It can be easily understood that various processes, devices, means, methods or steps can be used. Accordingly, the appended claims are intended to include within their scope these processes, apparatus, means, methods, or steps.

  As can be understood from the above description, at least the following technical solutions are disclosed.

(Appendix 1)
A table reconstruction device,
A column attribute determination unit configured to determine an attribute for each column in the table;
A description object determination unit configured to determine a description object in the table by linking a determined attribute for each column with a vocabulary knowledge base;
Determine the attribute set of each description object in the vocabulary knowledge base, calculate the first similarity between the attribute for each column in the table and each attribute in the attribute set, and the description object in the table A description target related attribute determination unit configured to determine an attribute related to
An apparatus comprising: a table reconstruction unit configured to reconstruct a table using the description object and attributes associated with the description object.

(Appendix 2)
The column attribute determination unit further includes:
If the table has a column title, link the column title of the table with an attribute in the vocabulary knowledge base to obtain a corresponding attribute for each column;
If the table does not have a column title, among a method of identification by regular expression expression, identification by surname dictionary, identification by address knowledge base, identification by map search engine, and identification by linking with an entity in the vocabulary knowledge base The apparatus of claim 1 configured to determine attributes for each column in the table using one or more identification methods.

(Appendix 3)
The description target determination unit is:
An entity column number determining subunit configured to obtain a column number N of columns including the entity based on the determined attribute for each column;
A description object set determination subunit configured to obtain a description object set corresponding to the attribute for each column;
A description target condition probability determination subunit configured to determine a condition probability that each description target in the description target set corresponding to the attribute for each column can be described by the attribute based on the vocabulary knowledge base;
The apparatus according to claim 1, further comprising: a description object determination subunit configured to determine a description object of the table based on a condition probability of each description object.

(Appendix 4)
The description target determination subunit further includes:
Each description object calculates the sum of condition probabilities or average condition probability in all description object sets,
The apparatus according to attachment 3, wherein the sum of the conditional probabilities of each description object or the average condition probability is arranged in order from the largest to the smallest, and the previous N description objects are set as the description objects of the table. .

(Appendix 5)
The description target related attribute determination unit further includes:
Calculating the second similarity between the example of each column in the table and the value of each attribute in the attribute set;
Obtain a weighted sum of the first similarity and the second similarity, obtain a weighted similarity,
The apparatus according to claim 1, configured to determine an attribute of a column having the weighted similarity greater than a predetermined threshold as an attribute related to the description target in the table.

(Appendix 6)
The description target related attribute determination unit further includes:
The apparatus of claim 1, configured to discard a description object if an attribute in the attribute set of the description object is not related to any of the attributes in the table.

(Appendix 7)
The column attribute determination unit further includes:
When an example in a column corresponds to a plurality of attributes, the column attribute is determined based on a predetermined condition,
The apparatus of claim 1 configured to ignore the column if a predetermined condition is not met.

(Appendix 8)
The apparatus according to claim 7, wherein, as the predetermined condition, when the number of examples corresponding to an attribute among a plurality of attributes is larger than the number of examples corresponding to other attributes, the attribute is determined as an attribute of the column. .

(Appendix 9)
The apparatus according to appendix 1, wherein the reconstructed table is a single description target table.

(Appendix 10)
A table reconstruction method,
Determining the attributes for each column in the table;
Determining the description target of the table by linking the determined attribute for each column with the vocabulary knowledge base;
Determine the attribute set of each description object in the vocabulary knowledge base, calculate the first similarity between the attribute for each column in the table and each attribute in the attribute set, and the description object in the table Determining the attributes associated with
Reconfiguring a table using the description object and attributes associated with the description object.

(Appendix 11)
The step of determining the attributes for each column in the table is:
If the table has a column title, linking the column title of the table with an attribute in the vocabulary knowledge base to obtain a corresponding attribute for each column;
If the table does not have a column title, among a method of identification by regular expression expression, identification by surname dictionary, identification by address knowledge base, identification by map search engine, and identification by linking with an entity in the vocabulary knowledge base The method according to claim 10, further comprising the step of: determining an attribute for each column in the table using one or more identification methods.

(Appendix 12)
The step of determining the description target of the table by linking the determined attribute for each column with the vocabulary knowledge base,
Obtaining a column number N of columns including the entity based on the determined attribute for each column;
Obtaining a description target set corresponding to the attribute for each column;
Determining, based on the vocabulary knowledge base, a condition probability that each description object in the description object set corresponding to the attribute for each column can be described by the attribute;
The method according to claim 10, further comprising: determining a description target of the table based on a condition probability of each description target.

(Appendix 13)
Based on the condition probability of each description object, the step of determining the description object of the table includes:
Calculating a sum of condition probabilities or an average condition probability for each description object set in all description object sets;
13. The method according to appendix 12, comprising: arranging the sum of the condition probabilities or the average condition probabilities of each description object in order from the largest to the smallest, and setting the previous N description objects as the description objects of the table.

(Appendix 14)
The step of determining an attribute related to the description object in the table further includes:
Calculating a second similarity between an example for each column in the table and a value of each attribute in the attribute set;
Obtaining a weighted sum of the first similarity and the second similarity to obtain a weighted similarity;
The method according to claim 10, including the step of determining an attribute of the column having the weighted similarity greater than a predetermined threshold as an attribute related to the description object in the table.

(Appendix 15)
The step of determining the attribute set of each description target in the vocabulary knowledge base and calculating the first similarity between the attribute for each column in the table and each attribute in the attribute set further includes:
The method according to claim 10, comprising the step of discarding the description object when the attribute in the attribute set of the description object is not related to any of the attributes in the table.

(Appendix 16)
The step of determining the attributes for each column in the table is:
If an instance in a column corresponds to multiple attributes, determining the column attribute based on a predetermined condition;
The method according to claim 10, comprising the step of ignoring the column if the predetermined condition is not satisfied.

(Appendix 17)
The method according to claim 16, wherein, as the predetermined condition, when the number of instances corresponding to an attribute among a plurality of attributes is larger than the number of instances corresponding to other attributes, the attribute is determined as an attribute of the column. .

(Appendix 18)
The method according to appendix 10, wherein the reconstructed table is a single description target table.

202 Column attribute determination unit 204 Description target determination unit 206 Description target related attribute determination unit 208 Table reconstruction unit 2042 Substring number determination subunit 2044 Description target set determination subunit 2046 Description target condition probability determination subunit 2048 Description target determination subunit 605 Input / output interface 606 Input unit 607 Output unit 608 Storage unit 609 Communication unit 610 Drive 611 Removable medium

Claims (10)

A table reconstruction device,
A column attribute determination unit configured to determine an attribute for each column in the table;
A description object determination unit configured to determine a description object in the table by linking a determined attribute for each column with a vocabulary knowledge base;
Determine the attribute set of each description object in the vocabulary knowledge base, calculate the first similarity between the attribute for each column in the table and each attribute in the attribute set, and the description object in the table A description target related attribute determination unit configured to determine an attribute related to
An apparatus comprising: a table reconstruction unit configured to reconstruct a table using the description object and attributes associated with the description object. The column attribute determination unit further includes:
If the table has a column title, link the column title of the table with an attribute in the vocabulary knowledge base to obtain a corresponding attribute for each column;
If the table does not have a column title, among a method of identification by regular expression expression, identification by surname dictionary, identification by address knowledge base, identification by map search engine, and identification by linking with an entity in the vocabulary knowledge base The apparatus of claim 1, wherein the apparatus is configured to determine an attribute for each column in the table using one or more identification methods. The description target determination unit is:
An entity column number determining subunit configured to obtain a column number N of columns including the entity based on the determined attribute for each column;
A description object set determination subunit configured to obtain a description object set corresponding to the attribute for each column;
A description target condition probability determination subunit configured to determine a condition probability that each description target in the description target set corresponding to the attribute for each column can be described by the attribute based on the vocabulary knowledge base;
The apparatus according to claim 1, further comprising: a description object determination subunit configured to determine a description object of the table based on a condition probability of each description object. The description target determination subunit further includes:
Each description object calculates the sum of condition probabilities or average condition probability in all description object sets,
The sum of the condition probabilities of each description object or the average condition probability is arranged in order from the largest to the smallest, and the previous N description objects are configured as the description objects of the table. apparatus. The description target related attribute determination unit further includes:
Calculating the second similarity between the example of each column in the table and the value of each attribute in the attribute set;
Obtain a weighted sum of the first similarity and the second similarity, obtain a weighted similarity,
The apparatus according to claim 1, configured to determine an attribute of a column having the weighted similarity greater than a predetermined threshold as an attribute related to the description object in the table. The description target related attribute determination unit further includes:
The apparatus of claim 1, configured to discard a description object if an attribute in the attribute set of the description object is not related to any of the attributes in the table. The column attribute determination unit further includes:
When an example in a column corresponds to a plurality of attributes, the column attribute is determined based on a predetermined condition,
The apparatus of claim 1, configured to ignore the column if a predetermined condition is not met.   8. The attribute according to claim 7, wherein, as the predetermined condition, when the number of instances corresponding to an attribute among a plurality of attributes is larger than the number of instances corresponding to other attributes, the attribute is determined as an attribute of the column. apparatus.   The apparatus according to claim 1, wherein the reconstructed table is a single description target table. A table reconstruction method in a table reconstruction device ,
The table reconstruction device is
Determine the attributes for each column in the table ,
By linking the determined attribute for each column with the vocabulary knowledge base, the description target of the table is determined ,
Wherein to confirm the attribute set of each description object in lexical knowledge base, to calculate a first similarity between the attributes in the attribute set with the attribute of each column in the table, associated with the recitations in the table the attributes determined, and by using the attributes relating to the description object and the description object, comprising the step of reconfiguring table method.

Images