CN110928867B

CN110928867B - Data fusion method and device

Info

Publication number: CN110928867B
Application number: CN201811015595.2A
Authority: CN
Inventors: 杨军伟; 丁峰; 季哲
Original assignee: Hangzhou Hikvision Digital Technology Co Ltd
Current assignee: Hangzhou Hikvision Digital Technology Co Ltd
Priority date: 2018-08-31
Filing date: 2018-08-31
Publication date: 2022-09-20
Anticipated expiration: 2038-08-31
Also published as: CN110928867A

Abstract

The application relates to a data fusion method and device, and belongs to the field of data retrieval. The method comprises the following steps: acquiring a plurality of data records belonging to a first object from each database table in a database table set, wherein each data record comprises a plurality of data items, the database table set comprises a plurality of database tables, and the first object is an object corresponding to data stored in any database table in the database table set; acquiring a first data set from the plurality of data records according to a first data item set corresponding to the first object, wherein the first data item set comprises a plurality of preset data items, and a data item corresponding to each data item in the first data set is a preset data item in the first data item set; and storing the first data set and the primary key corresponding to the first object in a structured database table, wherein the primary key comprises at least one datum in the first data set for identifying the first object. The method and the device can improve the efficiency of inquiring the database.

Description

Data fusion method and device

Technical Field

The present application relates to the field of data retrieval, and in particular, to a method and an apparatus for data fusion.

Background

With the application development of big data technology, the analysis and processing of mass data are more focused on the query of data. The query of data refers to querying a database, and in the industrial field, data belonging to the same object may be stored in different databases. Thus, when the data of the object needs to be queried, different databases need to be queried.

For example, suppose a company includes a payroll database in which the correspondence between the name and the payroll of each employee of the company is stored, and a academic database in which the correspondence between the name and the academic calendar of each employee of the company is stored. In this example, the object is an employee, and when data of an employee needs to be queried, the payroll of the employee is queried from the payroll database according to the name of the employee, and the academic calendar of the employee is queried from the academic calendar database according to the name of the employee, so as to query the data of the employee.

In the process of implementing the present application, the inventors found that the above manner has at least the following defects:

at present, when data of an object is queried, the data of the object may need to be queried in different databases respectively, and the query mode is low in efficiency.

Disclosure of Invention

In order to improve the efficiency of querying a database, the embodiment of the application provides a method and a device for data fusion. The technical scheme is as follows:

in a first aspect, an embodiment of the present application provides a method for data fusion, where the method includes:

acquiring a plurality of data records belonging to a first object from each database table in a database table set, wherein each data record comprises a plurality of data items, the database table set comprises a plurality of database tables, and the first object is an object corresponding to data stored in any database table in the database table set;

acquiring a first data set from the plurality of data records according to a first data item set corresponding to the first object, wherein the first data item set comprises a plurality of preset data items, and a data item corresponding to each data item in the first data set is a preset data item in the first data item set;

and storing the first data set and a primary key corresponding to the first object in a structured database table, wherein the primary key comprises at least one data used for identifying the first object in the first data set.

Optionally, each data record includes type identification information of the first object;

before the acquiring a first data set from the plurality of data records according to the data item set corresponding to the first object, the method further includes:

and acquiring a first data item set corresponding to the first object according to the type identification information.

Optionally, the obtaining, according to the type identification information, a first data item set corresponding to the first object includes:

acquiring the object type of the first object from the corresponding relation between the type identification information and the object type according to the type identification information;

and acquiring a first data item set corresponding to the first object from the corresponding relation between the object type and the first data item set according to the object type of the first object.

Optionally, the obtaining a first data set from the plurality of data records according to the first data item set corresponding to the first object includes:

modifying data items corresponding to data in the plurality of data records into standard data items;

and selecting standard data items from the data in the plurality of data records as data of preset data items in the first data item set, and forming the selected data into a first data set.

Optionally, the modifying the data items corresponding to the data in the multiple data records into standard data items includes:

acquiring a corresponding standard data item from the corresponding relation between a second data item set and the standard data item according to a data item corresponding to target data, wherein the target data is any data in any data record in the plurality of data records, and each second data item set in the corresponding relation between the second data item set and the standard data item comprises at least one data item;

and modifying the data item corresponding to the target data into the acquired standard data item.

Optionally, before storing the first data set and the primary key corresponding to the first object in a structured database table, the method further includes:

and acquiring a second object having a relationship with the first object from the database table set through a relationship rule set according to the plurality of data records, taking the identifier of the second object as new data and adding the new data to the first data set, and taking the relationship as a data item corresponding to the new data, wherein each relationship rule in the relationship rule set defines an object relationship and a condition for meeting the object relationship.

Optionally, after obtaining the first data set from the plurality of data records according to the data item set corresponding to the first object, the method further includes:

acquiring data of each preset data item in a third data item set from the first data set, and forming a second data set; and correspondingly storing the primary key of the first object and the second data set in an index database.

Optionally, after the storing the first data set and the primary key corresponding to the first object in a structured database table, the method further includes:

receiving a query request, wherein the query request carries query data used for identifying an object to be queried and at least one target data item;

and acquiring a first data record comprising the query data from the index database table, and if the first data record comprises data belonging to each target data item in the at least one target data item, sending the data belonging to each target data item.

Optionally, the method further includes:

if the first data record does not comprise data belonging to at least one target data item, acquiring a second data record comprising a main key from a structured database according to the main key in the first data record;

and acquiring the data belonging to each target data item from the second data record, and transmitting the data belonging to each target data item.

In a second aspect, an embodiment of the present application provides an apparatus for data fusion, where the apparatus includes:

the system comprises a first acquisition module, a first storage module and a second acquisition module, wherein the first acquisition module is used for acquiring a plurality of data records belonging to a first object from each database table in a database table set, each data record comprises a plurality of data items, the database table set comprises a plurality of database tables, and the first object is an object corresponding to data stored in any one of the database tables in the database table set;

a second obtaining module, configured to obtain a first data set from the multiple data records according to a first data item set corresponding to the first object, where the first data item set includes multiple preset data items, and a data item corresponding to each data item in the first data set is a preset data item in the first data item set;

a saving module, configured to save the first data set and a primary key corresponding to the first object in a structured database table, where the primary key includes at least one piece of data in the first data set, which is used to identify the first object.

the device further comprises:

and the third acquisition module is used for acquiring a first data item set corresponding to the first object according to the type identification information.

Optionally, the third obtaining module is configured to:

Optionally, the second obtaining module is configured to:

Optionally, the apparatus further comprises:

a fourth obtaining module, configured to obtain, according to the plurality of data records, a second object having a relationship with the first object from the database table set through a relationship rule set, use an identifier of the second object as new data and add the new data to the first data set, and use the relationship as a data item corresponding to the new data, where each relationship rule in the relationship rule set defines an object relationship and a condition that satisfies the object relationship.

Optionally, the apparatus further comprises:

a fifth obtaining module, configured to obtain, from the first data set, data of each preset data item in a third data item set, and form a second data set; and correspondingly storing the primary key of the first object and the second data set in an index database.

Optionally, the apparatus further comprises:

a receiving module, configured to receive a query request, where the query request carries query data for identifying an object to be queried and at least one target data item;

a sixth obtaining module, configured to obtain a first data record including the query data from the index database table;

a sending module, configured to send data belonging to each target data item in the at least one target data item if the first data record obtained by the sixth obtaining module includes the data belonging to each target data item.

Optionally, the sixth obtaining module is further configured to, if the first data record does not include data belonging to at least one target data item, obtain, according to a primary key in the first data record, a second data record including the primary key from a structured database;

the sending module is further configured to obtain data belonging to each target data item from the second data record, and send the data belonging to each target data item.

The technical scheme provided by the embodiment of the application can have the following beneficial effects:

by acquiring a first data set, wherein the first data set comprises data of a target object stored in each database table, data items to which each data in the first data set belongs are data items in the rule file, and the first data set and the main key corresponding to the target object are correspondingly stored in a structured database table, so that the data corresponding to the target object in different database tables and the main key corresponding to the target object are spliced into a data record and stored in the structured database, when the data of the target object is subsequently queried, each data corresponding to the target object can be queried from the structured database table, and the efficiency of data query is improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.

Fig. 1 is a flowchart of a method for data fusion provided in an embodiment of the present application;

FIG. 2 is a flow chart of another method for data fusion provided by embodiments of the present application;

FIG. 3 is a flowchart of a method for querying data according to an embodiment of the present disclosure;

fig. 4 is a schematic structural diagram of a data fusion apparatus provided in an embodiment of the present application;

fig. 5 is a schematic structural diagram of a terminal according to an embodiment of the present application.

Specific embodiments of the present application have been shown by way of example in the drawings and will be described in more detail below. These drawings and written description are not intended to limit the scope of the inventive concepts in any manner, but rather to illustrate the inventive concepts to those skilled in the art by reference to specific embodiments.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

At present, each industry has a corresponding database table set, the database table set includes a plurality of database tables belonging to the industry, each database stores data records of a plurality of objects, and for any object, the data record of the object includes a plurality of data belonging to the object and a type identifier of the object. The object may be a person or an event, etc.

For example, referring to the person information table shown in table 1, one object is zhang san, the object is a person, and the data record of zhang san includes data belonging to zhang san and a type identifier "1" for identifying the person.

TABLE 1

For a data record of any object in a database table, the data in the data record corresponds to a data item, and the data item is the type of the data. For example, in the data record of zhang san shown in table 1, the data item corresponding to the data "No 1" is the identification number, and the data item corresponding to the data "23" is the age. In a set of database tables, the corresponding data items for the same type of data in different database tables may be different. For example, see data items "years" in the payroll information table shown in table 2 and data items "ages" in the staff information table shown in table 1 as data items corresponding to the same type of data; for another example, refer to the data item "job" in the payroll information table shown in table 2 and the data item "job" in the staff information table shown in table 1, which are data items corresponding to the same type of data.

TABLE 2

In the application, when data in a database table set corresponding to a certain industry needs to be merged, a technician can analyze each database table in the database table set and manufacture an industry template, where the industry template includes a correspondence between a type identifier and an object type, a correspondence between an object type and a first data item set, a correspondence between a second data item set and a standard data item, a relationship rule set, a third data item set, and a data type corresponding to each data item in the third data item set.

Because the same object may be represented by different type identifiers in database tables of different industries, when a technician analyzes each database table in a database table set corresponding to a certain service, the type identifiers corresponding to different object types can be obtained from each database table, and then the object type and the type identifier corresponding to the object type are correspondingly stored in the corresponding relationship between the type identifier and the object type.

In different database tables included in the database table set, there may be multiple data records belonging to the same object, and it may not be necessary to fuse all data in the multiple data records into the same data record when fusing data, so a first data item set corresponding to the object type of the object is defined in the industry template, the first data item set includes data items corresponding to the data items that need to be fused in one data record, and a corresponding relationship between the object type and the first data item set is stored in the industry template.

In the database table set, corresponding data items of the same type of data in different database tables may be different, in order to facilitate data fusion, a technician acquires different data items corresponding to the same type of data when analyzing each database table in the database table set, forms the acquired different data items into a second data item set, selects one data item from the second data item set as a standard data item, and correspondingly stores the second data item set and the standard data item in a corresponding relationship between the second data item set and the standard data item in an industry template. For example, in the personal information table 1, both the data item "age" and the data item "age" correspond to the same type of data, and therefore it is possible to make up the data item "age" and the data item "age" into a second data item set, select "age" as a standard data item from the second data item set, and store the second data item set and the standard data item correspondence in the correspondence relationship between the second data item set and the standard data item as shown in table 3.

TABLE 3

A second set of data items	Standard data item
		Age and years	Age(s)
Telephone, handset number	Telephone set
		Address, address	Address
Work and occupation	Occupation of the world

The relationship rule set defines at least one preset relationship, the preset relationship is an object relationship between objects, and defines conditions satisfied by the object relationship. For example, the set of relationship rules defines a sibling relationship that satisfies the condition that when the first object and the second object are in a parent-child relationship and the first object and the third object are in a parent-child relationship, then the second object and the third object are in a sibling relationship.

In the embodiment of the present application, the fused data records are stored in a structured database table, and in addition, data and a primary key belonging to a preset data item are also obtained from each data record in the structured database table, where the data belonging to the preset data item is usually data with a high frequency of being queried, and the obtained data and the primary key are correspondingly stored in an index database table. Therefore, when the data of an object needs to be inquired, the data of the object can be searched from the index database table, the searched data comprises all the data needing to be inquired, the searched data is returned, and if the searched data does not comprise all the data needing to be inquired, the data is inquired from the structured database table, so that the efficiency of searching the data can be improved.

At least one preset data item is defined in the industry template and a third data item set is formed so as to obtain the data items of the frequently queried data included in the third data item set, which belong to the target data item. The industry template further defines a data type corresponding to each preset data item in the third data item set, and the data type can be integer, long integer, character type, floating point type, boolean type and the like.

After defining an industry template of a certain industry, a device may be used to merge each database table in the database table set of the industry according to the content in the industry template, the device may be a terminal, such as a desktop computer, a notebook computer, or a tablet computer, and the device may also be a server. The detailed fusion process can be seen in the contents of any of the following examples.

Referring to fig. 1, an embodiment of the present application provides a method for data fusion, where the method includes:

step 101: the method comprises the steps of obtaining a plurality of data records belonging to a first object from each database table in a database table set, wherein each data record comprises a plurality of data items, the database table set comprises a plurality of database tables, and the first object is an object corresponding to data stored in any database table in the database table set.

Step 102: and acquiring a first data set from the plurality of data records according to a first data item set corresponding to the first object, wherein the first data item set comprises a plurality of preset data items, and the data item corresponding to each data item in the first data set is a preset data item in the first data item set.

Step 103: and storing the first data set and a primary key corresponding to the first object in a structured database table, wherein the primary key comprises at least one datum in the first data set for identifying the first object.

In the embodiment of the application, because a plurality of data records belonging to a first object are obtained from a database table set, data needing to be fused are obtained from the plurality of data records according to a first data item set of the first object to obtain a first data set, and the first data set and the main key corresponding to the first object are correspondingly stored in a structured database table, data corresponding to the same object belonging to different database tables and the main key corresponding to the object are spliced into one data record and stored in the structured database, when the data of the target object is subsequently queried, each data corresponding to the target object can be queried from the structured database table, and the efficiency of data query is improved.

Referring to fig. 2, an embodiment of the present application provides a method for fusing data, where the method includes:

step 201: the method comprises the steps of obtaining a plurality of data records belonging to a first object from each database table in a database table set, wherein each data record comprises a plurality of data items, and the first object is an object corresponding to data stored in any database table in the database table set.

Optionally, the industry template may further include a storage path for each database table of the set of database tables. Therefore, each database table can be obtained according to the storage path of each database table, and a data table set is formed.

For any one database table in the database table set, each data record in the database table corresponds to an object. The manner of acquiring the plurality of data records belonging to the first object in this step may be:

acquiring a data record from a first database table as a data record of a first object, wherein the first database table is any one database table in a database table set, and acquiring at least one piece of data for identifying the identity of the first object from the data record of the first object as identification data; acquiring data records comprising at least one identification data from a second database table, wherein the data records comprising the at least one identification data are also data records of a first object, and the second database table is other database tables except the first database table in a database table set; a plurality of data records belonging to the first object is thus obtained from the set of database tables.

For example, a first data record is obtained from the personnel information table shown in table 1, the object corresponding to the first data record is "zhang san" of the personnel, and the data for identifying the identity of "zhang san" of the personnel in the data record is the identification data of "No 1" of the personnel, so the identification data of "No 1" of the identification number is "zhang san" of the personnel. The first data record including identification data "N01", i.e., the data record for "zhang san" of a person, is obtained from the payroll information table shown in table 2. Thus obtaining two data records belonging to Zhang III of people.

Step 202: and acquiring a first data item set of the first object according to the type identification of the first object.

Each data record of the first object comprises the type identification of the first object. In this step, the type identifier of the first object is read from any data record of the first object, the object type of the first object is obtained from the corresponding relationship between the type identifier and the object type in the industry template according to the type identifier of the first object, and the first data item set of the first object is obtained from the corresponding relationship between the object type and the first data item set in the industry template according to the object type of the first object.

For example, the first data record in the personnel information table shown in table 1 includes a type identifier "1", where the type identifier "1" in the personnel information table shown in table 1 is used to indicate that the object type corresponding to the first data record is a person, and the same applies to the meaning of the type identifier "1" in the first data record shown in table 2. Therefore, when the technician analyzes the staff information table shown in table 1 or the payroll information table shown in table 2, the object type corresponding to the type identifier "1" can be analyzed to be a staff, and the type identifier "1" and the staff are correspondingly stored in the corresponding relationship between the type identifier and the object type in the service template. In this way, in this step, the type identifier "1" of the first object may be read from the first data record shown in table 1 or the first data record shown in table 2, and according to the type identifier "1", the object type corresponding to the first object is obtained from the correspondence between the type identifier and the object type in the business template as "person".

When a business template is manufactured, a first data item set corresponding to an object type of personnel is set, a plurality of data items needing to be fused in one data record are preset in the first data item set, and the object type of personnel and the first data item set are correspondingly stored in the corresponding relation between the object type of the business template and the first data item set. In this way, in this step, according to the object type "person" of the first object, the first data item set corresponding to the first object is obtained from the correspondence between the object type in the business template and the first data item set. Assume that a plurality of data items included in the first data item set corresponding to the first object are "identification number", "name", "age", "sex", "telephone", "address", "education", "occupation", "department", and "hobby", respectively.

Step 203: and acquiring a first data set from the plurality of data records according to a first data item set corresponding to the first object, wherein a data item corresponding to each data item in the first data set is a preset data item in the first data item set.

Optionally, the first data set may be obtained through two steps of 2031 and 2032 as follows:

2031: and modifying the data items corresponding to the data in the plurality of data records into standard data items.

Optionally, in implementation, it may be determined whether a second data item set including a data item corresponding to target data exists in a correspondence relationship between the second data item set and a standard data item, where the target data is any data in any of the plurality of data records, and if the second data item set including a data item corresponding to the target data does not exist, the data item corresponding to the target data item is the standard data item; and if a second data item set comprising the data items corresponding to the target data exists, acquiring the standard data items corresponding to the second data item set from the corresponding relation between the second data item set and the standard data items, and modifying the data items corresponding to the target data into the acquired standard data items.

For example, for the first data record in the personal information table 1, as for the data "No 1" in the first data record, it is determined whether or not the second data item set including the data item "identification number" corresponding to the data "No 1" exists in the correspondence relationship between the second data item set and the standard data item as shown in table 3, and the obtained determination result is non-existent, so that the standard data item corresponding to the data "No 1" is the "identification number". As another example, for data "24" in the first data record, it is determined whether or not there is a second data item set including an age of a data item corresponding to the data "24" in the correspondence relationship between the second data item set and the standard data item as shown in table 3, the obtained determination result is present, and the second data item set includes the age and the age, so that the standard data item corresponding to the second data item set is acquired as the age in the correspondence relationship between the second data item set and the standard data item as shown in table 3, and the data item corresponding to the data "24" is set as the standard data item "age". Modifying the data item corresponding to the data in the first data record in the personnel information table 1 into a standard data item according to the mode, wherein the obtained result is shown in table 4; and modifying the data item corresponding to the data in the first data record in the payroll information table 2 into the standard data item in the above manner, and obtaining the results as shown in table 5.

TABLE 4

TABLE 5

2032: and selecting the standard data items from the data in the plurality of data records as the data of the preset data items in the first data item set, and combining the selected data into the first data set.

For example, the first set of data items corresponding to the first object includes "identification number", "name", "age", "sex", "telephone", "address", "education", "occupation", "department", and "hobby". Thus, from the data in the data record shown in table 4 and the data in the data record shown in table 5, the standard data item is selected as the data of the preset data item in the first data item set, and the selected data is grouped into the first data set, so that the obtained first data set is shown in table 6.

TABLE 6

Optionally, a second object having a preset relationship with the first object may be obtained from the database table set through the relationship rule set according to the plurality of data records of the first object, an identifier of the second object is used as new data and added to the first data set, and the preset relationship is used as a data item corresponding to the new data.

Step 204: and storing the corresponding relation between the first data set and a primary key corresponding to the first object in a structured database table, wherein the primary key comprises at least one piece of data used for identifying the first object in the first data set.

For example, in the first data set shown in table 6, the data for identifying the identity of the first object may be the identification number "No 1" and the name "zhang san", so the identification number "No 1" and/or the name "zhang san" may be used as the primary key of the first object, and then the corresponding relationship between the primary key of the first object and the first data set shown in table 6 is saved in the structured database table.

Step 205: and acquiring data of each preset data item in the third data item set from the first data set, forming a second data set, and storing the corresponding relation between the primary key of the first object and the second data set in an index database.

And the third data item set comprises data items of frequently queried data. In addition, the industry template further includes a data type corresponding to each preset data item in the third data item set, and when the second data set is stored in the index database, each data item in the second data set can be stored in the index database according to the data type corresponding to the data item of each data item. For example, if a data item corresponding to a certain data item is integer, the data item is stored in the index database according to the integer.

Optionally, when storing the data record comprising the first data set and the primary key in the structured database table, the data record may be cached in the cache space first. Therefore, each cached data record can be periodically acquired from the cache space, the data belonging to each preset data item in the third data item set is taken from the data record to form a second data set, and the corresponding relation between the main key in the data record and the second data set is stored in the index database.

In the embodiment of the application, for a first object corresponding to any one data record in any one database table in a database table set, a plurality of data records belonging to the first object are acquired from the database table set, a first data item set corresponding to the first object is acquired from a predefined industry template, data needing to be fused into one data record is acquired from the plurality of data records according to the first data item set to obtain the first data set, at least one data used for identifying the first object in the first data set forms a primary key, then the corresponding relation between the primary key and the first data set is stored in a structured database table, so that the data belonging to the same object in different database tables are fused into one data record, and when the database is queried, the data belonging to the same object can be directly queried from the structured database table, the query efficiency is improved.

Referring to fig. 3, an embodiment of the present application provides a method for querying data, where the method includes:

step 301: and receiving a query request, wherein the query request carries query data used for identifying an object to be queried and at least one target data item.

A query request transmitted by the terminal may be received, the query data being one or more of the at least one data for identifying the object.

Alternatively, the user may input query data of an object to be queried in the terminal, and input a target data item to which each data item to be acquired from the database belongs.

Correspondingly, the terminal receives the query data and the target data item input by the user and sends a query request comprising the query data and the target data item.

Step 302: and acquiring a first data record comprising query data from the index database table, and if the first data record comprises data belonging to each target data item in at least one target data item, transmitting the data belonging to each target data item to the terminal.

Step 303: if the first data record does not include data belonging to part or all of the target data item, a second data record including a primary key is obtained from the structured database according to the primary key in the first data record.

Step 304: data belonging to each target data item is acquired from the second data record, and the data belonging to each target data item is transmitted.

The following are embodiments of the apparatus of the present application that may be used to perform embodiments of the method of the present application. For details which are not disclosed in the embodiments of the apparatus of the present application, reference is made to the embodiments of the method of the present application.

Referring to fig. 4, an embodiment of the present application provides an apparatus 400 for data fusion, where the apparatus 400 includes:

a first obtaining module 401, configured to obtain, from each database table in a set of database tables, a plurality of data records belonging to a first object, where each data record includes a plurality of data items, the set of database tables includes a plurality of database tables, and the first object is an object corresponding to data stored in any one of the set of database tables;

a second obtaining module 402, configured to obtain a first data set from the multiple data records according to a first data item set corresponding to the first object, where the first data item set includes multiple preset data items, and a data item corresponding to each data item in the first data set is a preset data item in the first data item set;

a saving module 403, configured to save the first data set and a primary key corresponding to the first object in a structured database table, where the primary key includes at least one data in the first data set for identifying the first object.

the apparatus 400 further comprises:

and the third obtaining module is used for obtaining a first data item set corresponding to the first object according to the type identification information.

Optionally, the third obtaining module is configured to:

Optionally, the second obtaining module 402 is configured to:

Optionally, the apparatus 400 further includes:

the device comprises a receiving module, a query module and a query module, wherein the receiving module is used for receiving a query request, and the query request carries query data used for identifying an object to be queried and at least one target data item;

In the embodiment of the application, for a first object corresponding to any one data record in any one database table in a database table set, a first obtaining module obtains multiple data records belonging to the first object from the database table set, a second obtaining module obtains data needing to be fused into one data record from the multiple data records according to a first data item set of the first object to obtain a first data set, at least one data used for identifying the first object in the first data set forms a primary key, then a storage module stores a corresponding relation between the primary key and the first data set in a structured database table to realize that the data belonging to the same object in different database tables are fused into one data record, so that when querying the database, the data belonging to the same object can be directly queried from the structured database table, the query efficiency is improved.

With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Referring to fig. 5, an embodiment of the present application provides a terminal 500, where the terminal 500 may be configured to perform the method shown in fig. 1, fig. 2, or fig. 3. The terminal 500 may be a tablet computer, a notebook computer, or a desktop computer. Terminal 500 may also be referred to by other names such as user equipment, portable terminal, laptop terminal, desktop terminal, and the like.

In general, the terminal 500 includes: a processor 501 and a memory 502.

The processor 501 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and so on. The processor 501 may be implemented in at least one hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 501 may also include a main processor and a coprocessor, where the main processor is a processor for Processing data in an awake state, and is also called a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 501 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the display screen. In some embodiments, processor 501 may also include an AI (Artificial Intelligence) processor for processing computational operations related to machine learning.

Memory 502 may include one or more computer-readable storage media, which may be non-transitory. Memory 502 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in memory 502 is used to store at least one instruction for execution by processor 501 to implement the method of data fusion provided by method embodiments herein.

In some embodiments, the terminal 500 may further optionally include: a peripheral interface 503 and at least one peripheral. The processor 501, memory 502 and peripheral interface 503 may be connected by a bus or signal lines. Various peripheral devices may be connected to the peripheral interface 503 via a bus, signal line, or circuit board. Specifically, the peripheral device includes: at least one of radio frequency circuitry 504, touch screen display 505, camera 506, audio circuitry 507, positioning components 508, and power supply 509.

The peripheral interface 503 may be used to connect at least one peripheral related to I/O (Input/Output) to the processor 501 and the memory 502. In some embodiments, the processor 501, memory 502, and peripheral interface 503 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 501, the memory 502, and the peripheral interface 503 may be implemented on separate chips or circuit boards, which is not limited by the present embodiment.

The Radio Frequency circuit 504 is used for receiving and transmitting RF (Radio Frequency) signals, also called electromagnetic signals. The radio frequency circuitry 504 communicates with communication networks and other communication devices via electromagnetic signals. The rf circuit 504 converts an electrical signal into an electromagnetic signal for transmission, or converts a received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 504 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and so forth. The radio frequency circuitry 504 may communicate with other terminals via at least one wireless communication protocol. The wireless communication protocols include, but are not limited to: the world wide web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), Wireless local area networks, and/or WiFi (Wireless Fidelity) networks. In some embodiments, the rf circuit 504 may further include NFC (Near Field Communication) related circuits, which are not limited in this application.

The display screen 505 is used to display a UI (User Interface). The UI may include graphics, text, icons, video, and any combination thereof. When the display screen 505 is a touch display screen, the display screen 505 also has the ability to capture touch signals on or over the surface of the display screen 505. The touch signal may be input to the processor 501 as a control signal for processing. At this point, the display screen 505 may also be used to provide virtual buttons and/or a virtual keyboard, also referred to as soft buttons and/or a soft keyboard. In some embodiments, the display screen 505 may be one, providing the front panel of the terminal 500; in other embodiments, the display screens 505 may be at least two, respectively disposed on different surfaces of the terminal 500 or in a folded design; in still other embodiments, the display 505 may be a flexible display disposed on a curved surface or on a folded surface of the terminal 500. Even more, the display screen 505 can be arranged in a non-rectangular irregular figure, i.e. a shaped screen. The Display screen 505 may be made of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), and other materials.

The camera assembly 506 is used to capture images or video. Optionally, camera assembly 506 includes a front camera and a rear camera. Generally, a front camera is disposed at a front panel of the terminal, and a rear camera is disposed at a rear surface of the terminal. In some embodiments, the number of the rear cameras is at least two, and each rear camera is any one of a main camera, a depth-of-field camera, a wide-angle camera and a telephoto camera, so that the main camera and the depth-of-field camera are fused to realize a background blurring function, the main camera and the wide-angle camera are fused to realize panoramic shooting and a VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, camera assembly 506 may also include a flash. The flash lamp can be a monochrome temperature flash lamp or a bicolor temperature flash lamp. The double-color-temperature flash lamp is a combination of a warm-light flash lamp and a cold-light flash lamp, and can be used for light compensation at different color temperatures.

Audio circuitry 507 may include a microphone and a speaker. The microphone is used for collecting sound waves of a user and the environment, converting the sound waves into electric signals, and inputting the electric signals to the processor 501 for processing, or inputting the electric signals to the radio frequency circuit 504 to realize voice communication. The microphones may be provided in plural numbers, respectively, at different portions of the terminal 500 for the purpose of stereo sound collection or noise reduction. The microphone may also be an array microphone or an omni-directional pick-up microphone. The speaker is used to convert electrical signals from the processor 501 or the radio frequency circuit 504 into sound waves. The loudspeaker can be a traditional film loudspeaker or a piezoelectric ceramic loudspeaker. When the speaker is a piezoelectric ceramic speaker, the speaker can be used for purposes such as converting an electric signal into a sound wave audible to a human being, or converting an electric signal into a sound wave inaudible to a human being to measure a distance. In some embodiments, audio circuitry 507 may also include a headphone jack.

The positioning component 508 is used for positioning the current geographic Location of the terminal 500 for navigation or LBS (Location Based Service). The Positioning component 508 may be a Positioning component based on the Global Positioning System (GPS) in the united states, the beidou System in china, or the galileo System in russia.

Power supply 509 is used to power the various components in terminal 500. The power source 509 may be alternating current, direct current, disposable or rechargeable. When power supply 509 includes a rechargeable battery, the rechargeable battery may be a wired rechargeable battery or a wireless rechargeable battery. The wired rechargeable battery is a battery charged through a wired line, and the wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery may also be used to support fast charge technology.

In some embodiments, terminal 500 also includes one or more sensors 510. The one or more sensors 510 include, but are not limited to: acceleration sensor 511, gyro sensor 512, pressure sensor 513, fingerprint sensor 514, optical sensor 515, and proximity sensor 516.

The acceleration sensor 511 may detect the magnitude of acceleration in three coordinate axes of the coordinate system established with the terminal 500. For example, the acceleration sensor 511 may be used to detect components of the gravitational acceleration in three coordinate axes. The processor 501 may control the touch screen 505 to display the user interface in a landscape view or a portrait view according to the gravitational acceleration signal collected by the acceleration sensor 511. The acceleration sensor 511 may also be used for acquisition of motion data of a game or a user.

The gyro sensor 512 may detect a body direction and a rotation angle of the terminal 500, and the gyro sensor 512 may cooperate with the acceleration sensor 511 to acquire a 3D motion of the user on the terminal 500. The processor 501 may implement the following functions according to the data collected by the gyro sensor 512: motion sensing (such as changing the UI according to a user's tilting operation), image stabilization at the time of photographing, game control, and inertial navigation.

The pressure sensor 513 may be disposed on a side bezel of the terminal 500 and/or an underlying layer of the touch display screen 505. When the pressure sensor 513 is disposed on the side frame of the terminal 500, a user's holding signal of the terminal 500 may be detected, and the processor 501 performs left-right hand recognition or shortcut operation according to the holding signal collected by the pressure sensor 513. When the pressure sensor 513 is disposed at the lower layer of the touch display screen 505, the processor 501 controls the operability control on the UI interface according to the pressure operation of the user on the touch display screen 505. The operability control comprises at least one of a button control, a scroll bar control, an icon control and a menu control.

The fingerprint sensor 514 is used for collecting the fingerprint of the user, and the processor 501 identifies the identity of the user according to the fingerprint collected by the fingerprint sensor 514, or the fingerprint sensor 514 identifies the identity of the user according to the collected fingerprint. Upon recognizing that the user's identity is a trusted identity, the processor 501 authorizes the user to perform relevant sensitive operations including unlocking the screen, viewing encrypted information, downloading software, paying, and changing settings, etc. The fingerprint sensor 514 may be provided on the front, back, or side of the terminal 500. When a physical button or a vendor Logo is provided on the terminal 500, the fingerprint sensor 514 may be integrated with the physical button or the vendor Logo.

The optical sensor 515 is used to collect the ambient light intensity. In one embodiment, the processor 501 may control the display brightness of the touch display screen 505 based on the ambient light intensity collected by the optical sensor 515. Specifically, when the ambient light intensity is high, the display brightness of the touch display screen 505 is increased; when the ambient light intensity is low, the display brightness of the touch display screen 505 is turned down. In another embodiment, processor 501 may also dynamically adjust the shooting parameters of camera head assembly 506 based on the ambient light intensity collected by optical sensor 515.

A proximity sensor 516, also known as a distance sensor, is typically disposed on the front panel of the terminal 500. The proximity sensor 516 is used to collect the distance between the user and the front surface of the terminal 500. In one embodiment, when the proximity sensor 516 detects that the distance between the user and the front surface of the terminal 500 gradually decreases, the processor 501 controls the touch display screen 505 to switch from the bright screen state to the dark screen state; when the proximity sensor 516 detects that the distance between the user and the front surface of the terminal 500 becomes gradually larger, the processor 501 controls the touch display screen 505 to switch from the screen-rest state to the screen-on state.

Those skilled in the art will appreciate that the configuration shown in fig. 5 is not intended to be limiting of terminal 500 and may include more or fewer components than shown, or some components may be combined, or a different arrangement of components may be used.

Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.

It will be understood that the present application is not limited to the precise arrangements that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims

1. A method of data fusion, the method comprising:

acquiring a plurality of data records belonging to a first object from each database table in a database table set, wherein each data record comprises a plurality of data items and type identification information of the first object, the database table set comprises a plurality of database tables, and the first object is an object corresponding to data stored in any database table in the database table set;

according to the object type of the first object, acquiring a first data item set corresponding to the first object from the corresponding relation between the object type and the first data item set, wherein the first data item set comprises data items corresponding to data needing to be fused in one data record;

2. The method of claim 1, wherein said obtaining a first set of data from the plurality of data records based on the first set of data items corresponding to the first object comprises:

3. The method of claim 2, wherein modifying data items corresponding to data in the plurality of data records into standard data items comprises:

4. The method of claim 1, wherein prior to saving the first data set and the primary key correspondence corresponding to the first object in a structured database table, further comprising:

and acquiring a second object having a relationship with the first object from the database table set through a relationship rule set according to the plurality of data records, taking the identifier of the second object as new data and adding the new data to the first data set, and taking the relationship as a data item corresponding to the new data, wherein each relationship rule in the relationship rule set defines an object relationship and a condition meeting the object relationship.

5. The method of any one of claims 1 to 4, wherein after obtaining the first set of data from the plurality of data records based on the set of data items corresponding to the first object, further comprising:

acquiring data of each preset data item in a third data item set from the first data set, and forming a second data set; correspondingly storing the primary key of the first object and the second data set in an index database table; the third set of data items includes data items of frequently queried data.

6. The method of claim 5, wherein after saving the first data set and the primary key correspondence corresponding to the first object in a structured database table, further comprising:

7. The method of claim 6, wherein the method further comprises:

if the first data record does not comprise data belonging to at least one target data item, acquiring a second data record comprising a main key from a structured database table according to the main key in the first data record;

8. An apparatus for data fusion, the apparatus comprising:

the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring a plurality of data records belonging to a first object from each database table in a database table set, each data record comprises a plurality of data items and type identification information of the first object, the database table set comprises a plurality of database tables, and the first object is an object corresponding to data stored in any one of the database tables in the database table set;

a third obtaining module, configured to obtain, according to the type identification information, an object type of the first object from a correspondence between the type identification information and the object type;

acquiring a first data item set corresponding to the first object from the corresponding relation between the object type and the first data item set according to the object type of the first object, wherein the first data item set comprises data items corresponding to data needing to be fused in one data record;

9. The apparatus of claim 8, wherein the second obtaining module is to:

10. The apparatus of claim 9, wherein the second obtaining module is to:

11. The apparatus of claim 8, wherein the apparatus further comprises:

12. The apparatus of any one of claims 8 to 11, further comprising:

a fifth obtaining module, configured to obtain, from the first data set, data of each preset data item in a third data item set, and form a second data set; correspondingly storing the primary key of the first object and the second data set in an index database table; the third set of data items includes data items of frequently queried data.

13. The apparatus of claim 12, wherein the apparatus further comprises:

14. The apparatus of claim 13,

the sixth obtaining module is further configured to, if the first data record does not include data belonging to at least one target data item, obtain, according to a primary key in the first data record, a second data record including the primary key from a structured database table;