CN111382152B

CN111382152B - Data table processing method, device and storage medium

Info

Publication number: CN111382152B
Application number: CN201811607948.8A
Authority: CN
Inventors: 邹敏
Original assignee: Hangzhou Hikvision Digital Technology Co Ltd
Current assignee: Hangzhou Hikvision Digital Technology Co Ltd
Priority date: 2018-12-27
Filing date: 2018-12-27
Publication date: 2023-10-20
Anticipated expiration: 2038-12-27
Also published as: CN111382152A

Abstract

The invention discloses a data table processing method, a data table processing device and a storage medium, and belongs to the technical field of data processing. The method comprises the following steps: generating a first data table comprising preset fields, determining second data tables corresponding to each source data table in a plurality of source data tables included in a database based on the table structure of the first data table, wherein the table structure of each second data table is identical to the table structure of the first data table, and determining a target data table based on the obtained plurality of second data tables. The embodiment of the invention avoids the need of carrying out multiple fusion processing, saves the operation time and improves the processing efficiency of the data table.

Description

Data table processing method, device and storage medium

Technical Field

The embodiment of the invention relates to the technical field of data processing, in particular to a data table processing method, a data table processing device and a storage medium.

Background

Currently, when a database is used to store data, different data tables can be used for storing based on different data types. Thus, when it is necessary to mine and analyze data in the database, in order to improve the operation efficiency, the data tables may be processed in advance, for example, a plurality of data tables may be fused into one data table.

In the related art, the implementation process of fusing multiple data tables into one data table generally includes: and carrying out fusion processing on two data tables in the plurality of data tables each time through a data operation tool, so that the plurality of data tables can be fused into one data table after multiple fusion processing.

However, in the above implementation, when the number of data tables is very large, the fusion processing needs to be performed multiple times, resulting in a long processing time and low processing efficiency of the data tables.

Disclosure of Invention

The embodiment of the invention provides a data table processing method, a data table processing device and a storage medium, which can solve the problems of longer processing time and lower processing efficiency of a data table. The technical scheme is as follows:

in a first aspect, a data table processing method is provided, the method includes:

generating a first data table comprising preset fields;

determining a second data table corresponding to each source data table in a plurality of source data tables included in a database based on the table structure of the first data table, wherein the table structure of each second data table is identical to the table structure of the first data table;

and determining a target data table based on the obtained second data tables.

Optionally, when the number of the preset fields is multiple, determining, based on the table structure of the first data table, a second data table corresponding to each source data table in the multiple source data tables included in the database includes:

when each source data table comprises a part of preset fields in the plurality of preset fields, acquiring data corresponding to the part of preset fields from each source data table, generating a second data table corresponding to each source data table and having the same table structure as the first data table, respectively determining the acquired data as data corresponding to the part of preset fields in the generated second data table, and setting other preset fields except the part of preset fields in the generated second data table to be empty;

when each source data table includes each preset field of the preset fields, acquiring data corresponding to the preset fields from each source data table, generating a second data table corresponding to each source data table and having the same table structure as the first data table, and determining the acquired data as data corresponding to the preset fields in the generated second data table.

Optionally, before determining the acquired data as the data corresponding to the plurality of preset fields in the generated second data table, the method further includes:

respectively carrying out data conversion processing on the acquired data according to a preset conversion strategy;

correspondingly, the determining the acquired data as the data corresponding to the plurality of preset fields in the generated second data table respectively includes:

and respectively determining the data after the data conversion processing as data corresponding to the plurality of preset fields in the generated second data table.

Optionally, the determining the target data table based on the obtained multiple data tables includes:

vertically combining the obtained multiple data tables to obtain a third data table;

and carrying out data fusion processing on the third data table to obtain a target data table.

Optionally, the performing data fusion processing on the third data table to obtain a target data table includes:

determining a plurality of groups of row data in the third data table, wherein the fusion main keys of each group of row data in the plurality of groups of row data are the same, and the fusion main keys are main keys corresponding to the preset fields used in the data fusion processing process;

And carrying out data fusion processing on each group of data in the plurality of groups of data based on the priority of the plurality of source data tables to obtain the target data table.

Optionally, the performing data fusion processing on each set of data in the sets of line data based on priorities of the multiple source data tables includes:

for each group of row data in the plurality of groups of row data, determining target row data corresponding to a source data table with highest priority from the each group of row data according to the priorities of the plurality of source data tables;

and reserving the target row data, and deleting other row data except the target row data in each group of row data.

In a second aspect, there is provided a data table processing apparatus, the apparatus comprising:

the generation module is used for generating a first data table comprising preset fields;

the first determining module is used for determining a second data table corresponding to each source data table in a plurality of source data tables included in the database based on the table structure of the first data table, and the table structure of each second data table is identical to the table structure of the first data table;

and the second determining module is used for determining a target data table based on the obtained plurality of second data tables.

Optionally, the first determining module is configured to:

if the number of the preset fields is multiple, when each source data table comprises a part of preset fields in the multiple preset fields, acquiring data corresponding to the part of preset fields from each source data table, generating a second data table corresponding to each source data table and having the same table structure as the first data table, respectively determining the acquired data as data corresponding to the part of preset fields in the generated second data table, and setting other preset fields except the part of preset fields in the generated second data table to be empty;

Optionally, the first determining module is configured to:

Optionally, the second determining module is configured to:

In a third aspect, a computer readable storage medium is provided, where instructions are stored, the instructions, when executed by a processor, implement the data table processing method according to the first aspect.

In a fourth aspect, there is provided a computer program product comprising instructions which, when run on a computer, cause the computer to perform the data table processing method of the first aspect described above.

The technical scheme provided by the embodiment of the invention has the beneficial effects that:

and generating a first data table comprising preset fields, and then determining a second data table corresponding to each source data table in a plurality of source data tables included in the database based on the table structure of the first data table. The determined table structure of each second data table is the same as the table structure of the first data table, in other words, the data dimension and structure between each second data table are the same, so that the target data table can be determined based on the obtained plurality of second data tables. The embodiment of the invention avoids the need of carrying out multiple fusion processing, saves the operation time and improves the processing efficiency of the data table.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flowchart illustrating a method of data table processing, according to an example embodiment;

FIG. 2 is a flow chart illustrating a method of data table processing according to another exemplary embodiment;

FIG. 3 is a schematic diagram of a data table processing apparatus according to an exemplary embodiment;

fig. 4 is a schematic diagram of a computer device 400, shown according to another exemplary embodiment.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the embodiments of the present invention will be described in further detail with reference to the accompanying drawings.

Before describing the data table processing method provided by the embodiment of the invention in detail, the application scene and the implementation environment related to the embodiment of the invention are briefly described.

Firstly, the application scenario provided by the embodiment of the invention is briefly introduced.

In the process of mining, analyzing and other operations on data in a database, the data amount in the database is generally very large, so in order to improve the operation efficiency, data fusion processing can be generally performed on the data table. However, in the current processing method, since the shuffle operation needs to be performed once every time the fusion processing of two tables is performed, and the shuffle operation is time-consuming, when the number of data tables reaches several tens, several hundreds, several thousands or even more, a long running time is required for processing all the data tables, resulting in a low processing efficiency of the data tables. Also, since each fusion process requires consuming a lot of resources such as network, memory, etc., multiple fusion processes consume a lot of resources. Therefore, the embodiment of the present invention provides a data table processing method, which can avoid the above problems, and the specific implementation of the method is shown in the following fig. 1.

Next, the implementation environment provided by the embodiment of the present invention will be briefly described.

The data table processing method provided by the embodiment of the invention can be executed by computer equipment, wherein the computer equipment can comprise a database, and further, the database can be HBase and the like. In some embodiments, the computer device may be a tablet computer, a notebook computer, a desktop computer, a portable computer, etc., which is not limited in this embodiment of the present invention.

FIG. 1 is a flowchart illustrating a data table processing method that may be performed by the computer device described above, according to an exemplary embodiment, the method may include the following implementation steps:

step 101: a first data table is generated that includes a preset field.

The preset fields can be set by a technician according to operation requirements, and further, the number of the preset fields can be one or a plurality of the preset fields. For example, in some embodiments, the predetermined fields may include name, age, owner's name, identification number, home address, phone number, etc. That is, the computer device generates a first data table according to the preset fields set by the technician based on the operation requirement, and further, when the number of the preset fields is plural, the arrangement manner of the plural preset fields in the first data table may be determined according to the preset sequence, where the preset sequence may be preset for setting.

For example, in one possible implementation, the first data table may be as shown in table 1 below:

TABLE 1

Name of name	Age of	The animal sign	Identification card number	Household address	Telephone number


						......	......	......	......	......	......

Of course, it should be noted that, here, the format of the first data table is merely taken as an example, and in another embodiment, the format of the first data table may be other forms, which is not limited in this embodiment of the present invention.

In addition, it should be noted that the number of the preset fields may be understood as the data dimension of the first data table, for example, when the number of the preset fields is 10, it may be determined that the data dimension of the first data table is also 10.

Step 102: and determining a second data table corresponding to each source data table in the plurality of source data tables included in the database based on the table structure of the first data table, wherein the table structure of each second data table is identical to the table structure of the first data table.

As described above, the number of preset fields in the first data table may be one or more, and here, based on the two cases, the implementation of determining, based on the table structure of the first data table, the second data table corresponding to each source data table in the plurality of source data tables included in the database is described in detail, which is specifically as described in (1) and (2):

(1) When the number of the preset fields is plural, each source data table may include only a part of the preset fields, or may not include any preset fields of the preset fields, or may further include all preset fields of the preset fields. Based on the above three cases, based on the table structure of the first data table, determining the second data table corresponding to each source data table in the multiple source data tables included in the database may include the following three implementation manners:

the first implementation mode: when each source data table comprises a part of preset fields in the plurality of preset fields, acquiring data corresponding to the part of preset fields from each source data table, generating a second data table corresponding to each source data table and having the same table structure as the first data table, respectively determining the acquired data as data corresponding to the part of preset fields in the generated second data table, and setting other preset fields except the part of preset fields in the generated second data table to be empty.

Continuing with the above example, assuming that a source data table i includes four preset fields of name, age, accessory phase and identification card number, the computer device obtains data corresponding to the four preset fields from the source data table i, and generates a second data table as shown in table 1. And then, respectively determining the acquired data as data of four preset fields of name, age, accessory phase and identity card number. That is, the data corresponding to the name acquired from the source data table i is determined as the data corresponding to the preset field of the name in the generated second data table, the data corresponding to the age acquired from the source data table i is determined as the data corresponding to the preset field of the age in the generated second data table, the data corresponding to the attribute acquired from the source data table i is determined as the data corresponding to the preset field of the attribute in the generated second data table, and the data corresponding to the identification card number acquired from the source data table i is determined as the data corresponding to the preset field of the identification card number in the generated second data table. In addition, other preset fields not included in the source data table i in the second data table are set to be empty, so that the second data table corresponding to the source data table i is obtained, and it is easy to understand that the table structure of the second data table is the same as that of the first data table.

According to the implementation manner, based on the table structure of the first data table, logic conversion processing is optionally performed on part of preset fields in each source data table, so that a second data table corresponding to each source data table can be obtained.

Further, before the acquired data are respectively determined to be the data corresponding to the part of the preset fields in the generated second data table, data conversion processing can be performed on the acquired data according to a preset conversion strategy, and at this time, the data after the data conversion processing are respectively determined to be the data corresponding to the plurality of preset fields in the generated second data table.

The preset conversion policy may be set by a user according to an actual requirement, and may be set by default by the computer device, which is not limited in the embodiment of the present invention.

In some application scenarios, a technician may need only some of the acquired data, for example, only the last six digits of the acquired data may be needed for the identification card number, and for example, only the first two digits of the acquired data may be needed for the home address (for example, including province and city), in which case, in order to facilitate the data analysis of the technician, the acquired data may be subjected to data conversion processing according to a preset conversion policy, for example, the last six digits of the identification card number may be extracted. Further, the obtained data may be subjected to data conversion processing by udf (User Defined Function, user-defined function). And then, respectively determining the data after the data conversion processing as data corresponding to the plurality of preset fields in the generated second data table, namely, at the moment, the data corresponding to part of preset fields in the second data table are all data obtained after the data conversion processing.

The second implementation mode: when the source data tables do not include the preset fields, generating a second data table which corresponds to the source data table and has the same table structure as the first data table, and setting the preset fields in the generated second data table to be empty.

In this implementation manner, when a certain source data table j does not include any preset field in the multiple preset fields, a second data table with the same table structure as the first data table may be generated, where the second data table includes the multiple preset fields, but each preset field in the second data table is set to be empty, that is, all data in the second data table corresponding to the source data table j is empty.

Third implementation: when each source data table comprises each preset field in the plurality of preset fields, acquiring data corresponding to the plurality of preset fields from each source data table, generating a second data table corresponding to each source data table and having the same table structure as the first data table, and respectively determining the acquired data as data corresponding to the plurality of preset fields in the generated second data table.

Continuing with the above example, the computer device obtains data corresponding to six preset fields of name, age, owner's phase, identification number, home address and telephone number from a certain source data table k, and generates a second data table as shown in table 1. And then, respectively determining the acquired data as data corresponding to six preset fields of name, age, accessory phase, identity card number, home address and telephone number. That is, the data corresponding to the name acquired from the source data table k is determined as the data corresponding to the name in the generated second data table, the data corresponding to the age acquired from the source data table k is determined as the data corresponding to the age in the generated second data table, the data corresponding to the attribute acquired from the source data table k is determined as the data corresponding to the preset field in the generated second data table, the data corresponding to the identification card number acquired from the source data table k is determined as the data corresponding to the preset field in the generated second data table, the data corresponding to the home address acquired from the source data table k is determined as the data corresponding to the preset field in the generated second data table, and the data corresponding to the telephone number acquired from the source data table k is determined as the data corresponding to the preset field in the generated second data table. It will be appreciated that the table structure of the second data table is identical to the table structure of the first data table.

According to the implementation manner, based on the table structure of the first data table, logic conversion processing is optionally performed on each preset field in each source data table, so that a second data table corresponding to each source data table can be obtained.

Further, before the acquired data are respectively determined to be the data corresponding to the plurality of preset fields in the generated second data table, the computer device may respectively perform data conversion processing on the acquired data according to a preset conversion policy, and then, determine the data after the data conversion processing to be the data corresponding to the plurality of preset fields in the generated second data table.

(2) When the number of the preset fields is one, determining, based on the table structure of the first data table, specific implementation of the second data table corresponding to each source data table in the plurality of source data tables included in the database may include two possible implementation manners as follows:

the first implementation mode: when each source data table comprises a preset field, acquiring data corresponding to the preset field from each source data table, generating a second data table corresponding to each source data table and having the same table structure as the first data table, and determining the acquired data as data corresponding to the preset field in the generated second data table.

Similarly, before the acquired data is determined to be the data corresponding to the preset field in the generated second data table, the acquired data can be subjected to data conversion processing according to a preset conversion strategy, and then the data after the data conversion processing is determined to be the data corresponding to the preset field in the generated second data table.

The second implementation mode: when the preset field is not included in each source data table, generating a second data table corresponding to each source data table and having the same table structure as the first data table, and setting the preset field in the generated second data table to be empty.

Step 103: and vertically combining the obtained multiple second data tables to obtain a third data table.

Because the table structures of the plurality of second data tables are the same, that is, the data dimensions of each second data table are the same, the plurality of second data tables may be vertically combined at a time, as shown in fig. 2. For example, continuing with the above example, assume that the plurality of second data tables obtained includes M, N, Q, S, which respectively include 10 rows, 20 rows, 30 rows and 40 rows, and the third data table obtained after the vertical merging process includes 100 rows and 6 columns.

Step 104: and carrying out data fusion processing on the third data table to obtain a target data table.

In one possible implementation manner, the implementation process of performing the data fusion processing on the third data table may include: and determining a plurality of groups of line data in the third data table, wherein the fusion main key of each group of line data in the plurality of groups of line data is the same, the fusion main key refers to a main key corresponding to the preset field used in the data fusion processing process, and carrying out data fusion processing on each group of line data in the plurality of groups of line data based on the priority of the plurality of source data tables to obtain the target data table.

The third data table may include a plurality of rows of data with the same fusion primary key, where the data fusion process may be performed on the plurality of rows of data with the same fusion primary key. The fusion main key can be set by a technician according to actual requirements, and when the number of the preset fields is multiple, the fusion main key can be a main key corresponding to one or some preset fields in the multiple preset fields. In addition, the fusion primary key is only used for data fusion processing, that is, the fusion primary key is not suitable for the primary key in the target data table obtained later.

For example, the fusion key is illustrated as a name-corresponding key. The third data table may have a plurality of rows of data with the same name, and includes a plurality of groups, for example, the names of the rows of data are all "x", at this time, the rows of data are divided into one group, and the names of the rows of data are all "y", at this time, the rows of data are divided into one group, so that a plurality of groups of row data can be obtained. The computer device may perform a data fusion process on each set of data in the plurality of sets of line data based on the priorities of the plurality of source data tables.

Further, based on the priorities of the multiple source data tables, performing data fusion processing on each group of data in the multiple groups of line data, and the specific implementation of obtaining the target data table may include: for each group of line data in the plurality of groups of line data, determining target line data corresponding to a source data table with the highest priority from the each group of line data according to the priorities of the plurality of source data tables, reserving the target line data, and deleting other line data except the target line data in the each group of line data.

The priority of the multiple source data tables may be preset by a technician according to actual requirements, or may be set by default by a computer device, which is not limited in the embodiment of the present invention.

For ease of understanding, the description will be continued with the above examples as examples. Assume that there are multiple rows of data, each of which has a name of "x", in the third data table, namely, the 9 th row, the 12 th row and the 33 th row, wherein the 9 th row of data is the data in the source data table 1, the 12 th row of data is the data in the source data table 2, and the 33 th row of data is the data in the source data table 3. If the priority of the source data table 1 is highest, it may be determined that the target line data in the set of line data is the 9 th line data, that is, the 9 th line data may be retained in the third data table, and the 12 th line and the 33 th line data may be deleted, so as to implement data fusion for the line data. And so on, according to the implementation mode, the target data table can be obtained after data fusion processing is carried out on each group of data in the plurality of groups of data.

It should be noted that, the above description only uses the example of performing the data fusion processing on each set of data in the plurality of sets of line data to obtain the target data table based on the priority of the plurality of source data tables as an example, and in another embodiment, the data fusion processing may also be performed on each set of data in the plurality of sets of line data based on the priority of the preset field in the plurality of source data tables. For example, it is assumed that there are a plurality of rows of data named "x", namely, a 9 th row, a 12 th row and a 33 th row, in the third data table, wherein the 9 th row of data is the data in the source data table 1, the 12 th row of data is the data in the source data table 2, and the 33 th row of data is the data in the source data table 3. If the priority of the age and identification card number fields in the source data table 1 is highest compared with the other two source data tables, the data corresponding to the age and identification card number in the 9 th row is reserved in the third data table, and the data corresponding to the age and identification card number in the 12 th row and the 33 th row is deleted; if the priorities of the animal sign and telephone number fields in the source data table 2 are highest compared with the other two source data tables, data corresponding to the animal sign and telephone number in the 12 th row is reserved in the third data table, and data corresponding to the animal sign and telephone number in the 9 th row and the 33 th row is deleted; if the priority of the home address field in the source data table 3 is highest compared with the other two source data tables, data corresponding to the home address in the 33 th row and data corresponding to the home address in the 9 th and 12 th rows are retained in the third data table, and further, the finally retained data may be stored in one row, thereby completing the data fusion process for the group of row data.

It should be noted that, the steps 103 to 104 are used to implement the operation of determining the target data table based on the obtained multiple data tables.

In addition, it should be noted that, when the second data table has only one preset field and each source data table does not include the unique preset field, the obtained multiple data tables may be directly combined into the target data table without performing the above-mentioned fusion operation.

In the embodiment of the invention, a first data table comprising a preset field is generated, and then a second data table corresponding to each source data table in a plurality of source data tables included in a database is determined based on a table structure of the first data table. The table structure of each determined second data table is the same as the table structure of the first data table, in other words, the data dimensions between each second data table are the same, so that the obtained plurality of second data tables can be subjected to one-time vertical merging processing, and then the third data table obtained after the vertical merging processing is subjected to data merging processing, so that the target data table is obtained. The embodiment of the invention avoids the need of carrying out multiple fusion processing, saves the operation time and improves the processing efficiency of the data table.

FIG. 3 is a schematic diagram of a data table processing device that may be implemented in software, hardware, or a combination of both, according to an example embodiment. The data table processing apparatus may include:

a generating module 310, configured to generate a first data table including a preset field;

a first determining module 320, configured to determine, based on a table structure of the first data table, a second data table corresponding to each source data table in a plurality of source data tables included in a database, where a table structure of each second data table is the same as a table structure of the first data table;

the second determining module 330 is configured to perform vertical merging processing on the obtained multiple second data tables to obtain a third data table, and perform data fusion processing on the third data table to obtain a target data table.

Optionally, the first determining module 320 is configured to:

Optionally, the second determining module 330 is configured to:

In the embodiment of the invention, a first data table comprising a preset field is generated, and then a second data table corresponding to each source data table in a plurality of source data tables included in a database is determined based on a table structure of the first data table. The determined table structure of each second data table is the same as that of the first data table, in other words, the data dimensions between each second data table are the same, so that the target data table can be determined based on the obtained plurality of second data tables. The embodiment of the invention avoids the need of carrying out multiple fusion processing, saves the operation time and improves the processing efficiency of the data table.

It should be noted that: in the data table processing apparatus provided in the above embodiment, when implementing the data table processing method, only the division of the above functional modules is used for illustration, in practical application, the above functional allocation may be performed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the data table processing device and the data table processing method provided in the foregoing embodiments belong to the same concept, and specific implementation processes thereof are detailed in the method embodiments and are not repeated herein.

Fig. 4 shows a block diagram of a computer device 400 provided by an exemplary embodiment of the invention. In general, the computer device 400 includes: a processor 401 and a memory 402.

Processor 401 may include one or more processing cores such as a 4-core processor, an 8-core processor, etc. The processor 401 may be implemented in at least one hardware form of DSP (Digital Signal Processing ), FPGA (Field-Programmable Gate Array, field programmable gate array), PLA (Programmable Logic Array ). The processor 401 may also include a main processor, which is a processor for processing data in an awake state, also called a CPU (Central Processing Unit ), and a coprocessor; a coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 401 may integrate a GPU (Graphics Processing Unit, image processor) for rendering and drawing of content required to be displayed by the display screen. In some embodiments, the processor 401 may also include an AI (Artificial Intelligence ) processor for processing computing operations related to machine learning.

Memory 402 may include one or more computer-readable storage media, which may be non-transitory. Memory 402 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in memory 402 is used to store at least one instruction for execution by processor 401 to implement a data table processing method provided by an embodiment of a method in the present application.

In some embodiments, the computer device 400 may optionally further include: a peripheral interface 403 and at least one peripheral. The processor 401, memory 402, and peripheral interface 403 may be connected by a bus or signal line. The individual peripheral devices may be connected to the peripheral device interface 403 via buses, signal lines or a circuit board. Specifically, the peripheral device includes: at least one of radio frequency circuitry 404, a touch display 405, a camera 406, audio circuitry 407, a positioning component 408, and a power supply 409.

Peripheral interface 403 may be used to connect at least one Input/Output (I/O) related peripheral to processor 401 and memory 402. In some embodiments, processor 401, memory 402, and peripheral interface 403 are integrated on the same chip or circuit board; in some other embodiments, either or both of the processor 401, memory 402, and peripheral interface 403 may be implemented on separate chips or circuit boards, which is not limited in this embodiment.

The Radio Frequency circuit 404 is configured to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuitry 404 communicates with a communication network and other communication devices via electromagnetic signals. The radio frequency circuit 404 converts an electrical signal into an electromagnetic signal for transmission, or converts a received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 404 includes: antenna systems, RF transceivers, one or more amplifiers, tuners, oscillators, digital signal processors, codec chipsets, subscriber identity module cards, and so forth. The radio frequency circuitry 404 may communicate with other computer devices via at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: the world wide web, metropolitan area networks, intranets, generation mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and/or WiFi (Wireless Fidelity ) networks. In some embodiments, the radio frequency circuitry 404 may also include NFC (Near Field Communication ) related circuitry, which is not limiting of the application.

The display screen 405 is used to display a UI (User Interface). The UI may include graphics, text, icons, video, and any combination thereof. When the display screen 405 is a touch display screen, the display screen 405 also has the ability to collect touch signals at or above the surface of the display screen 405. The touch signal may be input as a control signal to the processor 401 for processing. At this time, the display screen 405 may also be used to provide virtual buttons and/or a virtual keyboard, also referred to as soft buttons and/or a soft keyboard. In some embodiments, the display screen 405 may be one, providing a front panel of the computer device 400; in other embodiments, the display 405 may be at least two, respectively disposed on different surfaces of the computer device 400 or in a folded design; in still other embodiments, the display 405 may be a flexible display disposed on a curved surface or a folded surface of the computer device 400. Even more, the display screen 405 may be arranged in an irregular pattern that is not rectangular, i.e. a shaped screen. The display 405 may be made of LCD (Liquid Crystal Display ), OLED (Organic Light-Emitting Diode) or other materials.

The camera assembly 406 is used to capture images or video. Optionally, camera assembly 406 includes a front camera and a rear camera. Typically, the front camera is disposed on a front panel of the computer device and the rear camera is disposed on a rear surface of the computer device. In some embodiments, the at least two rear cameras are any one of a main camera, a depth camera, a wide-angle camera and a tele camera, so as to realize that the main camera and the depth camera are fused to realize a background blurring function, and the main camera and the wide-angle camera are fused to realize a panoramic shooting and Virtual Reality (VR) shooting function or other fusion shooting functions. In some embodiments, camera assembly 406 may also include a flash. The flash lamp can be a single-color temperature flash lamp or a double-color temperature flash lamp. The dual-color temperature flash lamp refers to a combination of a warm light flash lamp and a cold light flash lamp, and can be used for light compensation under different color temperatures.

The audio circuit 407 may include a microphone and a speaker. The microphone is used for collecting sound waves of users and environments, converting the sound waves into electric signals, and inputting the electric signals to the processor 401 for processing, or inputting the electric signals to the radio frequency circuit 404 for realizing voice communication. The microphone may be provided in a plurality of different locations of the computer device 400 for stereo acquisition or noise reduction purposes. The microphone may also be an array microphone or an omni-directional pickup microphone. The speaker is used to convert electrical signals from the processor 401 or the radio frequency circuit 404 into sound waves. The speaker may be a conventional thin film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, not only the electric signal can be converted into a sound wave audible to humans, but also the electric signal can be converted into a sound wave inaudible to humans for ranging and other purposes. In some embodiments, audio circuit 407 may also include a headphone jack.

The location component 408 is used to locate the current geographic location of the computer device 400 to enable navigation or LBS (Location Based Service, location-based services). The positioning component 408 may be a positioning component based on the united states GPS (Global Positioning System ), the chinese beidou system, or the russian galileo system.

The power supply 409 is used to power the various components in the computer device 400. The power supply 409 may be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When power supply 409 comprises a rechargeable battery, the rechargeable battery may be a wired rechargeable battery or a wireless rechargeable battery. The wired rechargeable battery is a battery charged through a wired line, and the wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery may also be used to support fast charge technology.

In some embodiments, computer device 400 also includes one or more sensors 410. The one or more sensors 410 include, but are not limited to: acceleration sensor 411, gyroscope sensor 412, pressure sensor 413, fingerprint sensor 414, optical sensor 415, and proximity sensor 416.

The acceleration sensor 411 may detect the magnitudes of accelerations on three coordinate axes of the coordinate system established with the computer device 400. For example, the acceleration sensor 411 may be used to detect components of gravitational acceleration on three coordinate axes. The processor 401 may control the touch display screen 405 to display a user interface in a lateral view or a longitudinal view according to the gravitational acceleration signal acquired by the acceleration sensor 411. The acceleration sensor 411 may also be used for the acquisition of motion data of a game or a user.

The gyro sensor 412 may detect the body direction and the rotation angle of the computer device 400, and the gyro sensor 412 may collect the 3D motion of the user to the computer device 400 in cooperation with the acceleration sensor 411. The processor 401 may implement the following functions according to the data collected by the gyro sensor 412: motion sensing (e.g., changing UI according to a tilting operation by a user), image stabilization at shooting, game control, and inertial navigation.

The pressure sensor 413 may be disposed at a side frame of the computer device 400 and/or at an underlying layer of the touch screen 405. When the pressure sensor 413 is disposed at a side frame of the computer device 400, a grip signal of the computer device 400 by a user may be detected, and the processor 401 performs a left-right hand recognition or a shortcut operation according to the grip signal collected by the pressure sensor 413. When the pressure sensor 413 is disposed at the lower layer of the touch display screen 405, the processor 401 controls the operability control on the UI interface according to the pressure operation of the user on the touch display screen 405. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

The fingerprint sensor 414 is used to collect a fingerprint of the user, and the processor 401 identifies the identity of the user based on the fingerprint collected by the fingerprint sensor 414, or the fingerprint sensor 414 identifies the identity of the user based on the collected fingerprint. Upon recognizing that the user's identity is a trusted identity, the user is authorized by the processor 401 to perform relevant sensitive operations including unlocking the screen, viewing encrypted information, downloading software, paying for and changing settings, etc. The fingerprint sensor 414 may be provided on the front, back or side of the computer device 400. When a physical key or vendor Logo is provided on the computer device 400, the fingerprint sensor 414 may be integrated with the physical key or vendor Logo.

The optical sensor 415 is used to collect the ambient light intensity. In one embodiment, the processor 401 may control the display brightness of the touch display screen 405 according to the ambient light intensity collected by the optical sensor 415. Specifically, when the intensity of the ambient light is high, the display brightness of the touch display screen 405 is turned up; when the ambient light intensity is low, the display brightness of the touch display screen 405 is turned down. In another embodiment, the processor 401 may also dynamically adjust the shooting parameters of the camera assembly 406 according to the ambient light intensity collected by the optical sensor 415.

A proximity sensor 416, also referred to as a distance sensor, is typically provided on the front panel of the computer device 400. The proximity sensor 416 is used to collect distance between the user and the front of the computer device 400. In one embodiment, when the proximity sensor 416 detects a gradual decrease in the distance between the user and the front of the computer device 400, the processor 401 controls the touch display 405 to switch from the bright screen state to the off screen state; when the proximity sensor 416 detects a gradual increase in the distance between the user and the front of the computer device 400, the touch display 405 is controlled by the processor 401 to switch from the off-screen state to the on-screen state.

Those skilled in the art will appreciate that the architecture shown in fig. 4 is not limiting of the computer device 400, and may include more or fewer components than shown, or may combine certain components, or employ a different arrangement of components.

The embodiment of the application also provides a non-transitory computer readable storage medium, which when executed by a processor of a computer device, enables the computer device to execute the data table processing method provided by the embodiment shown in fig. 1.

The embodiment of the application also provides a computer program product containing instructions, which when run on a computer, cause the computer to execute the data table processing method provided in the embodiment shown in fig. 1.

It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program for instructing relevant hardware, where the program may be stored in a computer readable storage medium, and the storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

The foregoing description of the preferred embodiments of the application is not intended to limit the application to the precise form disclosed, and any such modifications, equivalents, and alternatives falling within the spirit and scope of the application are intended to be included within the scope of the application.

Claims

1. A method of data table processing, the method comprising:

generating a first data table comprising preset fields;

based on the priority of the multiple source data tables, carrying out data fusion processing on each group of data in the multiple groups of line data to obtain a target data table;

wherein, for a set of row data, the set of row data includes row data belonging to a first source data table and row data of a second source data table, the preset field includes at least one first preset field and at least one second preset field;

the data fusion processing is performed on each group of data in the multiple groups of line data based on the priorities of the multiple source data tables, so as to obtain the target data table, including:

The priority of the at least one first preset field belonging to the first source data table in the group of row data is highest, the data corresponding to the at least one first preset field belonging to the first source data table is reserved in the third data table, and the data corresponding to the at least one first preset field belonging to other source data tables is deleted;

and the priority of the at least one second preset field belonging to the second source data table in the group of row data is highest, the data corresponding to the at least one second preset field belonging to the second source data table is reserved in the third data table, and the data corresponding to the at least one second preset field in other source data tables is deleted.

2. The method of claim 1, wherein when the number of the preset fields is plural, the determining, based on the table structure of the first data table, a second data table corresponding to each of a plurality of source data tables included in the database includes:

when each source data table comprises a part of preset fields in a plurality of preset fields, acquiring data corresponding to the part of preset fields from each source data table, generating a second data table corresponding to each source data table and having the same table structure as the first data table, respectively determining the acquired data as data corresponding to the part of preset fields in the generated second data table, and setting other preset fields except the part of preset fields in the generated second data table to be empty;

3. The method of claim 2, wherein before determining the acquired data as the data corresponding to the plurality of preset fields in the generated second data table, respectively, further comprises:

4. A data table processing apparatus, the apparatus comprising:

The second determining module is used for vertically combining the obtained multiple data tables to obtain a third data table;

5. The apparatus of claim 4, wherein the first determination module is to:

6. The apparatus of claim 5, wherein the first determination module is to:

7. A computer readable storage medium having instructions stored thereon, which when executed by a processor, implement the steps of the method of any of claims 1-3.