CN117235079A - Data processing method, device and equipment - Google Patents
Data processing method, device and equipment Download PDFInfo
- Publication number
- CN117235079A CN117235079A CN202311524769.9A CN202311524769A CN117235079A CN 117235079 A CN117235079 A CN 117235079A CN 202311524769 A CN202311524769 A CN 202311524769A CN 117235079 A CN117235079 A CN 117235079A
- Authority
- CN
- China
- Prior art keywords
- field
- data
- data table
- target
- type
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title abstract description 15
- 238000012545 processing Methods 0.000 claims abstract description 206
- 238000012795 verification Methods 0.000 claims abstract description 81
- 238000000034 method Methods 0.000 claims abstract description 76
- 230000008569 process Effects 0.000 claims description 30
- 238000010586 diagram Methods 0.000 description 22
- 238000004590 computer program Methods 0.000 description 9
- 238000003780 insertion Methods 0.000 description 6
- 230000037431 insertion Effects 0.000 description 6
- 230000008676 import Effects 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 238000013524 data verification Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000013475 authorization Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The embodiment of the application provides a data processing method, device and equipment, and relates to the technical field of big data. The method comprises the following steps: acquiring a data processing request, wherein the data processing request comprises at least one service data and a first data table, the service data comprises at least one field, and the first data table comprises at least one target field type; for any one business data, determining the field type of each field in the business data; according to the field type of each field in at least one service data and at least one target field type, inserting at least one service data into the first data table to obtain a second data table; and performing de-duplication processing and verification processing on the second data table to obtain a target data table, storing the target data table into a database, wherein the target data table comprises at least one target field type and at least one target field corresponding to the at least one target field type, and the database is an in-memory database. The efficiency of data processing is improved.
Description
Technical Field
The embodiment of the application relates to the technical field of big data, in particular to a data processing method, a device and equipment.
Background
The enterprise can classify, summarize and store the plurality of business data according to each classification, so that the staff can quickly acquire the corresponding business data according to each classification.
In the related art, data processing may be performed by: and determining at least one field type corresponding to the plurality of service data to be processed. And determining a data table corresponding to at least one field type, and importing a plurality of business data to be processed into the corresponding data table according to at least one type to obtain at least one target data table. In the above process, a plurality of service data to be processed are manually imported into the corresponding data table, and the data processing efficiency is low due to the fact that the speed of importing a large amount of data is low due to factors such as network bandwidth, server processing capacity and the like.
Disclosure of Invention
The embodiment of the application provides a data processing method, a device and equipment, which are used for solving the problem of low data processing efficiency.
In a first aspect, an embodiment of the present application provides a data processing method, including:
acquiring a data processing request, wherein the data processing request comprises at least one service data and a first data table, the service data comprises at least one field, and the first data table comprises at least one target field type;
For any one business data, determining the field type of each field in the business data;
inserting the at least one service data into the first data table according to the field type of each field in the at least one service data and the at least one target field type to obtain a second data table;
and performing de-duplication processing and verification processing on the second data table to obtain a target data table, and storing the target data table into a database, wherein the target data table comprises the at least one target field type and at least one target field corresponding to the at least one target field type, and the database is an internal memory database.
In one possible implementation manner, according to the field type of each field in the at least one service data and the at least one target field type, the at least one service data is inserted into the first data table to obtain a second data table, which includes:
determining at least one candidate field type matched with the at least one target field type in the field types of the fields in the at least one service data;
determining at least one target field in the at least one service data according to the at least one type of the to-be-selected field;
And inserting the at least one target field into the first data table to obtain the second data table.
In a possible implementation manner, determining at least one target field in the at least one service data according to the at least one type of the to-be-selected field includes:
acquiring a preset field length and a preset field format corresponding to each type of the to-be-selected field;
determining a to-be-selected field corresponding to each to-be-selected field type, and a field length and a field format of each to-be-selected field in the at least one service data;
and for any one of the to-be-selected fields, if the field length of the to-be-selected field is matched with the preset field length and the field format of the to-be-selected field is matched with the preset field format, determining the to-be-selected field as the target field.
In one possible implementation, for any one of the target fields; inserting the target field into the first data table, comprising:
determining a target field type corresponding to the target field in the first data table;
and determining a target position in the first data table according to the type of the target field, and inserting the target field into the target position.
In one possible implementation manner, performing a deduplication process and a verification process on the second data table to obtain the target data table, where the method includes:
acquiring a plurality of data tables to be selected matched with at least one target field type in the second data table from the database;
according to the multiple data tables to be selected, performing de-duplication processing on the second data table to obtain a third data table;
and checking the third data table to obtain the target data table.
In a possible implementation manner, according to the multiple data tables to be selected, performing a deduplication process on the second data table to obtain a third data table, where the method includes:
for any one data table to be selected, determining at least one combined field which is the same as the data table to be selected in the second data table, wherein the combined field comprises a target field corresponding to each target field type in the second data table;
and deleting the at least one combined field in the second data table to obtain the third data table.
In one possible implementation manner, the verifying the third data table to obtain the target data table includes:
Acquiring at least one check rule, wherein the check rule is used for determining whether each field is valid or not;
according to the at least one verification rule, verifying each target field in the third data table to obtain a verification result;
and determining the target data table according to the verification result and the third data table.
In one possible implementation manner, determining the target data table according to the verification result and the third data table includes:
if the verification result is that the verification is passed, determining the third data table as the target data table;
and if the verification result is that the verification is not passed, determining at least one invalid field in the third data table according to the verification result, and deleting the at least one invalid field to obtain the target data table.
In one possible embodiment, the method further comprises:
obtaining a data query request, wherein the data query request comprises the at least one target field type;
according to the data query request, at least one target field corresponding to the at least one target field type is obtained from the database;
And sending the at least one target field type and at least one target field corresponding to each field type to preset equipment.
In a second aspect, an embodiment of the present application provides a data processing apparatus, the apparatus including:
the system comprises an acquisition module, a data processing module and a data processing module, wherein the acquisition module is used for acquiring a data processing request, the data processing request comprises at least one service data and a first data table, the service data comprises at least one field, and the first data table comprises at least one target field type;
the determining module is used for determining the field types of each field in the service data aiming at any one service data;
the inserting module is used for inserting the at least one service data into the first data table according to the field type of each field in the at least one service data and the at least one target field type to obtain a second data table;
the processing module is used for carrying out de-duplication processing and verification processing on the second data table to obtain a target data table, and storing the target data table into a database, wherein the target data table comprises at least one target field type and at least one target field corresponding to the at least one target field type, and the database is an internal memory database.
In a possible embodiment, the insertion module is specifically configured to:
determining at least one candidate field type matched with the at least one target field type in the field types of the fields in the at least one service data;
determining at least one target field in the at least one service data according to the at least one type of the to-be-selected field;
and inserting the at least one target field into the first data table to obtain the second data table.
In a possible embodiment, the insertion module is specifically configured to:
acquiring a preset field length and a preset field format corresponding to each type of the to-be-selected field;
determining a to-be-selected field corresponding to each to-be-selected field type, and a field length and a field format of each to-be-selected field in the at least one service data;
and for any one of the to-be-selected fields, if the field length of the to-be-selected field is matched with the preset field length and the field format of the to-be-selected field is matched with the preset field format, determining the to-be-selected field as the target field.
In a possible embodiment, the insertion module is specifically configured to:
Determining a target field type corresponding to the target field in the first data table;
and determining a target position in the first data table according to the type of the target field, and inserting the target field into the target position.
In a possible implementation manner, the processing module is specifically configured to:
acquiring a plurality of data tables to be selected matched with at least one target field type in the second data table from the database;
according to the multiple data tables to be selected, performing de-duplication processing on the second data table to obtain a third data table;
and checking the third data table to obtain the target data table.
In a possible implementation manner, the processing module is specifically configured to:
for any one data table to be selected, determining at least one combined field which is the same as the data table to be selected in the second data table, wherein the combined field comprises a target field corresponding to each target field type in the second data table;
and deleting the at least one combined field in the second data table to obtain the third data table.
In a possible implementation manner, the processing module is specifically configured to:
Acquiring at least one check rule, wherein the check rule is used for determining whether each field is valid or not;
according to the at least one verification rule, verifying each target field in the third data table to obtain a verification result;
and determining the target data table according to the verification result and the third data table.
In a possible implementation manner, the processing module is specifically configured to:
if the verification result is that the verification is passed, determining the third data table as the target data table;
and if the verification result is that the verification is not passed, determining at least one invalid field in the third data table according to the verification result, and deleting the at least one invalid field to obtain the target data table.
In one possible implementation, the apparatus further comprises a query module.
Wherein, the inquiry module is used for:
obtaining a data query request, wherein the data query request comprises the at least one target field type;
according to the data query request, at least one target field corresponding to the at least one target field type is obtained from the database;
And sending the at least one target field type and at least one target field corresponding to each field type to preset equipment.
In a third aspect, an embodiment of the present application provides a data processing apparatus, including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of the first aspects.
In a fourth aspect, embodiments of the present application provide a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method of any one of the first aspects.
In a fifth aspect, an embodiment of the present application provides a computer program product comprising a computer program which, when executed by a processor, implements the method of any of the first aspects.
The data processing method, the device and the equipment provided by the embodiment of the application acquire a data processing request, wherein the data processing request comprises at least one service data and a first data table, the service data comprises at least one field, and the first data table comprises at least one target field type. For any one service data, determining the field type of each field in the service data. And inserting at least one service data into the first data table according to the field type of each field in the at least one service data and the at least one target field type to obtain a second data table. And performing de-duplication processing and verification processing on the second data table to obtain a target data table. And storing a target data table into a database, wherein the target data table comprises at least one target field type and at least one target field corresponding to the at least one target field type, and the data is an in-memory database. In the above process, since at least one service data may be inserted into the first data table according to a field type of each field in at least one service data and at least one target field type, a plurality of service data to be processed may not be manually imported into the corresponding data table. And the method can be imported and stored in a memory database, so that the situation that the importing speed of mass data is low due to factors such as network bandwidth, server processing capacity and the like is avoided, and the data processing efficiency is improved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.
Fig. 1 is a schematic diagram of an application scenario provided in an embodiment of the present application;
FIG. 2 is a schematic flow chart of a task processing method according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a process for obtaining a data processing request according to an embodiment of the present application;
FIG. 4 is a flowchart illustrating another data processing method according to an embodiment of the present application;
FIG. 5 is a schematic diagram of a process for setting a data table according to an embodiment of the present application;
FIG. 6 is a schematic diagram illustrating a data query process according to an embodiment of the present application;
FIG. 7 is a schematic diagram of a data processing procedure according to an embodiment of the present application;
FIG. 8 is a schematic diagram of a data processing apparatus according to an embodiment of the present application;
FIG. 9 is a schematic diagram of another data processing apparatus according to an embodiment of the present application;
fig. 10 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application.
Specific embodiments of the present application have been shown by way of the above drawings and will be described in more detail below. The drawings and the written description are not intended to limit the scope of the inventive concepts in any way, but rather to illustrate the inventive concepts to those skilled in the art by reference to the specific embodiments.
Detailed Description
Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the application. Rather, they are merely examples of apparatus and methods consistent with aspects of the application as detailed in the accompanying claims.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
It should be noted that, the user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) related to the present application are information and data authorized by the user or fully authorized by each party, and the collection, use and processing of the related data need to comply with related laws and regulations and standards, and provide corresponding operation entries for the user to select authorization or rejection.
It should be noted that the data processing method and apparatus of the present application may be used in the field of big data, and may be used in any field other than big data, and the application field of the data processing method and apparatus of the present application is not limited.
In order to facilitate understanding, an application scenario to which the embodiment of the present application is applicable is described below with reference to fig. 1.
Fig. 1 is a schematic diagram of an application scenario provided in an embodiment of the present application. Referring to fig. 1, a terminal device 101 and a server 102 are included. The terminal device 101 may be a mobile phone, a tablet computer, a computer, etc. The server 102 is provided with a batch import system. The user may determine a plurality of service data by an application of the terminal device 101. And transmits the plurality of service data to the server 102. The batch import system of the server 102 imports a plurality of service data into corresponding field types according to the plurality of field types, generates a target data table, and stores the target data table into a database. The target data table includes at least one service data corresponding to at least one field type. The user may send a data query request to the server 102 through the terminal device 101 to obtain service data corresponding to at least one target field type.
In the related art, data processing may be performed by: and determining at least one field type corresponding to the plurality of service data to be processed. And determining a data table corresponding to at least one field type, and manually importing a plurality of service data to be processed into the corresponding data table according to at least one type to obtain at least one target data table. In the above process, a plurality of service data to be processed are manually imported into the corresponding data table, and the data processing efficiency is low due to the fact that the speed of importing a large amount of data is low due to factors such as network bandwidth, server processing capacity and the like.
In the embodiment of the application, a data processing request is acquired, the data processing request comprises at least one service data and a first data table, the service data comprises at least one field, and the first data table comprises at least one target field type. For any one service data, determining the field type of each field in the service data. And inserting at least one service data into the first data table according to the field type of each field in the at least one service data and the at least one target field type to obtain a second data table. And performing de-duplication processing and verification processing on the second data table to obtain a target data table. And storing a target data table into a database, wherein the target data table comprises at least one target field type and at least one target field corresponding to the at least one target field type, and the data is an in-memory database. In the above process, since at least one service data may be inserted into the first data table according to a field type of each field in at least one service data and at least one target field type, a plurality of service data to be processed may not be manually imported into the corresponding data table. And the method can be imported and stored in a memory database, so that the situation that the importing speed of mass data is low due to factors such as network bandwidth, server processing capacity and the like is avoided, and the data processing efficiency is improved.
The method according to the present application will be described below by way of specific examples. It should be noted that the following embodiments may exist alone or in combination with each other, and for the same or similar content, the description will not be repeated in different embodiments.
Fig. 2 is a flow chart of a task processing method according to an embodiment of the present application. Referring to fig. 2, the method may include:
s201, acquiring a data processing request.
The execution body of the embodiment of the application can be a data processing device or a data processing apparatus arranged in the data processing device. The data processing means may be implemented by software or by a combination of software and hardware. The data processing device may be a server.
The data processing request includes at least one service data, and a first data table. The traffic data includes at least one field. The first data table includes at least one target field type therein.
The format of at least one service data is a preset format. The preset format may be set in advance and stored in a preset storage space of the data processing apparatus. The preset format may be xls format.
If the format of the at least one service data is not the preset format, performing format conversion processing on the at least one service data to obtain the at least one service data with the preset format.
The user may determine at least one service data, and the first data table, in an application of the terminal device. The terminal equipment responds to the input selection operation of the user and generates a data processing request. And sends a data processing request to the data processing device.
Next, a procedure of acquiring a data processing request will be described with reference to fig. 3. Fig. 3 is a schematic diagram of a process for acquiring a data processing request according to an embodiment of the present application. Referring to fig. 3, the interface includes an interface 301 to an interface 302. The interfaces 301 to 302 may be pages provided by the terminal device. Referring to interface 301, a user opens an application program for data processing in a terminal device, and the terminal device displays an operation page of the application program in response to a click operation of the user. The operation page comprises a plurality of drop-down selection menus corresponding to the service data, drop-down selection menus corresponding to the field types and a text input box.
Referring to interface 302, the user determines data file 1 in the drop-down selection menu corresponding to service data 1, and determines data file 2 in the drop-down selection menu corresponding to service data 2. The data file 1 and the data file 2 comprise a plurality of business data, and the format of the data file 1 and the data file 2 is a preset format. The user determines that the field type 1 is a name in the drop-down selection menu corresponding to the field type 1, determines that the field type 2 is a gender in the drop-down selection menu corresponding to the field type 2, determines that the field type 3 is an account identifier in the drop-down selection menu corresponding to the field type 3, and clicks a determination chart. And the terminal equipment responds to the input selection operation of the user and determines a plurality of business data corresponding to the data file 1 and the data file 2. A first data table is generated based on the plurality of field types. And generating a data processing request according to the plurality of service data and the first data table. A data processing request is sent to a data processing device. The data processing request sent by the terminal device and received by the data processing device may be specifically as shown in table 1:
TABLE 1
S202, determining the field types of all fields in the service data according to any one service data.
The service data may be parsed to obtain a plurality of fields in the service data. And determining the field type of each field in the service data according to the field identification corresponding to each field.
For example, the parsing process is performed on the plurality of service data shown in table 1, and the obtaining a plurality of fields of each service data may be specifically as shown in table 2:
TABLE 2
From the multiple fields shown in table 2, it may be determined that the field types of the fields in the service data are name, gender, age, account identifier.
S203, inserting at least one service data into the first data table according to the field type of each field in at least one service data and at least one target field type to obtain a second data table.
The second data table may be obtained by inserting at least one service data into the first data table in the following manner: determining at least one to-be-selected field type matched with at least one target field type in the field types of the fields in at least one service data; determining at least one target field in at least one service data according to at least one type of the to-be-selected field; and inserting at least one target field into the first data table to obtain a second data table.
For example, as shown in the above example, the target field types are determined to be name, gender, and account identification. Among the field types of the fields in the at least one service data shown in table 2, at least one candidate field type matching with at least one target field is determined to be a name, a gender and an account identifier. According to at least one candidate field type, at least one target field is determined as a name field, a gender field and an account identification field in at least one service data shown in the above table 2. The name field, the gender field and the account identification field are inserted into the first data table, and the obtained second data table may be specifically shown in table 3:
TABLE 3 Table 3
S204, performing de-duplication processing and verification processing on the second data table to obtain a target data table, and storing the target data table into a database.
The target data table includes at least one target field type, and at least one target field corresponding to the at least one target field type. The database is a memory database.
The target data table can be obtained by performing deduplication processing and verification processing on the second data table in the following manner: acquiring a plurality of first fields corresponding to each target field type in a database; performing de-duplication processing on the second data table according to a plurality of first fields corresponding to each target field type to obtain a third data table; and performing verification processing on the third data table to obtain a target data table.
The verification process is used to verify the validity of the fields under business rules and business terms.
For example, according to the above example, the second data table is determined as shown in the above table 3. A plurality of first fields corresponding to the name, gender and account identification are obtained in a database. According to the multiple first fields corresponding to each target field type, performing deduplication processing on the second data table, where the obtained third data table may specifically be as shown in the packet 4:
TABLE 4 Table 4
And the data processing equipment performs verification processing on the third data table shown in the table 4 according to the verification rule to obtain a verification result as verification passing. Accordingly, the data processing apparatus determines the third data table shown in table 4 as the target data table. And stores the third data table shown in table 4 in a database of the data processing apparatus.
The data processing equipment is provided with a batch import system based on a micro-service architecture, and the middle platform is based on an MVC open source architecture. The memory database of the data processing equipment is used as a temporary data storage and verification database, multithreading processing is performed aiming at large data quantity import, and an open source comparison algorithm is adopted for data verification. When a large amount of data is imported, multithread parallel processing can be used, the situation that the data processing speed is low due to single-thread sequential processing is avoided, and the data processing efficiency is improved.
For example, 20 threads may be set to perform data processing, and the data processing amount of each thread is 10000 pieces of data. When the multithreading is performed, corresponding threads can be allocated to each piece of data to be processed for processing according to the current data processing amount to be processed and the current data processing amount of each thread, so that the situation that the processing amount of the same thread is large in the same period of time is avoided.
The data processing method provided by the embodiment of the application acquires the data processing request. For any one service data, determining the field type of each field in the service data. And inserting at least one service data into the first data table according to the field type of each field in the at least one service data and the at least one target field type to obtain a second data table. And performing de-duplication processing and verification processing on the second data table to obtain a target data table, and storing the target data table into a database. In the above process, since at least one service data may be inserted into the first data table according to a field type of each field in at least one service data and at least one target field type, a plurality of service data to be processed may not be manually imported into the corresponding data table. And the method can be imported and stored in a memory database, so that the situation that the importing speed of mass data is low due to factors such as network bandwidth, server processing capacity and the like is avoided, and the data processing efficiency is improved.
On the basis of any of the above embodiments, a detailed procedure of data processing will be described below with reference to fig. 4.
Fig. 4 is a flowchart of another data processing method according to an embodiment of the present application. Referring to fig. 4, the method includes:
s401, acquiring a data processing request.
The user may generate a plurality of data tables according to at least one field type before the terminal device determines the data processing request. When determining the data processing request, the data table corresponding to at least one field type can be directly acquired.
The operator program of the terminal device generates an operation page by constructing a progressive frame (Vue frame) of the user interface. The user can dynamically determine at least one field type in the operation page in a dragging mode and generate a data table. And meanwhile, the user can set the preset field length and the preset field format corresponding to each field type so as to check the service data when the service data is inserted into the data table. After the setting is finished, storing the plurality of data tables, and the preset field length and the preset field format corresponding to at least one field type in each data table into a preset storage space of the terminal equipment. When determining the data processing request, the user can directly acquire the corresponding data table, and the preset field length and the preset field format corresponding to at least one field type in the data table.
Next, a procedure of setting the data table will be described with reference to fig. 5. Fig. 5 is a schematic diagram of a process of setting a data table according to an embodiment of the present application. Referring to fig. 5, the interface 501-502 are included. The interfaces 501-502 may be pages provided by a terminal device. Referring to interface 501, a user opens an application for data processing in a terminal device. And the terminal equipment responds to the clicking operation of the user and displays an operation page of the application program. The operation page includes an operation area for determining at least one field type and a field type icon of a plurality of field types.
Referring to the interface 502, in response to a drag operation input to at least one field type, a field type icon of the at least one field type is displayed in an operation area, and a data table corresponding to the at least one field type is generated in a preset area. The field type icons of the at least one field type include a name icon, an account identification icon, a business type icon, and a contact way icon. And the user determines that the data table corresponding to at least one field type generated in the preset area has no error, and clicks the determination chart. And the terminal equipment responds to the input click operation of the user and acquires the preset field length and the preset field format corresponding to each field type. And storing the data table corresponding to at least one field type, the preset field length and the preset field format corresponding to each field type into a preset storage space.
For example, the data processing request acquired by the data processing apparatus may be specifically as shown in table 5:
TABLE 5
S402, determining the field types of all fields in the service data according to any one service data.
For example, the parsing process is performed on the plurality of service data shown in table 5, and the obtaining a plurality of fields of each service data may be specifically as shown in table 6:
TABLE 6
According to the fields of the plurality of service data shown in table 6, it can be determined that the field types of the service data 5 to 7 include name, gender, service type and account identifier.
S403, determining at least one candidate field type matched with at least one target field type in the field types of the fields in at least one service data.
For example, as shown in the above example, the field types of the service data 5 to 7 are determined to include name, gender, service type and account identifier, and the target field types are determined to include name, account identifier and service type. Thus, among the field types of the fields in the at least one business datum, it is determined that at least one candidate field type matching the at least one target field type is a name, an account identification, and a business type.
S404, determining at least one target field in at least one service data according to at least one type of the selected field.
The at least one target field may be determined in the at least one service data according to the at least one type of the candidate field by: acquiring a preset field length and a preset field format corresponding to each type of the to-be-selected field; determining a to-be-selected field corresponding to each to-be-selected field type, and a field length and a field format of each to-be-selected field in at least one service data; for any one of the to-be-selected fields, if the field length of the to-be-selected field is matched with the preset field length and the field format of the to-be-selected field is matched with the preset field format, determining the to-be-selected field as a target field.
A plurality of field types, and a preset field length and a preset field format corresponding to each field type may be set in advance. And storing the field types, the preset field length and the preset field format corresponding to each field type into a preset storage space of the data processing equipment.
For example, if the type of the field to be selected is an identification number, the preset length may be 18. If the type of the to-be-selected field is a mobile phone number, the preset length may be 11.
And if the field length of the to-be-selected field is not matched with the preset field length and/or the field format of the to-be-selected field is matched with the preset field format, determining the to-be-selected field as an invalid field. And generating prompt information and sending the prompt information to preset equipment. The hint information is used to indicate that the length or format of the field has errors.
For example, as shown in the above example, the type of field to be selected is determined to be a name, an account identifier, and a service type. The data processing device obtains, in a preset storage space, a preset field length and a preset field format corresponding to each type of the to-be-selected field, where the preset field length and the preset field format are specifically as shown in table 7:
TABLE 7
The determining the candidate field corresponding to each candidate field type and the field length and the field format of each candidate field in at least one service data shown in table 6 may be specifically as shown in table 8:
TABLE 8
According to the tables 7 and 8, if it is determined that the field length of each of the candidate fields matches the corresponding preset field length and the field format of each of the candidate fields matches the corresponding preset field format, then a plurality of the fields shown in table 8 are determined as target fields. The plurality of target fields may be specifically as shown in table 9:
TABLE 9
S405, inserting at least one target field into the first data table to obtain a second data table.
For any one of the target fields, at least one of the target fields may be inserted into the first data table by: determining a target field type corresponding to the target field in a first data table; and determining a target position in the first data table according to the type of the target field, and inserting the target field into the target position.
For example, as shown in the above example, determining the first data table may be as shown in table 10:
table 10
The data processing device determines, according to the target field types, a target position corresponding to each target field type in a first data table shown in table 10, and inserts the target field into the target position, so that the second data table may be specifically shown in table 11:
TABLE 11
S406, acquiring a plurality of data tables to be selected, which are matched with at least one target field type in the second data table, from the database.
The database may be provided in the data processing apparatus or may be provided independently in other servers. The database stores a plurality of target data tables generated by data processing in a history period.
For example, the data processing apparatus may obtain, in the database, a plurality of candidate data tables matching at least one target field type in the second data table, as shown in table 12:
Table 12
S407, performing de-duplication processing on the second data table according to the plurality of data tables to be selected to obtain a third data table.
The second data table may be subjected to deduplication according to the plurality of first fields corresponding to each target field type, to obtain a third data table: for any one data table to be selected, determining at least one combined field which is the same as the data table to be selected in a second data table, wherein the combined field comprises a target field corresponding to each target field type in the second data table; and deleting at least one combined field in the second data table to obtain a third data table.
For example, as shown in the above example, the second data table is determined to be table 11. The same combined fields as the data table to be selected are determined to comprise a name field E, a business type field agricultural insurance and an account identification field account E in the second data table. Therefore, in the second data table shown in table 11, the third data table obtained by deleting the combined field may be specifically as shown in table 13:
TABLE 13
S408, at least one check rule is obtained.
A check rule is used to determine if each field is valid.
At least one check rule may be set in advance and stored in a preset memory space of the data processing apparatus.
S409, performing verification processing on each target field in the third data table according to at least one verification rule to obtain a verification result.
For example, the data processing device acquires at least one check rule from the preset storage space as check rule 1 to check rule 10. And the data processing equipment performs verification processing on each target field in the third data table shown in the table 13 according to the verification rule 1-10 to obtain a verification result as verification passing.
S410, determining a target data table according to the verification result and the third data table, and storing the target data table into a database.
The target data table may be determined according to the check result and the third data table by: if the verification result is that the verification is passed, determining the third data table as a target data table; if the verification result is that the verification is not passed, determining at least one invalid field in the third data table according to the verification result, and deleting the at least one invalid field to obtain the target data table.
For example, as shown in the above example, the verification result is determined to be verification passing. Accordingly, the data processing apparatus determines the third data table shown in the above table 13 as the target data table.
After storing the target data table in the database, the user may obtain at least one field corresponding to the at least one field type in the database. The at least one field corresponding to the at least one field type may be obtained by: acquiring a data query request, wherein the data query request comprises at least one target field type; according to the data query request, at least one target field corresponding to the at least one target field type is obtained from the database; and sending at least one target field type and at least one target field corresponding to each field type to preset equipment.
The preset device may be a terminal device that sends a data processing request, or may be other devices set in advance by a user.
Next, a process of data query will be described with reference to fig. 6. Fig. 6 is a schematic diagram of a data query process according to an embodiment of the present application. Referring to fig. 6, the interface 601-602 is included. Interface 601-interface 602 are pages provided by the terminal device. Referring to interface 601, a user opens an application program for data query in a terminal device. And the terminal equipment responds to clicking operation of a user and opens a query page of the application program. The query page includes a drop down selection menu corresponding to the plurality of fields, and a text entry box. Please refer to the interface 602, which is configured to determine that the target field type is a name in the drop-down selection menu corresponding to the field type 1, determine that the target field type is a gender in the drop-down selection menu corresponding to the field type 2, determine that the target field type is a service type in the drop-down selection menu corresponding to the field type 3, and click the determination icon. And the terminal equipment responds to the input selection operation of the user and generates a data query request. The data query request includes a destination field type name, gender, and service type. The terminal device sends a data query request to the server.
After receiving the data query request sent by the terminal device, the server obtains at least one target field corresponding to the at least one target field type from the database according to the data query request, where the at least one target field type may be specifically as shown in table 14:
TABLE 14
The server transmits to the terminal device at least one target field type shown in table 14 and at least one target field corresponding to each field type.
The data processing method provided by the embodiment of the application acquires the data processing request. For any one service data, determining the field type of each field in the service data. At least one candidate field type matched with at least one target field is determined in the field types of the fields in the at least one service data. At least one target field is determined in the at least one service data according to the at least one type of the candidate field. And inserting at least one target field into the first data table to obtain a second data table. And acquiring a plurality of data tables to be selected matched with at least one target field type in the second data table from the database. And performing de-duplication processing on the second data table according to the plurality of data tables to be selected to obtain a third data table. At least one verification rule is obtained. And according to at least one verification rule, performing verification processing on each target field in the third data table to obtain a verification result. And determining a target data table according to the verification result and the third data table, and storing the target data table into a database. In the above process, after the third data table is determined, the duplicate removal and verification processing can be performed on the third data table, so that the accuracy of data processing is improved. And at least one service data can be inserted into the first data table according to the field type of each field in at least one service data and at least one target field type instead of manually importing a plurality of service data to be processed into the corresponding data table. And the method can be imported and stored in a memory database, so that the situation that the importing speed of mass data is low due to factors such as network bandwidth, server processing capacity and the like is avoided, and the data processing efficiency is improved.
On the basis of any of the above embodiments, the process of data processing is exemplified below with reference to fig. 7.
Fig. 7 is a schematic diagram of a data processing procedure according to an embodiment of the present application. Referring to fig. 7, a terminal device 701 and a data processing device 702 are included. The data processing device 702 may be a server, with a data processing system provided in the data processing device 702. The user may determine at least one service data, and a first data table, in an application of the terminal device 701. The terminal device 701 generates a data processing request in response to an input selection operation by the user. And sends a data processing request to the data processing device 702. The data processing request may be specifically as shown in table 15:
TABLE 15
/>
The data processing system of the data processing apparatus 702 determines that the plurality of fields corresponding to the service data shown in table 15 may be specifically as shown in table 16:
table 16
The data processing system of the data processing apparatus 702 may determine, from the fields of the plurality of service data shown in table 16, that the field types of the service data 8 to the service data 10 include name, gender, region, service type, and account identification. The data processing system of the data processing device 702 determines, among the field types of the fields in the at least one business datum, that the at least one candidate field type matching the at least one target field type is a name, a region, and a business type. The data processing system of the data processing apparatus 702 obtains, in the preset storage space, a preset field length and a preset field format corresponding to each type of the to-be-selected field. And determining the selected field corresponding to each selected field type, and the field length and field format of each selected field in at least one service data shown in table 16. The determining at least one target field in at least one service data according to the field length, the corresponding preset length, the field format and the corresponding preset field format of each candidate field may be specifically as shown in table 17:
TABLE 17
The data processing system of the data processing device 702 inserts at least one target field shown in table 17 into the first data table, and the second data table may be specifically shown in table 18:
TABLE 18
The data processing system of the data processing device 702 obtains in the database a plurality of candidate data tables matching at least one target field type in the second data table shown in table 18. And according to the multiple data tables to be selected, performing deduplication processing on the second data table shown in table 18 to obtain a third data table, which may be specifically shown in table 19:
TABLE 19
The data processing system of the data processing device 702 obtains at least one check rule in the preset storage space, and performs check processing on each target field in the third data table shown in the table 19 according to the at least one check rule, so as to obtain a check result as a check passing result. Accordingly, the data processing system of the data processing apparatus 702 determines the target data table as the third data table shown in table 19. The data processing system of the data processing device 702 stores the target data table to a database.
In the data processing process provided by the embodiment of the application, the data processing request is acquired. For any one service data, determining the field type of each field in the service data. At least one candidate field type matched with at least one target field is determined in the field types of the fields in the at least one service data. At least one target field is determined in the at least one service data according to the at least one type of the candidate field. And inserting at least one target field into the first data table to obtain a second data table. And acquiring a plurality of data tables to be selected matched with at least one target field type in the second data table from the database. And performing de-duplication processing on the second data table according to the plurality of data tables to be selected to obtain a third data table. At least one verification rule is obtained. And according to at least one verification rule, performing verification processing on each target field in the third data table to obtain a verification result. And determining a target data table according to the verification result and the third data table, and storing the target data table into a database. In the above process, after the third data table is determined, the duplicate removal and verification processing can be performed on the third data table, so that the accuracy of data processing is improved. And at least one service data can be inserted into the first data table according to the field type of each field in at least one service data and at least one target field type instead of manually importing a plurality of service data to be processed into the corresponding data table. And the method can be imported and stored in a memory database, so that the situation that the importing speed of mass data is low due to factors such as network bandwidth, server processing capacity and the like is avoided, and the data processing efficiency is improved.
Fig. 8 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application. Referring to fig. 8, the data processing apparatus 10 may include:
an obtaining module 11, configured to obtain a data processing request, where the data processing request includes at least one service data, and a first data table, where the service data includes at least one field, and the first data table includes at least one target field type;
a determining module 12, configured to determine, for any one service data, a field type of each field in the service data;
an inserting module 13, configured to insert the at least one service data into the first data table according to a field type of each field in the at least one service data and the at least one target field type, so as to obtain a second data table;
and the processing module 14 is configured to perform deduplication processing and verification processing on the second data table to obtain a target data table, and store the target data table in a database, where the target data table includes the at least one target field type and at least one target field corresponding to the at least one target field type, and the database is a memory database.
In one possible embodiment, the insertion module 13 is specifically configured to:
determining at least one candidate field type matched with the at least one target field type in the field types of the fields in the at least one service data;
determining at least one target field in the at least one service data according to the at least one type of the to-be-selected field;
and inserting the at least one target field into the first data table to obtain the second data table.
In one possible embodiment, the insertion module 13 is specifically configured to:
acquiring a preset field length and a preset field format corresponding to each type of the to-be-selected field;
determining a to-be-selected field corresponding to each to-be-selected field type, and a field length and a field format of each to-be-selected field in the at least one service data;
and for any one of the to-be-selected fields, if the field length of the to-be-selected field is matched with the preset field length and the field format of the to-be-selected field is matched with the preset field format, determining the to-be-selected field as the target field.
In one possible embodiment, the insertion module 13 is specifically configured to:
Determining a target field type corresponding to the target field in the first data table;
and determining a target position in the first data table according to the type of the target field, and inserting the target field into the target position.
In one possible implementation, the processing module 14 is specifically configured to:
acquiring a plurality of data tables to be selected matched with at least one target field type in the second data table from the database;
according to the multiple data tables to be selected, performing de-duplication processing on the second data table to obtain a third data table;
and checking the third data table to obtain the target data table.
In one possible implementation, the processing module 14 is specifically configured to:
for any one data table to be selected, determining at least one combined field which is the same as the data table to be selected in the second data table, wherein the combined field comprises a target field corresponding to each target field type in the second data table;
and deleting the at least one combined field in the second data table to obtain the third data table.
In one possible implementation, the processing module 14 is specifically configured to:
Acquiring at least one check rule, wherein the check rule is used for determining whether each field is valid or not;
according to the at least one verification rule, verifying each target field in the third data table to obtain a verification result;
and determining the target data table according to the verification result and the third data table.
In one possible implementation, the processing module 14 is specifically configured to:
if the verification result is that the verification is passed, determining the third data table as the target data table;
and if the verification result is that the verification is not passed, determining at least one invalid field in the third data table according to the verification result, and deleting the at least one invalid field to obtain the target data table.
The data processing device provided by the embodiment of the application can execute the technical scheme shown in the embodiment of the method, and the implementation principle and the beneficial effects are similar, and are not repeated here.
Fig. 9 is a schematic structural diagram of another data processing apparatus according to an embodiment of the present application. With reference to fig. 9, the data processing apparatus 10 further includes a query module 15 based on the embodiment shown in fig. 8.
Wherein, the query module 15 is configured to:
obtaining a data query request, wherein the data query request comprises the at least one target field type;
according to the data query request, at least one target field corresponding to the at least one target field type is obtained from the database;
and sending the at least one target field type and at least one target field corresponding to each field type to preset equipment.
The data processing device provided by the embodiment of the application can execute the technical scheme shown in the embodiment of the method, and the implementation principle and the beneficial effects are similar, and are not repeated here.
Fig. 10 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application. Referring to fig. 10, the data processing apparatus 20 may include: a memory 21, and a processor 22. The memory 21, the processor 22, are illustratively interconnected by a bus 23.
The memory 21 is used for storing program instructions;
the processor 22 is arranged to execute the program instructions stored in the memory for causing the data processing device 20 to perform the method as shown in the method embodiments described above.
The data processing device provided by the embodiment of the present application may execute the technical solution shown in the foregoing method embodiment, and its implementation principle and beneficial effects are similar, and will not be described herein again.
Embodiments of the present application provide a computer-readable storage medium having stored therein computer-executable instructions for implementing the above-described method when the computer-executable instructions are executed by a processor.
Embodiments of the present application may also provide a computer program product comprising a computer program which, when executed by a processor, performs the above-described method.
All or part of the steps for implementing the method embodiments described above may be performed by hardware associated with program instructions. The foregoing program may be stored in a readable memory. The program, when executed, performs steps including the method embodiments described above; and the aforementioned memory (storage medium) includes: read-only memory (ROM), random-access memory (Random Access Memory, RAM), flash memory, hard disk, solid state disk, magnetic tape, floppy disk (floppy disk), optical disk (optical disk), and any combination thereof.
Embodiments of the present application are described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processing unit of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various modifications and variations can be made to the embodiments of the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the embodiments of the present application fall within the scope of the claims and the equivalents thereof, the present application is also intended to include such modifications and variations.
In the present disclosure, the term "include" and variations thereof may refer to non-limiting inclusion; the term "or" and variations thereof may refer to "and/or". The terms "first," "second," and the like, herein, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. In the present application, "a plurality of" means two or more. "and/or", describes an association relationship of an association object, and indicates that there may be three relationships, for example, a and/or B, and may indicate: a exists alone, A and B exist together, and B exists alone. The character "/" generally indicates that the context-dependent object is an "or" relationship.
Claims (12)
1. A method of data processing, comprising:
acquiring a data processing request, wherein the data processing request comprises at least one service data and a first data table, the service data comprises at least one field, and the first data table comprises at least one target field type;
for any one business data, determining the field type of each field in the business data;
inserting the at least one service data into the first data table according to the field type of each field in the at least one service data and the at least one target field type to obtain a second data table;
And performing de-duplication processing and verification processing on the second data table to obtain a target data table, and storing the target data table into a database, wherein the target data table comprises the at least one target field type and at least one target field corresponding to the at least one target field type, and the database is an internal memory database.
2. The method of claim 1, wherein inserting the at least one service data into the first data table according to the field type of each field in the at least one service data and the at least one target field type to obtain a second data table comprises:
determining at least one candidate field type matched with the at least one target field type in the field types of the fields in the at least one service data;
determining at least one target field in the at least one service data according to the at least one type of the to-be-selected field;
and inserting the at least one target field into the first data table to obtain the second data table.
3. The method of claim 2, wherein determining at least one target field in the at least one business data according to the at least one candidate field type comprises:
Acquiring a preset field length and a preset field format corresponding to each type of the to-be-selected field;
determining a to-be-selected field corresponding to each to-be-selected field type, and a field length and a field format of each to-be-selected field in the at least one service data;
and for any one of the to-be-selected fields, if the field length of the to-be-selected field is matched with the preset field length and the field format of the to-be-selected field is matched with the preset field format, determining the to-be-selected field as the target field.
4. A method according to claim 2 or 3, characterized in that for any one of the target fields; inserting the target field into the first data table, comprising:
determining a target field type corresponding to the target field in the first data table;
and determining a target position in the first data table according to the type of the target field, and inserting the target field into the target position.
5. A method according to any one of claims 1-3, wherein performing a deduplication process and a verification process on the second data table to obtain the target data table comprises:
acquiring a plurality of data tables to be selected matched with at least one target field type in the second data table from the database;
According to the multiple data tables to be selected, performing de-duplication processing on the second data table to obtain a third data table;
and checking the third data table to obtain the target data table.
6. The method of claim 5, wherein performing a deduplication process on the second data table according to the plurality of candidate data tables to obtain a third data table, comprising:
for any one data table to be selected, determining at least one combined field which is the same as the data table to be selected in the second data table, wherein the combined field comprises a target field corresponding to each target field type in the second data table;
and deleting the at least one combined field in the second data table to obtain the third data table.
7. The method of claim 5, wherein performing a verification process on the third data table to obtain the target data table comprises:
acquiring at least one check rule, wherein the check rule is used for determining whether each field is valid or not;
according to the at least one verification rule, verifying each target field in the third data table to obtain a verification result;
And determining the target data table according to the verification result and the third data table.
8. The method of claim 7, wherein determining the target data table based on the verification result and the third data table comprises:
if the verification result is that the verification is passed, determining the third data table as the target data table;
and if the verification result is that the verification is not passed, determining at least one invalid field in the third data table according to the verification result, and deleting the at least one invalid field to obtain the target data table.
9. A method according to any one of claims 1-3, wherein the method further comprises:
obtaining a data query request, wherein the data query request comprises the at least one target field type;
according to the data query request, at least one target field corresponding to the at least one target field type is obtained from the database;
and sending the at least one target field type and at least one target field corresponding to each field type to preset equipment.
10. A data processing apparatus, the apparatus comprising:
The system comprises an acquisition module, a data processing module and a data processing module, wherein the acquisition module is used for acquiring a data processing request, the data processing request comprises at least one service data and a first data table, the service data comprises at least one field, and the first data table comprises at least one target field type;
the determining module is used for determining the field types of each field in the service data aiming at any one service data;
the inserting module is used for inserting the at least one service data into the first data table according to the field type of each field in the at least one service data and the at least one target field type to obtain a second data table;
the processing module is used for carrying out de-duplication processing and verification processing on the second data table to obtain a target data table, and storing the target data table into a database, wherein the target data table comprises at least one target field type and at least one target field corresponding to the at least one target field type, and the database is an internal memory database.
11. A data processing apparatus, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 9.
12. A non-transitory computer readable storage medium storing computer instructions, wherein the computer instructions are for causing a computer to perform the method of any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311524769.9A CN117235079A (en) | 2023-11-16 | 2023-11-16 | Data processing method, device and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311524769.9A CN117235079A (en) | 2023-11-16 | 2023-11-16 | Data processing method, device and equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117235079A true CN117235079A (en) | 2023-12-15 |
Family
ID=89086651
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311524769.9A Pending CN117235079A (en) | 2023-11-16 | 2023-11-16 | Data processing method, device and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117235079A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107870979A (en) * | 2017-09-30 | 2018-04-03 | 平安科技（深圳）有限公司 | Electronic installation, extract tables of data data method and storage medium |
| CN111143450A (en) * | 2019-12-13 | 2020-05-12 | 远光软件股份有限公司 | Method and device for importing data |
| CN112380213A (en) * | 2020-11-13 | 2021-02-19 | 苏宁云计算有限公司 | Information acquisition method and device, computer equipment and storage medium |
| CN114936214A (en) * | 2022-03-21 | 2022-08-23 | 上海东普信息科技有限公司 | Data real-time updating method, device, equipment and storage medium |
| CN115587575A (en) * | 2022-09-26 | 2023-01-10 | 中国建设银行股份有限公司 | Data table creation method, target data query method, device and equipment |
| US20230139783A1 (en) * | 2021-11-03 | 2023-05-04 | Capital One Services, Llc | Schema-adaptable data enrichment and retrieval |
-
2023
- 2023-11-16 CN CN202311524769.9A patent/CN117235079A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107870979A (en) * | 2017-09-30 | 2018-04-03 | 平安科技（深圳）有限公司 | Electronic installation, extract tables of data data method and storage medium |
| CN111143450A (en) * | 2019-12-13 | 2020-05-12 | 远光软件股份有限公司 | Method and device for importing data |
| CN112380213A (en) * | 2020-11-13 | 2021-02-19 | 苏宁云计算有限公司 | Information acquisition method and device, computer equipment and storage medium |
| US20230139783A1 (en) * | 2021-11-03 | 2023-05-04 | Capital One Services, Llc | Schema-adaptable data enrichment and retrieval |
| CN114936214A (en) * | 2022-03-21 | 2022-08-23 | 上海东普信息科技有限公司 | Data real-time updating method, device, equipment and storage medium |
| CN115587575A (en) * | 2022-09-26 | 2023-01-10 | 中国建设银行股份有限公司 | Data table creation method, target data query method, device and equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109558525B (en) | Test data set generation method, device, equipment and storage medium | |
| CN111062013B (en) | Account filtering method and device, electronic equipment and machine-readable storage medium | |
| CN106874335B (en) | Behavior data processing method and device and server | |
| CN107291745B (en) | Data index management method and device | |
| CN105574041B (en) | Data recombination method and device | |
| CN105187399A (en) | Resource processing method and device | |
| CN108154024A (en) | A kind of data retrieval method, device and electronic equipment | |
| CN111538672A (en) | Test case layered test method, computer device and computer-readable storage medium | |
| CN109542890B (en) | Data modification method, device, computer equipment and storage medium | |
| CN117235079A (en) | Data processing method, device and equipment | |
| CN109359279B (en) | Report generation method, report generation device, computer equipment and storage medium | |
| CN106446080B (en) | Data query method, query service equipment, client equipment and data system | |
| CN108121733B (en) | Data query method and device | |
| CN110851437A (en) | Storage method, device and equipment | |
| CN109242701B (en) | Method and device for submitting bill-feeding application and computer equipment | |
| CN113626509A (en) | Data access method and device, electronic equipment and readable storage medium | |
| CN108182202B (en) | Content update notification method, content update notification device, electronic equipment and storage medium | |
| CN111901299A (en) | Application authentication method and device, electronic equipment and storage medium | |
| CN117633020A (en) | Data processing method, device and equipment | |
| CN105260425A (en) | Cloud disk based file display method and apparatus | |
| CN114818645B (en) | Automatic report generation method, device, equipment and medium based on data body | |
| CN113792137B (en) | Method, system, intelligent terminal and storage medium for searching middle-stage research and development materials | |
| CN117635081B (en) | Data processing method, device, equipment and storage medium | |
| CN116909601A (en) | System updating method, device and equipment | |
| CN116226127A (en) | Service information query method, device and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |