CN112597158A

CN112597158A - Data matching method and device, computer equipment and storage medium

Info

Publication number: CN112597158A
Application number: CN202011496932.1A
Authority: CN
Inventors: 余磊
Original assignee: Ping An Puhui Enterprise Management Co Ltd
Current assignee: Ping An Puhui Enterprise Management Co Ltd
Priority date: 2020-12-17
Filing date: 2020-12-17
Publication date: 2021-04-02

Abstract

The application relates to the field of artificial intelligence, and provides a data matching method, a data matching device, computer equipment and a storage medium, wherein the method comprises the following steps: acquiring to-be-detected data and a target data set imported into an Excel document; judging whether a data matching instruction is received or not; if yes, reading the VBA code, and generating a VBA task corresponding to the data to be detected and the target data set; copying the data to be detected and the target data set to a memory based on the execution logic of the VBA task; respectively calculating a first hash value of each target data contained in the target data set and an appointed hash value of the data to be detected based on a hash algorithm; generating a hash chain table corresponding to the target data set based on the first hash value and the target data; and carrying out data matching processing on the appointed hash value by using the hash chain table to generate a corresponding data matching result. The data matching processing efficiency is improved. The method and the device can also be applied to the field of block chains, and the data such as the data matching result can be stored on the block chains.

Description

Data matching method and device, computer equipment and storage medium

Technical Field

The application relates to the technical field of artificial intelligence, in particular to a data matching method, a data matching device, computer equipment and a storage medium.

Background

Data matching of different data in Excel is a more common business scenario at present. When the data matching requirement is met for the data to be detected and the data in the target data set, the existing data matching mode is that business personnel generally use functions of Excel, such as vlookup and the like to perform data matching, and therefore the business personnel need to have certain Excel function knowledge and perform function writing by themselves, and have certain skill requirements on the business personnel. In addition, the logic of the data matching method based on the function generally performs a one-by-one loop check on the data to be detected and each data in the target data set to determine whether data matching the data to be detected exists in the target data set. However, when the order of magnitude of the target data set that needs to be subjected to data matching is large, for example, more than one hundred thousand orders of magnitude, it takes a long time to perform data matching by using the function-based data matching method in Excel, and the processing efficiency is low.

Disclosure of Invention

The application mainly aims to provide a data matching method, a data matching device, computer equipment and a storage medium, and aims to solve the technical problems that a function-based data matching mode needs to take a long time to perform data matching in Excel and the processing efficiency is low.

The application provides a data matching method, which comprises the following steps:

acquiring to-be-detected data imported into an Excel document and a target data set;

judging whether a data matching instruction triggered by a user is received;

if a data matching instruction triggered by a user is received, reading a preset VBA code, and generating a VBA task corresponding to the to-be-detected data and the target data set based on the data matching instruction;

copying the data to be detected and the target data set to a memory based on the execution logic of the VBA task;

respectively calculating a first hash value of each target data contained in the target data set in the memory and calculating an appointed hash value of the to-be-detected data in the memory based on a preset hash algorithm;

generating a hash chain table corresponding to the target data set based on the first hash value and the target data;

and performing data matching processing on the appointed hash value by using the hash chain table to generate a data matching result corresponding to the data to be detected.

Optionally, the step of generating a hash chain table corresponding to the target data set based on the first hash value and the target data includes:

acquiring a first hash value of each target data contained in the target data set;

performing numerical value comparison processing on all the first hash values, and screening out a second hash value with the largest value in all the first hash values;

taking a numerical value corresponding to the second hash value as a length value;

determining placement position information corresponding to each target data according to each first hash value;

and constructing a corresponding hash chain table based on the placing position information, the target data and the length value, wherein the hash chain table comprises a plurality of arrays with the same number as the length value, each array comprises a chain table, and the target data corresponding to the placing position information matched with the arrays are stored in the chain table.

Optionally, the step of performing data matching processing on the designated hash value by using the hash chain table to generate a data matching result corresponding to the data to be detected includes:

determining the designated placement position information of the designated hash value in the hash chain table based on the corresponding relation between the preset hash value and the placement position;

according to the appointed placement position information, inquiring an appointed array corresponding to the appointed placement position information from the Hash chain table;

judging whether data are stored in the designated array or not;

if the data are stored in the appointed array, judging whether the appointed data matched with the data to be detected exist in all the data stored in the appointed array;

if the designated data exists in all the data stored in the designated array, generating a first matching result of the data matched with the data to be detected in the target data set;

and if the designated data does not exist in all the data stored in the designated array, generating a second matching result of the data which does not exist in the target data set and is matched with the data to be detected.

Optionally, the step of determining whether there is designated data matched with the data to be detected in all data stored in the designated array includes:

judging whether the number of data contained in the specified data is 1 or not;

if the number of the data contained in the designated data is not 1, comparing whether the data to be detected is the same as any data in the designated array or not based on a parallel comparison instruction;

if the data to be detected is the same as any data in the appointed array, judging that appointed data matched with the data to be detected exists in all data contained in the appointed array;

and if the data to be detected is different from any data in the appointed array, judging that the appointed data matched with the data to be detected does not exist in all the data contained in the appointed array.

Optionally, before the step of copying the data to be detected and the target data set to the memory based on the execution logic of the VBA task, the method includes:

acquiring a current memory occupation value of the memory, and acquiring a preset memory usage occupation ratio threshold;

calculating the current available memory value based on the memory occupation value;

judging whether the memory available value is larger than the memory usage ratio threshold value;

and if the memory usable value is larger than the memory use ratio threshold, generating a copy instruction for copying the data to be detected and the target data set into a memory.

Optionally, before the steps of obtaining the current memory occupancy value and obtaining the preset memory usage occupancy ratio threshold, the method includes:

calling a preset task database, wherein historical data matching tasks, the quantity value of a data set contained in each historical data matching task and the memory use ratio of each historical data matching task under normal operation are stored in the task database;

acquiring a first quantity value of data quantity contained in the target data set;

using the task database to perform query processing, and querying a second numerical value having a specific matching relationship with the first numerical value, wherein the specific matching relationship means that the absolute value of the difference between the second numerical value and the first numerical value is within a preset numerical range;

screening out appointed historical data matching tasks corresponding to the second numerical value from the task database, and searching out appointed memory usage ratio values corresponding to the appointed historical tasks respectively;

calculating the average value of all the specific memory usage ratio values;

and taking the average value as the memory usage ratio threshold value.

Optionally, after the step of performing data matching processing on the designated hash value by using the hash chain table to generate a data matching result corresponding to the data to be detected, the method includes:

acquiring current time, and judging whether the current time is in a preset time range;

if the current time is within a preset time range, acquiring preset mail login information and acquiring a designated mail address corresponding to a designated user;

logging in a corresponding mail server according to the mail login information;

and sending the data matching result to the specified mail address through the mail server.

The present application further provides a data matching apparatus, including:

the first acquisition module is used for acquiring the data to be detected and the target data set imported into the Excel document;

the first judgment module is used for judging whether a data matching instruction triggered by a user is received;

the first generation module is used for reading a preset VBA code if a data matching instruction triggered by a user is received, and generating a VBA task corresponding to the to-be-detected data and the target data set based on the data matching instruction;

the copying module is used for copying the data to be detected and the target data set to a memory based on the execution logic of the VBA task;

the first calculation module is used for calculating a first hash value of each target data contained in the target data set in the memory and calculating a designated hash value of the to-be-detected data in the memory based on a preset hash algorithm;

a second generation module, configured to generate a hash chain table corresponding to the target data set based on the first hash value and the target data;

and the matching module is used for performing data matching processing on the appointed hash value by using the hash chain table to generate a data matching result corresponding to the data to be detected.

The present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

The present application also provides a computer-readable storage medium having stored thereon a computer program which, when being executed by a processor, carries out the steps of the above-mentioned method.

The data matching method, the data matching device, the computer equipment and the storage medium have the following beneficial effects:

according to the data matching method, the data matching device, the computer equipment and the storage medium, after a user imports the data to be detected and the target data set into the Excel document and triggers a data matching instruction, the preset VBA code can be intelligently read to generate a corresponding VBA task. And then copying the data to be detected and the target data set to the memory based on the execution logic of the VBA task, and further performing data matching processing on the data to be detected and the target data set in the memory by utilizing a Hash method, so that a data matching result corresponding to the data to be detected is quickly generated, and the processing efficiency of data matching is effectively improved. According to the method and the device, the VBA tool developed based on the VBA codes is used for executing data matching comparison processing in the Excel, other software or databases and other modes with higher cost are not needed to be installed, corresponding data matching processing can be executed based on a Hash method after the related data are directly copied in the Excel, and the safety and the portability of the data are effectively guaranteed. In addition, the process of realizing data matching based on the VBA tool is simple to operate, most of common business operators can use the data matching tool conveniently, complex training is not needed, data matching tasks which can be completed by learning Excel functions and the like before can be completed quickly only by copying the data and clicking corresponding trigger buttons, user experience is effectively improved, and processing efficiency of data matching is improved.

Drawings

FIG. 1 is a schematic flow chart diagram illustrating a data matching method according to an embodiment of the present application;

FIG. 2 is a schematic structural diagram of a data matching apparatus according to an embodiment of the present application;

fig. 3 is a schematic structural diagram of a computer device according to an embodiment of the present application.

The implementation, functional features and advantages of the objectives of the present application will be further explained with reference to the accompanying drawings.

Detailed Description

It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. As used herein, the term "and/or" includes all or any element and all combinations of one or more of the associated listed items.

It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

Referring to fig. 1, a data matching method according to an embodiment of the present application includes:

s1: acquiring to-be-detected data imported into an Excel document and a target data set;

s2: judging whether a data matching instruction triggered by a user is received;

s3: if a data matching instruction triggered by a user is received, reading a preset VBA code, and generating a VBA task corresponding to the to-be-detected data and the target data set based on the data matching instruction;

s4: copying the data to be detected and the target data set to a memory based on the execution logic of the VBA task;

s5: respectively calculating a first hash value of each target data contained in the target data set in the memory and calculating an appointed hash value of the to-be-detected data in the memory based on a preset hash algorithm;

s6: generating a hash chain table corresponding to the target data set based on the first hash value and the target data;

s7: and performing data matching processing on the appointed hash value by using the hash chain table to generate a data matching result corresponding to the data to be detected.

As described in the above steps S1-S7, the main implementation of the present method embodiment is a data matching apparatus. In practical applications, the data matching device may be implemented by a virtual device, such as a software code, or by an entity device in which a relevant execution code is written or integrated, and may perform human-computer interaction with a user through a keyboard, a mouse, a remote controller, a touch panel, or a voice control device. The data matching device in this embodiment performs data matching processing on the data to be detected and the target data set based on a hash method, thereby quickly generating a data matching result corresponding to the data to be detected. Specifically, first, to-be-detected data imported into an Excel document and a target data set are obtained. The data to be detected is data that needs to be subjected to data matching processing with the target data set, and the target data set may be a data set including a large data volume, such as a hundred thousand data volume, a million data volume, and the like. And then judging whether a data matching instruction triggered by a user is received. The triggering manner of the data matching instruction is not particularly limited, and may be implemented, for example, by a user clicking or pressing a preset function button, or by a user inputting related voice data or text data, and the like. And if a data matching instruction triggered by a user is received, reading a preset VBA code, and generating a VBA task corresponding to the to-be-detected data and the target data set based on the data matching instruction. The VBA is short For Visual Basic For Application, is a macro language of the Visual Basic, is an Application program development tool established in Office, and can be used For secondary development of Excel to customize and expand functions of the Excel. After a user imports the data to be detected and the target data set into an Excel document and triggers a corresponding data matching instruction, a data matching device reads a preset VBA code to generate a specified VBA task, namely, a VBA task corresponding to the data to be detected and the target data set is generated, and the VBA code can be code data generated by a programmer in advance according to data matching requirements. The VBA task is a task for performing data matching processing on the to-be-detected data by using the target data set, and includes: and automatically executing a data matching program in the VBA task to complete data matching processing between the target data set and the data to be detected. Or based on a data matching instruction input by a user in the Excel document, executing a VBA task corresponding to the data matching instruction so as to complete data matching processing between the target data set and the data to be detected. After the VBA task is generated, the data to be detected and the target data set are copied to a memory based on an execution logic of the VBA task, so that data matching processing of the data to be detected and the target data set is performed in the memory. The data to be detected and the target data set are read into the memory, so that corresponding data matching processing is subsequently performed in the memory, the data processing speed can be increased, and the data matching efficiency is improved. And then respectively calculating a first hash value of each target data contained in the target data set in the memory based on a preset hash algorithm, and calculating an appointed hash value of the to-be-detected data in the memory. The data to be detected and the target data in the target data set adopt the same hash algorithm to obtain respective hash values so as to ensure the accuracy of the subsequent data matching. The data to be detected can be input into a preset hash algorithm to calculate and obtain a corresponding appointed hash value. The hash algorithm may be any hash function, and may include, but is not limited to, corresponding functions such as a direct addressing method, a square-based middle-of-square method, a remainder-dividing method, and a random number method, which is not limited in this embodiment. A remainder division method is preferably adopted as the hash algorithm, and for example, the hash algorithm includes: h (key) mod p, mod being a remainder operation, p being a positive integer, preferably a prime number. Similarly, the first hash value of each target data included in the target data set in the memory may be calculated by referring to the calculation manner of the specified hash value. And after the first hash value and the target data are obtained, generating a hash chain table corresponding to the target data set based on the first hash value and the target data. Wherein, the second hash value with the largest value can be screened from all the first hash values to be used as the length value; determining placement position information corresponding to each target data based on the first hash value; and then constructing a corresponding hash chain table according to the placement position information, the target data and the length value. Specifically, the hash chain table includes a plurality of arrays having the same number as the length value, each array includes a chain table, and target data corresponding to the placement position information matched with the arrays are stored in the chain table. And finally, performing data matching processing on the appointed hash value by using the hash chain table to generate a data matching result corresponding to the data to be detected. The method comprises the steps that firstly, based on the corresponding relation between a preset hash value and a placement position, the information of the appointed placement position of the appointed hash value in the hash chain table is determined; then according to the appointed placement position information, inquiring an appointed array corresponding to the appointed placement position information from the hash chain table; if specified data matched with the data to be detected exists in all the data stored in the specified array, a first matching result of the data matched with the data to be detected in the target data set is generated, and if not, a second matching result of the data matched with the data to be detected in the target data set is generated. In this embodiment, after the user imports the to-be-detected data and the target data set into an Excel document and triggers a data matching instruction, a preset VBA code is intelligently read to generate a corresponding VBA task. And then copying the data to be detected and the target data set to the memory based on the execution logic of the VBA task, and further performing data matching processing on the data to be detected and the target data set in the memory by utilizing a Hash method, so that a data matching result corresponding to the data to be detected is quickly generated, and the processing efficiency of data matching is effectively improved. According to the embodiment, the data matching comparison processing in the Excel is executed by using the VBA tool developed based on the VBA codes, the method for executing the corresponding data matching processing based on the Hash method can be directly executed after the copying of the related data in the Excel is completed without installing other software or databases and other high-cost modes, and the safety and the portability of the data are effectively guaranteed. In addition, the process of realizing data matching based on the VBA tool is simple to operate, most of common business operators can use the data matching tool conveniently, complex training is not needed, data matching tasks which can be completed by learning Excel functions and the like before can be completed quickly only by copying the data and clicking corresponding trigger buttons, user experience is effectively improved, and processing efficiency of data matching is improved.

Further, in an embodiment of the present application, the step S6 includes:

s600: acquiring a first hash value of each target data contained in the target data set;

s601: performing numerical value comparison processing on all the first hash values, and screening out a second hash value with the largest value in all the first hash values;

s602: taking a numerical value corresponding to the second hash value as a length value;

s603: determining placement position information corresponding to each target data according to each first hash value;

s604: and constructing a corresponding hash chain table based on the placing position information, the target data and the length value, wherein the hash chain table comprises a plurality of arrays with the same number as the length value, each array comprises a chain table, and the target data corresponding to the placing position information matched with the arrays are stored in the chain table.

As described in steps S600 to S604, the step of generating the hash chain table corresponding to the target data set based on the first hash value and the target data may specifically include: first, a first hash value of each target data included in the target data set is obtained. The first hash value of each target data included in the target data set may be calculated based on the hash algorithm. Specifically, each target data may be input to the hash algorithm, respectively, to calculate a one-to-one corresponding first hash value. In addition, the hash algorithm may be any hash function, and for example, the hash algorithm may include, but is not limited to, corresponding functions such as a direct addressing method, a square-based method, a remainder-dividing method, a random number method, and the like. And after the first hash values are obtained, comparing the numerical values of all the first hash values, and screening out a second hash value with the largest value in all the first hash values. And then, taking the numerical value corresponding to the second hash value as a length value. After the numerical value comparison processing is performed on all the calculated first hash values, the second hash value with the largest value among all the first hash values is used as the length value of the constructed hash chain table, so that the storage positions for storing the target data in the hash chain table can be uniformly distributed as far as possible. And then determining the placement position information corresponding to the target data according to the first hash values. The hash value and the placement position have the same corresponding relationship, that is, for any hash value a, the placement position information corresponding to the hash value a is an array a. And finally, constructing a corresponding hash chain table based on the placing position information, the target data and the length value, wherein the hash chain table comprises a plurality of arrays with the same number as the length value, each array comprises a chain table, and the chain table stores the target data corresponding to the placing position information matched with the arrays. If the length value is m, the hash chain table includes m arrays (array 0, array 1, …, array m-1), and each array is arranged in sequence from small to large according to the numerical value of the array, which can be understood as a one-dimensional array ST [ m ] composed of m pointer components, and as long as the data element with hash value i is inserted into the chain table with head pointer ST [ i ]. For example, if the target data corresponding to the hash value a is 12, the target data 12 is further filled in the linked list included in the array a. If the target data corresponding to the hash value a is 12, 28, after the target data 12 is inserted into the array a, the target data 28 is inserted into the last link point of the linked list filled with 12 of the array a, and thus, the hash linked list can be created according to the placement position information, the target data, and the length value according to the data processing method. According to the data matching method and device, the hash chain table corresponding to the target data set can be generated quickly based on the first hash value and the target data, so that the specified hash value can be subjected to data matching processing conveniently and quickly by using the hash chain table subsequently, a data matching result corresponding to the to-be-detected data can be generated quickly, and the processing efficiency of data matching is improved effectively.

Further, in an embodiment of the present application, the step S7 includes:

s700: determining the designated placement position information of the designated hash value in the hash chain table based on the corresponding relation between the preset hash value and the placement position;

s701: according to the appointed placement position information, inquiring an appointed array corresponding to the appointed placement position information from the Hash chain table;

s702: judging whether data are stored in the designated array or not;

s703: if the data are stored in the appointed array, judging whether the appointed data matched with the data to be detected exist in all the data stored in the appointed array;

s704: if the designated data exists in all the data stored in the designated array, generating a first matching result of the data matched with the data to be detected in the target data set;

s705: and if the designated data does not exist in all the data stored in the designated array, generating a second matching result of the data which does not exist in the target data set and is matched with the data to be detected.

As described in the foregoing steps S700 to S705, the step of performing data matching processing on the specified hash value by using the hash chain table to generate a data matching result corresponding to the data to be detected may specifically include: firstly, based on the corresponding relation between the preset hash value and the placement position, the information of the appointed placement position of the appointed hash value in the hash chain table is determined. Wherein, the corresponding relation between the hash value and the placing position is that the two values are the same. For example, for a specified hash value b, the specified placement position information corresponding to the specified hash value b is an array b. And after the appointed placing position information is obtained, inquiring an appointed array corresponding to the appointed placing position information from the hash chain table according to the appointed placing position information. And then judging whether the data is stored in the appointed array. And if no data is stored in the designated array, directly generating a matching result of the data which is not matched with the data to be detected in the target data set. And if the data is stored in the appointed array, further judging whether the appointed data matched with the data to be detected exists in all the data stored in the appointed array. The data comparison processing of the data to be detected and each data stored in the designated array can be executed in parallel by adopting a parallel comparison instruction, so that the data comparison processing speed is effectively improved. And if the specified data exists in all the data stored in the specified array, generating a first matching result of the data which is matched with the data to be detected and exists in the target data set. And if the specified data does not exist in all the data stored in the specified array, generating a second matching result of the data which does not exist in the target data set and is matched with the data to be detected. In the embodiment, the generated hash chain table corresponding to the target data set is created based on the hash method, that is, each target data is allocated to the corresponding array in the hash chain table for storage according to the hash value characteristics (i.e. the first hash value) of each target data in the target data set, so that the correct array can be quickly found from the hash chain table subsequently according to the hash value characteristics (i.e. the designated hash value) of the data to be detected, and then whether the target data same as the designated hash value exists is inquired from the correct array chain table, such a data matching manner effectively reduces the number of data inquiry, improves the processing efficiency of data matching, avoids the full matching of the target data set, and greatly improves the inquiry probability compared with a primary matching manner of using each target data in the target data set and performing data matching to be detected, compared with a mode of searching in sequence, the method is improved in geometric grade, and the efficiency of searching and matching by using the hash chain table is improved obviously along with the increase of the data quantity.

Further, in an embodiment of the present application, the step S703 includes:

s7030: judging whether the number of data contained in the specified data is 1 or not;

s7031: if the number of the data contained in the designated data is not 1, comparing whether the data to be detected is the same as any data in the designated array or not based on a parallel comparison instruction;

s7032: if the data to be detected is the same as any data in the appointed array, judging that appointed data matched with the data to be detected exists in all data contained in the appointed array;

s7033: and if the data to be detected is different from any data in the appointed array, judging that the appointed data matched with the data to be detected does not exist in all the data contained in the appointed array.

As described in the foregoing steps S7030 to S7033, the step of determining whether there is any specified data matching the data to be detected in all the data stored in the specified array may specifically include: first, it is determined whether the number of data included in the designated data is 1. And if the number of the data contained in the specified data is not 1, comparing whether the data to be detected is the same as any data in the specified array or not based on a parallel comparison instruction. Wherein, if the number of data included in the specified data is not 1, it indicates that the number of data included in the specified data is at least 2 or more. In addition, the parallel comparison instruction may be a single instruction stream multiple data (SIMD) instruction. And if the data to be detected is the same as any data in the appointed array, judging that the appointed data matched with the data to be detected exists in all the data contained in the appointed array. And if the data to be detected is different from any data in the specified array, judging that the specified data matched with the data to be detected does not exist in all the data contained in the specified array. In this embodiment, when the number of data included in the designated data is greater than 1, in the process of determining whether there is designated data matching the data to be detected in all the data stored in the designated array, the data matching processing between the data to be detected and each data in the designated array is performed simultaneously by using the parallel computing capability of the parallel comparison instruction, which is beneficial to further increasing the processing rate of data matching and increasing the generation speed of the generated data matching result.

Further, in an embodiment of the present application, before the step S4, the method includes:

s400: acquiring a current memory occupation value of the memory, and acquiring a preset memory usage occupation ratio threshold;

s401: calculating the current available memory value based on the memory occupation value;

s402: judging whether the memory available value is larger than the memory usage ratio threshold value;

s403: and if the memory usable value is larger than the memory use ratio threshold, generating a copy instruction for copying the data to be detected and the target data set into a memory.

As described in steps S400 to S403, before executing the process of copying the to-be-detected data and the target data set to the memory based on the execution logic of the VBA task, a calculation process of calculating a current available memory value may be performed, and then it is determined whether to directly copy the to-be-detected data and the target data set to the memory based on a calculation result. Specifically, the step of copying the to-be-detected data and the target data set to a memory based on the execution logic of the VBA task may further include: firstly, the current memory occupation value of the memory is obtained, and a preset memory usage occupation ratio threshold value is obtained. The memory occupation value can be inquired from the task manager. The memory usage percentage threshold may be generated by statistically analyzing data related to past historical tasks. And then calculating the current available memory value based on the memory occupation value. The memory occupation value is generally in a percentage form, and a current available memory value can be calculated by adopting a calculation formula of 1-memory occupation value. And then judging whether the memory available value is larger than the memory use ratio threshold value. And if the memory usable value is larger than the memory use ratio threshold, generating a copy instruction for copying the to-be-detected data and the target data set into a memory. In this embodiment, before copying the to-be-detected data and the target data set to the memory, and performing matching processing on the internally copied data by using the memory, a current available value of the memory is obtained intelligently, and only when the available value of the memory is greater than the memory usage proportion threshold, data copying processing for copying the to-be-detected data and the target data set to the memory is performed subsequently, so that the problem that data matching efficiency is low and a system is stuck even when the current available value of the memory cannot meet the requirement for performing data matching processing due to a data matching process is effectively avoided. Further, if the memory available value is smaller than the memory usage proportion threshold, a difference between the predicted value and the memory usage value is calculated, and then additional processing of memory resources corresponding to the difference is performed according to the difference. Wherein the value of the additional memory resource is at least greater than the difference. When the current available value of the memory can not meet the requirement of data matching processing, corresponding memory additional processing is executed, so that the data matching processing can be smoothly performed in the memory subjected to the additional processing of the memory resource subsequently, and the normal operation of the data matching processing can be ensured.

Further, in an embodiment of the present application, before the step S400, the method includes:

s4000: calling a preset task database, wherein historical data matching tasks, the quantity value of a data set contained in each historical data matching task and the memory use ratio of each historical data matching task under normal operation are stored in the task database;

s4001: acquiring a first quantity value of data quantity contained in the target data set;

s4002: using the task database to perform query processing, and querying a second numerical value having a specific matching relationship with the first numerical value, wherein the specific matching relationship means that the absolute value of the difference between the second numerical value and the first numerical value is within a preset numerical range;

s4003: screening out appointed historical data matching tasks corresponding to the second numerical value from the task database, and searching out appointed memory usage ratio values corresponding to the appointed historical tasks respectively;

s4004: calculating the average value of all the specific memory usage ratio values;

s4005: and taking the average value as the memory usage ratio threshold value.

As described in the foregoing steps S4000 to S4005, before the obtaining process of obtaining the preset memory usage percentage threshold is performed, a generating process of generating the memory usage percentage threshold may be further included. Specifically, the steps of obtaining the current memory occupancy value and obtaining the preset memory usage occupancy ratio threshold may further include: firstly, calling a preset task database, wherein historical data matching tasks, the quantity value of a data set contained in each historical data matching task and the memory usage ratio of each historical data matching task under normal operation are stored in the task database. Then, a first quantity value of the data quantity contained in the target data set is obtained. For example, if the target data set contains 50 ten thousand data volumes, the corresponding first quantity value is 50 ten thousand. After the first quantity value is obtained, the task database is used for query processing, and a second quantity value having a specific matching relation with the first quantity value is queried, wherein the specific matching relation means that the absolute value of the difference value between the second quantity value and the first quantity value is within a preset value range. In addition, the preset value range is not particularly limited, and may be set according to actual use requirements, for example, 2 ten thousand. And then screening out the assigned historical data matching tasks corresponding to the second numerical value from the task database, and searching out the assigned memory usage ratio value corresponding to each assigned historical task. And finally, calculating the average value of all the specified memory use ratio values, and taking the average value as the memory use ratio threshold value. According to the data matching method and device, the value of the memory use ratio threshold is intelligently determined by carrying out statistical analysis on actual data in the historical data matching task, so that whether the data to be detected and the target data set are suitable for data matching processing or not can be intelligently judged according to the memory use ratio threshold subsequently, and the execution intelligence in the process of processing the data matching task is effectively improved.

Further, in an embodiment of the present application, after the step S7, the method includes:

s710: acquiring current time, and judging whether the current time is in a preset time range;

s711: if the current time is within a preset time range, acquiring preset mail login information and acquiring a designated mail address corresponding to a designated user;

s712: logging in a corresponding mail server according to the mail login information;

s713: and sending the data matching result to the specified mail address through the mail server.

As described in the foregoing steps S710 to S713, after the step of performing data matching processing on the designated hash value by using the hash chain table to generate a data matching result corresponding to the data to be detected, the method may further include: and acquiring the current time, and judging whether the current time is in a preset time range. The preset time range refers to a time range in which the working time period is located, and specific numerical values of the preset time range are not limited and can be selected according to actual requirements, for example, the preset time range can be set to 9:00-21: 00. And if the current time is within a preset time range, acquiring preset mail login information and acquiring a specified mail address corresponding to a specified user. And then logging in a corresponding mail server according to the mail login information. And finally, sending the data matching result to the specified mail address through the mail server. In this embodiment, after the data matching result corresponding to the data to be detected is generated, the matching result is sent to the designated email address by using the email server, so that the designated user can timely know the final matching result of the data matching at this time. In addition, the data matching result is sent to the appointed mail address only when the current time is within the preset time range, so that the adverse interference on the appointed user in the rest time period can be avoided, and the intelligence of mail sending is effectively improved.

The data matching method in the embodiment of the application can also be applied to the field of block chains, for example, the data such as the hash chain table are stored on the block chain. The block chain is used for storing and managing the hash chain table, so that the safety and the non-tamper property of the hash chain table can be effectively ensured.

The block chain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism and an encryption algorithm. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.

The block chain underlying platform can comprise processing modules such as user management, basic service, intelligent contract and operation monitoring. The user management module is responsible for identity information management of all blockchain participants, and comprises public and private key generation maintenance (account management), key management, user real identity and blockchain address corresponding relation maintenance (authority management) and the like, and under the authorization condition, the user management module supervises and audits the transaction condition of certain real identities and provides rule configuration (wind control audit) of risk control; the basic service module is deployed on all block chain node equipment and used for verifying the validity of the service request, recording the service request to storage after consensus on the valid request is completed, for a new service request, the basic service firstly performs interface adaptation analysis and authentication processing (interface adaptation), then encrypts service information (consensus management) through a consensus algorithm, transmits the service information to a shared account (network communication) completely and consistently after encryption, and performs recording and storage; the intelligent contract module is responsible for registering and issuing contracts, triggering the contracts and executing the contracts, developers can define contract logics through a certain programming language, issue the contract logics to a block chain (contract registration), call keys or other event triggering and executing according to the logics of contract clauses, complete the contract logics and simultaneously provide the function of upgrading and canceling the contracts; the operation monitoring module is mainly responsible for deployment, configuration modification, contract setting, cloud adaptation in the product release process and visual output of real-time states in product operation, such as: alarm, monitoring network conditions, monitoring node equipment health status, and the like.

Referring to fig. 2, an embodiment of the present application further provides a data matching apparatus, including:

the first acquisition module 1 is used for acquiring the data to be detected and the target data set imported into the Excel document;

the first judging module 2 is used for judging whether a data matching instruction triggered by a user is received;

the first generating module 3 is configured to read a preset VBA code if a data matching instruction triggered by a user is received, and generate a VBA task corresponding to the to-be-detected data and the target data set based on the data matching instruction;

the copying module 4 is configured to copy the to-be-detected data and the target data set to a memory based on the execution logic of the VBA task;

the first calculating module 5 is configured to calculate, based on a preset hash algorithm, a first hash value of each piece of target data included in the target data set in the memory, and calculate an assigned hash value of the to-be-detected data in the memory;

a second generating module 6, configured to generate a hash chain table corresponding to the target data set based on the first hash value and the target data;

and the matching module 7 is configured to perform data matching processing on the specified hash value by using the hash chain table, and generate a data matching result corresponding to the to-be-detected data.

In this embodiment, the implementation processes of the functions and actions of the first obtaining module, the first judging module, the first generating module, the copying module, the first calculating module, the second generating module and the matching module in the data matching apparatus are specifically described in the implementation processes corresponding to steps S1 to S7 in the data matching method, and are not described herein again.

Further, in an embodiment of the application, the second generating module includes:

a first obtaining unit, configured to obtain a first hash value of each piece of target data included in the target data set;

the comparison unit is used for carrying out numerical value comparison processing on all the first hash values and screening out a second hash value with the largest value in all the first hash values;

a first determining unit, configured to use a numerical value corresponding to the second hash value as a length value;

a second determining unit, configured to determine, according to each of the first hash values, placement position information corresponding to each of the target data;

and the construction unit is used for constructing a corresponding hash chain table based on the placement position information, the target data and the length value, wherein the hash chain table comprises a plurality of arrays with the same number as the length value, each array comprises a chain table, and the chain table stores the target data corresponding to the placement position information matched with the arrays.

In this embodiment, the implementation processes of the functions and actions of the first obtaining unit, the comparing unit, the first determining unit, the second determining unit and the constructing unit in the data matching apparatus are specifically described in the implementation processes corresponding to steps S600 to S604 in the data matching method, and are not described herein again.

Further, in an embodiment of the present application, the matching module includes:

a third determining unit, configured to determine, based on a preset correspondence between hash values and placement positions, information of the designated placement positions of the designated hash values in the hash chain table;

the query unit is used for querying a specified array corresponding to the specified placement position information from the hash chain table according to the specified placement position information;

the first judgment unit is used for judging whether the specified array stores data or not;

the second judging unit is used for judging whether specified data matched with the data to be detected exists in all the data stored in the specified array or not if the data is stored in the specified array;

the first generating unit is used for generating a first matching result of data matched with the data to be detected in the target data set if the designated data exists in all the data stored in the designated array;

and the second generating unit is used for generating a second matching result of the data which is not matched with the data to be detected in the target data set if the designated data does not exist in all the data stored in the designated array.

In this embodiment, the implementation processes of the functions and actions of the third determining unit, the querying unit, the first determining unit, the second determining unit, the first generating unit and the second generating unit in the data matching apparatus are specifically described in the implementation processes corresponding to steps S700 to S705 in the data matching method, and are not described herein again.

Further, in an embodiment of the application, the second determining unit includes:

a judgment subunit, configured to judge whether the number of data included in the designated data is 1;

the comparison subunit is configured to compare, if the number of data included in the designated data is not 1, whether the data to be detected is the same as any one of the data in the designated array based on a parallel comparison instruction;

the first judging subunit is configured to judge that designated data matched with the data to be detected exists in all data included in the designated array if the data to be detected is the same as any one of the data in the designated array;

and the second judging subunit is configured to judge that there is no designated data matched with the data to be detected in all data included in the designated array if the data to be detected is different from any data in the designated array.

In this embodiment, the implementation processes of the functions and actions of the judging subunit, the comparing subunit, the first judging subunit and the second judging subunit in the data matching apparatus are specifically described in the implementation processes corresponding to steps S7030 to S7033 in the data matching method, and are not described herein again.

Further, in an embodiment of the present application, the data matching apparatus includes:

the second acquisition module is used for acquiring the current memory occupation value of the memory and acquiring a preset memory usage occupation ratio threshold;

the second calculation module is used for calculating the current available memory value based on the memory occupation value;

the second judgment module is used for judging whether the available value of the memory is larger than the memory usage ratio threshold value;

and a third generating module, configured to generate a copy instruction for copying the to-be-detected data and the target data set to a memory if the memory availability value is greater than the memory usage percentage threshold.

In this embodiment, the implementation processes of the functions and actions of the second obtaining module, the second calculating module, the second judging module and the third generating module in the data matching apparatus are specifically described in the implementation processes corresponding to steps S400 to S403 in the data matching method, and are not described herein again.

the system comprises a calling module, a task database and a task execution module, wherein the task database stores historical data matching tasks, the quantity value of a data set contained in each historical data matching task and the memory usage ratio of each historical data matching task under normal operation;

the third acquisition module is used for acquiring a first quantity value of the data quantity contained in the target data set;

the query module is used for performing query processing by using the task database and querying a second numerical value having a specific matching relationship with the first numerical value, wherein the specific matching relationship means that the absolute value of the difference between the second numerical value and the first numerical value is within a preset numerical range;

the screening module is used for screening out the assigned historical data matching tasks corresponding to the second numerical value from the task database and searching out the assigned memory usage ratio values corresponding to the assigned historical tasks respectively;

the third calculation module is used for calculating the average value of all the specified memory usage ratio values;

and the determining module is used for taking the average value as the memory use ratio threshold value.

In this embodiment, the implementation processes of the functions and actions of the calling module, the third obtaining module, the querying module, the screening module, the third calculating module and the determining module in the data matching device are specifically described in the implementation processes corresponding to steps S4000 to S4005 in the data matching method, and are not described herein again.

the third judgment module is used for acquiring the current time and judging whether the current time is within a preset time range;

a fourth obtaining module, configured to obtain preset mail login information and obtain an appointed mail address corresponding to an appointed user if the current time is within a preset time range;

the login module is used for logging in a corresponding mail server according to the mail login information;

and the sending module is used for sending the data matching result to the specified mail address through the mail server.

In this embodiment, the implementation processes of the functions and actions of the third determining module, the fourth obtaining module, the login module and the sending module in the data matching apparatus are specifically described in the implementation processes corresponding to steps S710 to S713 in the data matching method, and are not described herein again.

Referring to fig. 3, a computer device, which may be a server and whose internal structure may be as shown in fig. 3, is also provided in the embodiment of the present application. The computer device comprises a processor, a memory, a network interface, a display screen, an input device and a database which are connected through a system bus. Wherein the processor of the computer device is designed to provide computing and control capabilities. The memory of the computer device comprises a storage medium and an internal memory. The storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operating system and computer programs in the storage medium to run. The database of the computer equipment is used for storing data to be detected, a target data set, a VBA code, a VBA task, target data, a first hash value, a designated hash value, a hash chain table and a data matching result. The network interface of the computer device is used for communicating with an external terminal through a network connection. The display screen of the computer equipment is an indispensable image-text output equipment in the computer, and is used for converting digital signals into optical signals so that characters and figures are displayed on the screen of the display screen. The input device of the computer equipment is the main device for information exchange between the computer and the user or other equipment, and is used for transmitting data, instructions, some mark information and the like to the computer. The computer program is executed by a processor to implement a data matching method.

The processor executes the steps of the data matching method:

judging whether a data matching instruction triggered by a user is received;

Those skilled in the art will appreciate that the structure shown in fig. 3 is only a block diagram of a part of the structure related to the present application, and does not constitute a limitation to the apparatus and the computer device to which the present application is applied.

An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements a data matching method, and specifically:

judging whether a data matching instruction triggered by a user is received;

To sum up, according to the data matching method, the data matching device, the computer device and the storage medium provided in the embodiments of the present application, after a user imports data to be detected and a target data set into an Excel document and triggers a data matching instruction, a preset VBA code is intelligently read to generate a corresponding VBA task. And then copying the data to be detected and the target data set to the memory based on the execution logic of the VBA task, and further performing data matching processing on the data to be detected and the target data set in the memory by utilizing a Hash method, so that a data matching result corresponding to the data to be detected is quickly generated, and the processing efficiency of data matching is effectively improved. According to the embodiment of the application, the VBA tool developed based on the VBA codes is used for executing data matching comparison processing in the Excel, other software or databases and other modes with higher cost are not needed to be installed, corresponding data matching processing can be executed based on a Hash method after the related data are copied in the Excel, and the safety and the portability of data are effectively guaranteed. In addition, the process of realizing data matching based on the VBA tool is simple to operate, most of common business operators can use the data matching tool conveniently, complex training is not needed, data matching tasks which can be completed by learning Excel functions and the like before can be completed quickly only by copying the data and clicking corresponding trigger buttons, user experience is effectively improved, and processing efficiency of data matching is improved.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and can include the processes of the embodiments of the methods described above when the computer program is executed. Any reference to memory, storage, database, or other medium provided herein and used in the examples may include non-volatile and/or volatile memory. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), double-rate SDRAM (SSRSDRAM), Enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, apparatus, article, or method. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, apparatus, article, or method that includes the element.

The above description is only a preferred embodiment of the present application, and not intended to limit the scope of the present application, and all modifications of equivalent structures and equivalent processes, which are made by the contents of the specification and the drawings of the present application, or which are directly or indirectly applied to other related technical fields, are also included in the scope of the present application.

Claims

1. A method of data matching, comprising:

judging whether a data matching instruction triggered by a user is received;

2. The data matching method according to claim 1, wherein the step of generating a hash chain table corresponding to the target data set based on the first hash value and the target data comprises:

3. The data matching method according to claim 1, wherein the step of performing data matching processing on the designated hash value by using the hash chain table to generate a data matching result corresponding to the data to be detected includes:

judging whether data are stored in the designated array or not;

4. The data matching method according to claim 3, wherein the step of determining whether the designated data matching the data to be detected exists in all the data stored in the designated array comprises:

judging whether the number of data contained in the specified data is 1 or not;

5. The data matching method according to claim 1, wherein before the step of copying the data to be detected and the target data set to a memory based on the execution logic of the VBA task, the method comprises:

6. The data matching method according to claim 5, wherein the step of obtaining the current memory usage value and the step of obtaining the preset memory usage duty ratio threshold value comprise:

calculating the average value of all the specific memory usage ratio values;

and taking the average value as the memory usage ratio threshold value.

7. The data matching method according to claim 1, wherein after the step of performing data matching processing on the designated hash value by using the hash chain table to generate a data matching result corresponding to the data to be detected, the data matching method comprises:

logging in a corresponding mail server according to the mail login information;

8. A data matching apparatus, comprising:

9. A computer device comprising a memory and a processor, the memory having stored therein a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.