CN117726253A - External data processing method, device, equipment and storage medium - Google Patents
External data processing method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN117726253A CN117726253A CN202311685591.6A CN202311685591A CN117726253A CN 117726253 A CN117726253 A CN 117726253A CN 202311685591 A CN202311685591 A CN 202311685591A CN 117726253 A CN117726253 A CN 117726253A
- Authority
- CN
- China
- Prior art keywords
- logistics
- information
- data
- processed
- standard
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 31
- 238000012545 processing Methods 0.000 claims abstract description 69
- 238000012795 verification Methods 0.000 claims abstract description 48
- 238000013507 mapping Methods 0.000 claims abstract description 33
- 238000013500 data storage Methods 0.000 claims abstract description 32
- 238000000034 method Methods 0.000 claims abstract description 23
- 238000006243 chemical reaction Methods 0.000 claims description 30
- 230000006870 function Effects 0.000 claims description 16
- 230000014509 gene expression Effects 0.000 claims description 14
- 238000012163 sequencing technique Methods 0.000 claims description 9
- 238000010801 machine learning Methods 0.000 claims description 6
- 230000005587 bubbling Effects 0.000 claims description 5
- 230000008569 process Effects 0.000 claims description 5
- 230000001788 irregular Effects 0.000 abstract description 7
- 238000007726 management method Methods 0.000 abstract description 7
- 238000005516 engineering process Methods 0.000 abstract description 3
- 238000012544 monitoring process Methods 0.000 abstract description 3
- 238000004364 calculation method Methods 0.000 abstract description 2
- 238000003058 natural language processing Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 3
- 230000006872 improvement Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 238000012790 confirmation Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 238000002372 labelling Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 241000282693 Cercopithecidae Species 0.000 description 1
- 230000001174 ascending effect Effects 0.000 description 1
- 238000012550 audit Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000001351 cycling effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 239000003999 initiator Substances 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The present invention relates to the field of data processing technologies, and in particular, to an external data processing method, apparatus, device, and storage medium, where the method includes: acquiring historical standard order data to construct a data storage mapping relation; acquiring to-be-processed logistics order data and performing verification processing; when the to-be-processed logistics order data passes the verification, acquiring updating information which corresponds to the to-be-processed logistics order data and is fed back by the approval node so as to generate an initial logistics data set; carrying out standardization processing on the initial logistics data set to obtain a plurality of standard logistics information; encrypting the plurality of standard logistics information one by one to obtain a plurality of encrypted logistics information; storing a plurality of encrypted logistics information to a corresponding database based on the constructed data storage mapping relation; the method disclosed by the application can be used for carrying out standardized processing on the to-be-processed logistics order data, is convenient for the EPR system to carry out cost calculation, can provide basis for subsequent settlement, and realizes unified monitoring and management of irregular orders.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a method, an apparatus, a device, and a storage medium for processing external data.
Background
The logistics orders formed by each single customer placing orders on different shopping platforms are irregular logistics orders, and because the irregular logistics orders come from different shopping platforms, the information labeling methods of the different shopping platforms are different, namely, the information labeling methods cannot be unified with the cost calculation mode preset by the EPR system, so that after the irregular orders are input into the EPR system, great difference exists between the calculated cost and the actual cost, namely, the conventional EPR system cannot realize unified processing and management of the irregular logistics order data, cannot provide data support and basis for settlement of network points and large customers, and the overall operation efficiency of enterprises is reduced.
It can be seen that there is a need for improvements and improvements in the art.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention aims to provide an external data processing method, an external data processing device, external data processing equipment and a storage medium, which can perform standardized processing on the logistics order data to be processed and ensure that the cost calculated by an EPR system is correct and effective.
The first aspect of the present invention provides an external data processing method, including: acquiring historical standard order data, and constructing a data storage mapping relation according to the historical standard order data; acquiring to-be-processed logistics order data, and checking the to-be-processed logistics order data; when the to-be-processed logistics order data passes the verification, acquiring update information which corresponds to the to-be-processed logistics order data and is fed back by an approval node, and generating an initial logistics data set according to the update information; carrying out standardization processing on the initial logistics data set to obtain a plurality of standard logistics information; encrypting the plurality of standard logistics information one by one to obtain a plurality of encrypted logistics information; and storing the plurality of encrypted logistics information into a corresponding database based on the constructed data storage mapping relation.
Optionally, in a first implementation manner of the first aspect of the present invention, the acquiring historical standard order data and constructing the data storage mapping relationship according to the historical standard order data specifically includes: acquiring historical standard order data, wherein the historical standard order data comprises a plurality of historical standard orders; extracting standard information in the historical standard order, and confirming the data quantity in the standard information by adopting a regular expression; comparing the confirmed data quantity with a preset data quantity threshold value, and dividing a plurality of historical standard orders into first-class logistics information and second-class logistics information according to a comparison result; and constructing a data storage mapping relation according to the first type of logistics information and the second type of logistics information.
Optionally, in a second implementation manner of the first aspect of the present invention, the constructing a data storage mapping relationship according to the first type of logistics information and the second type of logistics information specifically includes: in the first type of logistics information, obtaining update nodes corresponding to the historical standard orders, and calculating the update times corresponding to the historical standard orders according to the number of the update nodes; adopting an bubbling sequencing method, sequencing a plurality of historical standard orders according to the updating times, and obtaining first appointed logistics information according to a sequencing result; establishing a first corresponding relation between the first appointed logistics information and the cache database, and establishing a second corresponding relation between the second type logistics information and other logistics information except the first appointed logistics information in the first type logistics information and the relation database; and integrating the first corresponding relation and the second corresponding relation to obtain a data storage mapping relation.
Optionally, in a third implementation manner of the first aspect of the present invention, the obtaining the to-be-processed logistics order data and performing verification processing on the to-be-processed logistics order data specifically includes: acquiring to-be-processed logistics order data, and extracting to-be-processed information in the to-be-processed logistics order data based on an NLP language model; judging whether the information to be processed comprises necessary filling information or not by adopting a pre-trained classifier based on a machine learning algorithm, wherein the necessary filling information at least comprises a bill number and an information source; carrying out authenticity verification on the freight bill number by adopting a regular expression; when the information to be processed does not comprise the necessary filling information or the order number does not pass the authenticity verification, the order data to be processed does not pass the verification, and an information error instruction is returned.
Optionally, in a fourth implementation manner of the first aspect of the present invention, when the to-be-processed logistics order data passes the verification, update information corresponding to the to-be-processed logistics order data and fed back by the approval node is obtained, and an initial logistics data set is generated according to the update information, which specifically includes: when the to-be-processed logistics order data passes the verification, acquiring the service type of the to-be-processed order data; confirming approval nodes corresponding to the to-be-processed order data according to the service types, wherein the approval nodes comprise a filling node and at least one updating node; and acquiring updating information fed back by the updating node, and generating an initial logistics data set according to the updating information, wherein the initial logistics data set comprises a plurality of initial logistics information.
Optionally, in a fifth implementation manner of the first aspect of the present invention, the normalizing the initial logistic data set to obtain a plurality of standard logistic information specifically includes: confirming the information type which is included in the initial logistics data set and corresponds to the initial logistics information; a KMP algorithm is adopted, and a preset standard conversion table is searched according to the information type so as to obtain a unit conversion function corresponding to the information type; and carrying out unit conversion processing on the plurality of initial logistics information one by adopting a corresponding unit conversion function to obtain a plurality of standard logistics information.
Optionally, in a sixth implementation manner of the first aspect of the present invention, the encrypting processing is performed on the plurality of standard logistics information one by one to obtain a plurality of encrypted logistics information, which specifically includes: generating an RSA key corresponding to the standard logistics information by using an encryption library, and storing the generated RSA key in a PEM format; adopting an RSA encryption algorithm and the generated RSA secret key to encrypt the standard logistics information based on a cipher block link mode to obtain a plurality of encrypted logistics information; and when the preset updating time is reached, updating the RSA key corresponding to the standard logistics information and storing the RSA key.
A second aspect of the present invention provides an external data processing apparatus, comprising: the acquisition module is used for acquiring historical standard order data and constructing a data storage mapping relation according to the historical standard order data; the verification module is used for acquiring the to-be-processed logistics order data and carrying out verification processing on the to-be-processed logistics order data; the updating module is used for acquiring updating information which corresponds to the to-be-processed logistics order data and is fed back by the approval node when the to-be-processed logistics order data passes the verification, and generating an initial logistics data set according to the updating information; the processing module is used for carrying out standardized processing on the initial logistics data set to obtain a plurality of standard logistics information; the encryption module is used for encrypting the plurality of standard logistics information one by one to obtain a plurality of encrypted logistics information; and the storage module is used for storing the plurality of encrypted logistics information into the corresponding database based on the constructed data storage mapping relation.
Optionally, in a first implementation manner of the second aspect of the present invention, the acquiring module includes: a first acquisition unit configured to acquire historical standard order data including a plurality of historical standard orders; the extraction unit is used for extracting standard information in the historical standard order and confirming the data quantity in the standard information by adopting a regular expression; the comparison unit is used for comparing the confirmed data quantity with a preset data quantity threshold value and dividing the historical standard orders into first-class logistics information and second-class logistics information according to the comparison result; and the construction unit is used for constructing a data storage mapping relation according to the first type of logistics information and the second type of logistics information.
Optionally, in a second implementation manner of the second aspect of the present invention, the acquiring module further includes: the second acquisition unit is used for acquiring update nodes corresponding to the historical standard orders in the first type of logistics information, and calculating the update times corresponding to the historical standard orders according to the number of the update nodes; the ordering unit is used for ordering the plurality of historical standard orders according to the updating times by adopting an bubbling ordering method, and obtaining first appointed logistics information according to an ordering result; the establishing unit is used for establishing a first corresponding relation between the first appointed logistics information and the cache database, and establishing second class logistics information and a second corresponding relation between other logistics information except the first appointed logistics information in the first class logistics information and the relation database; and the integration unit is used for integrating the first corresponding relation and the second corresponding relation to obtain a data storage mapping relation.
Optionally, in a third implementation manner of the second aspect of the present invention, the verification module includes: the third acquisition unit is used for acquiring the to-be-processed logistics order data and extracting to-be-processed information in the to-be-processed logistics order data based on the NLP language model; the judging unit is used for judging whether the information to be processed comprises necessary filling information or not by adopting a pre-trained classifier based on a machine learning algorithm, wherein the necessary filling information at least comprises a bill number and an information source; the verification unit is used for carrying out authenticity verification on the freight bill number by adopting a regular expression; and the return unit is used for returning an information error instruction when the information to be processed does not comprise the necessary filling information or the order number fails to pass the authenticity verification, and the data of the order to be processed fails to pass the verification.
Optionally, in a fourth implementation manner of the second aspect of the present invention, the updating module includes: the fourth acquisition unit is used for acquiring the service type of the order data to be processed when the order data of the logistics to be processed passes the verification; the first confirmation unit is used for confirming an approval node corresponding to the to-be-processed order data according to the service type, and the approval node comprises a filling node and at least one updating node; the first generation unit is used for acquiring the update information fed back by the update node, and generating an initial logistics data set according to the update information, wherein the initial logistics data set comprises a plurality of initial logistics information.
Optionally, in a fifth implementation manner of the second aspect of the present invention, the processing module includes: the second confirming unit is used for confirming the information type which is included in the initial logistics data set and corresponds to the initial logistics information; the matching unit is used for searching a preset standard conversion table according to the information type by adopting a KMP algorithm so as to obtain a unit conversion function corresponding to the information type; and the conversion unit is used for carrying out unit conversion processing on the plurality of initial logistics information one by adopting the corresponding unit conversion function to obtain a plurality of standard logistics information.
Optionally, in a sixth implementation manner of the second aspect of the present invention, the encryption module includes: a second generation unit for generating an RSA key corresponding to the standard logistics information using the encryption library, and storing the generated RSA key in PEM format; the encryption unit is used for encrypting the standard logistics information based on the cipher block link mode by adopting an RSA encryption algorithm and the generated RSA key to obtain a plurality of encrypted logistics information; and the updating unit is used for updating the RSA key corresponding to the standard logistics information and storing the RSA key when the preset updating time is reached.
A third aspect of the present invention provides an external data processing apparatus comprising: a memory and at least one processor, the memory having instructions stored therein; at least one of the processors invokes the instructions in the memory to cause the external data processing apparatus to perform the steps of the external data processing method of any one of the preceding claims.
A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon instructions which, when executed by a processor, implement the steps of the external data processing method of any of the above.
In the technical scheme of the invention, the data storage mapping relation is constructed by acquiring historical standard order data; acquiring to-be-processed logistics order data and performing verification processing; when the to-be-processed logistics order data passes the verification, acquiring updating information which corresponds to the to-be-processed logistics order data and is fed back by the approval node so as to generate an initial logistics data set; carrying out standardization processing on the initial logistics data set to obtain a plurality of standard logistics information; encrypting the plurality of standard logistics information one by one to obtain a plurality of encrypted logistics information; storing a plurality of encrypted logistics information to a corresponding database based on the constructed data storage mapping relation; the method disclosed by the application can be used for carrying out standardized processing on the logistics order data to be processed, ensuring that the cost calculated by the EPR system is accurate and effective, providing a basis for subsequent settlement, realizing unified monitoring and management of irregular orders and improving the overall operation efficiency of enterprises.
Drawings
FIG. 1 is a first flowchart of an external data processing method according to an embodiment of the present invention;
FIG. 2 is a second flowchart of an external data processing method according to an embodiment of the present invention;
FIG. 3 is a third flowchart of an external data processing method according to an embodiment of the present invention;
FIG. 4 is a fourth flowchart of an external data processing method according to an embodiment of the present invention;
FIG. 5 is a fifth flowchart of an external data processing method according to an embodiment of the present invention;
FIG. 6 is a sixth flowchart of an external data processing method according to an embodiment of the present invention;
FIG. 7 is a seventh flowchart of an external data processing method according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of an external data processing apparatus according to an embodiment of the present invention;
fig. 9 is a schematic structural diagram of an external data processing device according to an embodiment of the present invention.
Detailed Description
The present invention provides an external data processing method, apparatus, device and storage medium, and the terms "first", "second", "third", "fourth", etc. (if any) in the description and claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments described herein may be implemented in other sequences than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus.
For ease of understanding, a specific flow of an embodiment of the present invention is described below with reference to fig. 1, where an embodiment of an external data processing method in an embodiment of the present invention includes:
101. acquiring historical standard order data, and constructing a data storage mapping relation according to the historical standard order data;
in this embodiment, the historical standard order data may be order data pre-stored in the EPR system, where a unit of the logistics information included in the order data is consistent with a standard unit of the EPR system; after the storage mapping relation is established in advance, the logistics information included in the order data to be processed is stored correspondingly conveniently, the management standardization of the EPR system on the data is improved, and the user can review the data conveniently.
102. Acquiring to-be-processed logistics order data, and checking the to-be-processed logistics order data;
in this embodiment, after the to-be-processed logistics order data is obtained, verification processing is first performed to ensure that the obtained to-be-processed logistics order data is the logistics order data of the logistics company, so that the logistics orders of other logistics companies are prevented from being processed in error, and the data processing efficiency is improved.
103. When the to-be-processed logistics order data passes the verification, acquiring update information which corresponds to the to-be-processed logistics order data and is fed back by an approval node, and generating an initial logistics data set according to the update information;
In this embodiment, after the to-be-processed logistics order data passes the verification, update information of the to-be-processed logistics order data is obtained according to the approval node corresponding to the to-be-processed logistics order data, so as to ensure that the logistics order data is sufficient and perfect.
104. Carrying out standardization processing on the initial logistics data set to obtain a plurality of standard logistics information;
in this embodiment, by performing standardization processing on a plurality of pieces of initial logistics information included in the initial logistics data set one by one, the unit of the initial logistics information is converted into a standard unit consistent with that preset by the EPR system, so as to ensure that the cost calculated by the EPR system according to each piece of logistics information is accurate and effective.
105. Encrypting the plurality of standard logistics information one by one to obtain a plurality of encrypted logistics information;
in this embodiment, before storing the standard logistics information in the corresponding database, the standard logistics information is encrypted, so that confidentiality of the information can be effectively protected, only authorized users can access and understand the sensitive data, and the enterprise management standardization is improved.
106. And storing the plurality of encrypted logistics information into a corresponding database based on the constructed data storage mapping relation.
In this embodiment, the encrypted logistics information is divided into first appointed logistics information or other first type logistics information or second type logistics information according to the data amount corresponding to the plurality of encrypted logistics information, and then the plurality of encrypted logistics information is stored into a corresponding database according to the constructed first corresponding relation and second corresponding relation, wherein the database comprises a relation database and a cache database; storing the encrypted logistics information into a corresponding database, so that the subsequent data searching is facilitated, and the unified management of the data is facilitated; for example, taking reimbursement logistics as an example, logistics information such as a bill number, a commodity name, a weight (kg), a volume weight (kg), a logistics state and the like is stored in a BXDJ table in a relational database, and logistics state information such as a logistics number, a logistics state and the like is stored in a cache database, wherein the cache database can be a remote dictionary service database.
The application discloses an external data processing method, which comprises the steps of obtaining historical standard order data to construct a data storage mapping relation; acquiring to-be-processed logistics order data and performing verification processing; when the to-be-processed logistics order data passes the verification, acquiring updating information which corresponds to the to-be-processed logistics order data and is fed back by the approval node so as to generate an initial logistics data set; carrying out standardization processing on the initial logistics data set to obtain a plurality of standard logistics information; encrypting the plurality of standard logistics information one by one to obtain a plurality of encrypted logistics information; storing a plurality of encrypted logistics information to a corresponding database based on the constructed data storage mapping relation; the method disclosed by the application can be used for carrying out standardized processing on the logistics order data to be processed, ensuring that the cost calculated by the EPR system is accurate and effective, providing a basis for subsequent settlement, realizing unified monitoring and management of irregular orders and improving the overall operation efficiency of enterprises.
Referring to fig. 2, a second embodiment of the external data processing method according to the present invention includes:
201. acquiring historical standard order data, wherein the historical standard order data comprises a plurality of historical standard orders;
202. extracting standard information in the historical standard order, and confirming the data quantity in the standard information by adopting a regular expression;
in this embodiment, the data amount in the standard information is confirmed by using a regular expression, which is a text pattern that can be used to describe, match or search the text pattern; regular expressions are commonly used to search, replace, or match a particular pattern in a string of characters, which consists of a string of special characters representing a particular character set and matching pattern; the regular expression may be implemented using a findall function in the re module.
203. Comparing the confirmed data quantity with a preset data quantity threshold value, and dividing a plurality of historical standard orders into first-class logistics information and second-class logistics information according to a comparison result;
in this embodiment, the preset data amount threshold is preset by a manager according to the data amount of the daily order; and when the data volume corresponding to each standard information, which is confirmed in the historical standard order information, is not larger than a preset data volume threshold value, dividing the historical standard order information into first-class logistics information, and otherwise, dividing the historical standard order information into second-class logistics information.
204. And constructing a data storage mapping relation according to the first type of logistics information and the second type of logistics information.
Referring to fig. 3, a third embodiment of an external data processing method according to an embodiment of the present invention includes:
301. in the first type of logistics information, obtaining update nodes corresponding to the historical standard orders, and calculating the update times corresponding to the historical standard orders according to the number of the update nodes;
in this embodiment, the historical standard orders may correspond to one or more update nodes, where each update node updates the historical standard orders once, that is, the number of update nodes corresponds to the number of updates.
302. Adopting an bubbling sequencing method, sequencing a plurality of historical standard orders according to the updating times, and obtaining first appointed logistics information according to a sequencing result;
in this embodiment, the historical standard order with the largest number of update nodes or the largest number of corresponding update times to be updated is selected as the first appointed logistics information.
In this embodiment, the ranking of the plurality of historical standard orders is performed using a bubble ranking method, which is a simple ranking algorithm that repeatedly traverses the sequence to be ranked, compares two elements at a time, and swaps them if their order is wrong; the task of traversing the sequence is repeated until no more exchanges are needed, i.e. the sequence has been ordered.
The basic steps of the bubble ordering algorithm are as follows: 1. comparing adjacent elements, if the first is larger than the second (in ascending order), exchanging them two; 2. doing the same for each pair of adjacent elements, from the first pair to the last pair of the end; 3. repeating the above steps for all elements except the last one; 4. the above steps continue to be repeated for fewer and fewer elements at a time until no pair of numbers need to be compared.
303. Establishing a first corresponding relation between the first appointed logistics information and the cache database, and establishing a second corresponding relation between the second type logistics information and other logistics information except the first appointed logistics information in the first type logistics information and the relation database;
304. and integrating the first corresponding relation and the second corresponding relation to obtain a data storage mapping relation.
Referring to fig. 4, a fourth embodiment of the external data processing method according to the embodiment of the present invention includes:
401. acquiring to-be-processed logistics order data, and extracting to-be-processed information in the to-be-processed logistics order data based on an NLP language model;
in this embodiment, named Entity Recognition (NER) in Natural Language Processing (NLP) technology may be used to identify the person name, place name, organization name, time, number, etc. in the order data of the stream to be processed, and the extracted view constitutes the information to be processed.
402. Judging whether the information to be processed comprises necessary filling information or not by adopting a pre-trained classifier based on a machine learning algorithm, wherein the necessary filling information at least comprises a bill number and an information source;
in this embodiment, a classifier may be obtained by machine learning using a monkey learn to determine whether the information to be processed includes the necessary padding information.
In this embodiment, the information source may be confirmed by an app key, and specifically, the app key is used to confirm the open platform of a specific external system.
403. Carrying out authenticity verification on the freight bill number by adopting a regular expression;
in this embodiment, different carrier companies and service providers may have different ticket formats and specifications, and whether the acquired ticket number meets a specific format requirement may be confirmed by using a regular expression, so as to implement verification of authenticity of the ticket number, and ensure that the acquired to-be-processed logistics order data is the ticket of the logistics company.
404. When the information to be processed does not comprise the necessary filling information or the order number does not pass the authenticity verification, the order data to be processed does not pass the verification, and an information error instruction is returned;
in this embodiment, when the order data to be processed fails to pass the inspection, an information error command is returned, the next order data to be processed is obtained again for processing, and the order data to be processed that fails to pass the inspection is returned to the manual processing to confirm the error condition.
Referring to fig. 5, a fifth embodiment of the external data processing method according to the embodiment of the present invention includes:
501. when the to-be-processed logistics order data passes the verification, acquiring the service type of the to-be-processed order data;
in this embodiment, the service type refers to a service logistics type related to economic traffic in service data, and the service type may be reimbursement logistics, approval logistics, and the like.
502. Confirming approval nodes corresponding to the to-be-processed order data according to the service types, wherein the approval nodes comprise a filling node and at least one updating node;
in this embodiment, the update node may also be referred to as an audit node, a payment node, etc., and the reporting node may be considered as an initiator of the service flow; for example, a user logs in the ERP system, adds a reimbursement bill, submits after filling, and here the user fills the node for the business logistics in the circulation node.
503. Acquiring updating information fed back by an updating node, and generating an initial logistics data set according to the updating information, wherein the initial logistics data set comprises a plurality of initial logistics information;
in this embodiment, the initial logistics information may be trade name, length (cm), width (cm), height (cm), weight (kg), bulk weight (kg), logistics status, etc.
Referring to fig. 6, a sixth embodiment of an external data processing method according to an embodiment of the present invention includes:
601. confirming the information type which is included in the initial logistics data set and corresponds to the initial logistics information;
602. a KMP algorithm is adopted, and a preset standard conversion table is searched according to the information type so as to obtain a unit conversion function corresponding to the information type;
in this embodiment, a rule matching algorithm may be used to find a preset standard conversion table to obtain a unit conversion function corresponding to an information type, where the rule matching algorithm may be a KMP algorithm, and a basic idea of the KMP algorithm is that: starting from the first character of the main string test and the pattern string pattern, comparing the characters of the two character strings one by one, if a certain character is not matched, backtracking the main string to the second character, backtracking the sub-string to the first character, then comparing one by one, if a certain character is not matched, backtracking the main string to the third character, backtracking the sub-string to the first character, then comparing one by one, and cycling until all the sub-string characters are successfully matched, thereby realizing the acquisition of the unit conversion function.
In this embodiment, the standard conversion table records a plurality of information types and unit conversion functions corresponding to the information types, where the unit conversion functions may be established by an interpolation algorithm, such as a linear difference algorithm, and the established unit conversion functions may calculate corresponding target unit values according to the input original unit values, and may ensure that the conversion is accurate in the whole range.
603. Performing unit conversion processing on the plurality of initial logistics information one by adopting a corresponding unit conversion function to obtain a plurality of standard logistics information;
in this embodiment, when the initial logistics information is the length, the unit thereof is converted into cm; when the initial logistics information is width, converting the unit of the initial logistics information into cm; when the initial logistics information is high, converting the unit of the initial logistics information into cm; when the initial logistics information is weight, converting the unit into kg; when the initial logistic information is the volume weight, the unit is converted into kg.
Referring to fig. 7, a seventh embodiment of an external data processing method according to an embodiment of the present invention includes:
701. generating an RSA key corresponding to the standard logistics information by using an encryption library, and storing the generated RSA key in a PEM format;
in this embodiment, an RSA key may be generated using a cryptograph library, where the RSA key is a key pair, and includes a public key and a private key; an RSA key is generated through an encryption library, so that the key is convenient to manage; after the key is generated, only the authorized user can use the correct key to perform decryption operation, so that the original plaintext information is restored.
702. Adopting an RSA encryption algorithm and the generated RSA secret key to encrypt the standard logistics information based on a cipher block link mode to obtain a plurality of encrypted logistics information;
In this embodiment, selecting a secure encryption algorithm is the first step of protecting information, the key is the key used to encrypt and decrypt the information, and the encryption mode defines how to encrypt the data block; and the encryption of standard logistics information is realized through the cooperation of an algorithm, a secret key and a mode.
In this embodiment, in order to further improve the security of the information transmission process when transmitting or storing the encrypted logistics information, a secure communication protocol, such as TLS/SSL, may be used to protect the encrypted logistics information from being tampered or eavesdropped during the transmission process.
703. When the preset updating time is reached, updating an RSA key corresponding to the standard logistics information and storing the RSA key;
in this embodiment, the key may be updated once every month, that is, the preset update time may be set to number 1 of each month, and when the first day of each month is reached, the key is updated through the encryption library and the update is stored; periodically updating keys is an important step in maintaining information security to address constant security threats and attacks.
The method for processing external data in the embodiment of the present invention is described above, and the external data processing apparatus in the embodiment of the present invention is described below, referring to fig. 8, where an embodiment of the external data processing apparatus in the embodiment of the present invention includes:
The acquiring module 801 is configured to acquire historical standard order data, and construct a data storage mapping relationship according to the historical standard order data; the verification module 802 is configured to obtain to-be-processed logistics order data, and perform verification processing on the to-be-processed logistics order data; the updating module 803 is configured to obtain, when the to-be-processed logistics order data passes the verification, updating information corresponding to the to-be-processed logistics order data and fed back by the approval node, and generate an initial logistics data set according to the updating information; the processing module 804 is configured to perform standardization processing on the initial logistics data set to obtain a plurality of standard logistics information; the encryption module 805 is configured to encrypt the plurality of standard logistics information one by one to obtain a plurality of encrypted logistics information; the storage module 806 is configured to store the plurality of encrypted logistics information into a corresponding database based on the constructed data storage mapping relationship.
In this embodiment, the obtaining module 801 includes: a first obtaining unit 8011, configured to obtain historical standard order data, where the historical standard order data includes a plurality of historical standard orders; the extraction unit 8012 is used for extracting standard information in the historical standard order and confirming the data volume in the standard information by adopting a regular expression; the comparing unit 8013 is configured to compare the confirmed data amount with a preset data amount threshold, and divide the plurality of historical standard orders into first-class logistics information and second-class logistics information according to a comparison result; and the construction unit 8014 is configured to construct a data storage mapping relationship according to the first type of logistics information and the second type of logistics information.
In this embodiment, the obtaining module 801 further includes: a second obtaining unit 8015, configured to obtain update nodes corresponding to the historical standard orders in the first type of logistics information, and calculate the update times corresponding to the historical standard orders according to the number of the update nodes; the ordering unit 8016 is configured to perform ordering processing on the plurality of historical standard orders according to the update times by using an bubbling ordering method, and obtain first specified logistics information according to an ordering result; the establishing unit 8017 is configured to establish a first correspondence between the first specified logistics information and the cache database, and establish a second correspondence between the second type of logistics information and the relationship database and other logistics information in the first type of logistics information except the first specified logistics information; the integrating unit 8018 is configured to integrate the first correspondence and the second correspondence to obtain a data storage mapping relationship.
In this embodiment, the verification module 802 includes: a third obtaining unit 8021, configured to obtain to-be-processed logistics order data, and extract to-be-processed information in the to-be-processed logistics order data based on the NLP language model; a judging unit 8022, configured to judge whether the information to be processed includes necessary filling information, where the necessary filling information includes at least a bill number and an information source, by using a pre-trained classifier based on a machine learning algorithm; the verification unit 8023 is used for carrying out authenticity verification on the freight bill number by adopting a regular expression; and a return unit 8024, configured to return an information error instruction when the information to be processed does not include the necessary filling information or the order number fails the authenticity verification, and the order data to be processed fails the verification.
In this embodiment, the updating module 803 includes: a fourth obtaining unit 8031, configured to obtain a service type of the order data to be processed when the order data of the logistics to be processed passes the verification; a first confirmation unit 8032, configured to confirm, according to a service type, an approval node corresponding to the to-be-processed order data, where the approval node includes a filling node and at least one updating node; the first generation unit 8033 is configured to obtain update information fed back by the update node, and generate an initial logistics data set according to the update information, where the initial logistics data set includes a plurality of initial logistics information.
In this embodiment, the processing module 804 includes: a second confirming unit 8041 for confirming an information type corresponding to the initial logistics information included in the initial logistics data set; the matching unit 8042 is configured to search a preset standard conversion table according to an information type by adopting a KMP algorithm, so as to obtain a unit conversion function corresponding to the information type; the conversion unit 8043 is configured to perform unit conversion processing on the plurality of initial logistics information one by using a corresponding unit conversion function, so as to obtain a plurality of standard logistics information.
In this embodiment, the encryption module 805 includes: a second generation unit 8051, configured to generate an RSA key corresponding to the standard logistics information using the encryption base, and store the generated RSA key in PEM format; the encryption unit 8052 is configured to encrypt the standard logistics information based on the cipher block link mode by using an RSA encryption algorithm and the generated RSA key, so as to obtain a plurality of encrypted logistics information; and an updating unit 8053, configured to update the RSA key corresponding to the standard logistics information and store the RSA key when a preset update time is reached.
The external data processing apparatus in the embodiment of the present invention is described in detail above in fig. 8 from the point of view of the modularized functional entity, and the external data processing device in the embodiment of the present invention is described in detail below from the point of view of hardware processing.
Fig. 9 is a schematic structural diagram of an external data processing device according to an embodiment of the present invention, where the external data processing device 900 may have a relatively large difference due to different configurations or performances, and may include one or more processors (central processing units, CPU) 910 (e.g., one or more processors) and a memory 920, and one or more storage media 930 (e.g., one or more mass storage devices) storing application programs 933 or data 932. Wherein the memory 920 and storage medium 930 may be transitory or persistent storage. The program stored on the storage medium 930 may include one or more modules (not shown), each of which may include a series of instruction operations on the external data processing apparatus 900. Still further, the processor 910 may be configured to communicate with a storage medium 930 and execute a series of instruction operations in the storage medium 930 on the external data processing apparatus 900 to implement the steps of the external data processing method provided in the above-described method embodiments.
The external data processing device 900 may also include one or more power supplies 940, one or more wired or wireless network interfaces 950, one or more input/output interfaces 960, and/or one or more operating systems 931, such as Windows Serve, mac OS X, unix, linux, freeBSD, and the like. It will be appreciated by those skilled in the art that the external data processing device structure shown in the present application is not limiting on the external data processing device-based structure, and may include more or fewer components than shown, or may combine certain components, or may be arranged in different ways.
The present invention also provides a computer readable storage medium, which may be a non-volatile computer readable storage medium, and which may also be a volatile computer readable storage medium, having stored therein instructions which, when executed on a computer, cause the computer to perform the steps of the external data processing method.
It will be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process of the system or apparatus and unit described above may refer to the corresponding process in the foregoing method embodiment, which is not repeated herein.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a storage medium, including instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a random access memory (random access memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
Finally, it should be noted that: the foregoing is merely a preferred example of the present invention, and the present invention is not limited thereto, but it is to be understood that modifications and equivalents of some of the technical features described in the foregoing embodiments may be made by those skilled in the art, although the present invention has been described in detail with reference to the foregoing embodiments. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A method of external data processing, comprising:
acquiring historical standard order data, and constructing a data storage mapping relation according to the historical standard order data;
acquiring to-be-processed logistics order data, and checking the to-be-processed logistics order data;
when the to-be-processed logistics order data passes the verification, acquiring update information which corresponds to the to-be-processed logistics order data and is fed back by an approval node, and generating an initial logistics data set according to the update information;
carrying out standardization processing on the initial logistics data set to obtain a plurality of standard logistics information;
encrypting the plurality of standard logistics information one by one to obtain a plurality of encrypted logistics information;
and storing the plurality of encrypted logistics information into a corresponding database based on the constructed data storage mapping relation.
2. The external data processing method according to claim 1, wherein the obtaining the historical standard order data and constructing the data storage mapping relation according to the historical standard order data specifically comprises:
acquiring historical standard order data, wherein the historical standard order data comprises a plurality of historical standard orders;
extracting standard information in the historical standard order, and confirming the data quantity in the standard information by adopting a regular expression;
Comparing the confirmed data quantity with a preset data quantity threshold value, and dividing a plurality of historical standard orders into first-class logistics information and second-class logistics information according to a comparison result;
and constructing a data storage mapping relation according to the first type of logistics information and the second type of logistics information.
3. The external data processing method according to claim 2, wherein the constructing a data storage mapping relationship according to the first type of logistics information and the second type of logistics information specifically includes:
in the first type of logistics information, obtaining update nodes corresponding to the historical standard orders, and calculating the update times corresponding to the historical standard orders according to the number of the update nodes;
adopting an bubbling sequencing method, sequencing a plurality of historical standard orders according to the updating times, and obtaining first appointed logistics information according to a sequencing result;
establishing a first corresponding relation between the first appointed logistics information and the cache database, and establishing a second corresponding relation between the second type logistics information and other logistics information except the first appointed logistics information in the first type logistics information and the relation database;
and integrating the first corresponding relation and the second corresponding relation to obtain a data storage mapping relation.
4. The method for processing external data according to claim 1, wherein the step of obtaining the to-be-processed logistics order data and verifying the to-be-processed logistics order data specifically comprises:
acquiring to-be-processed logistics order data, and extracting to-be-processed information in the to-be-processed logistics order data based on an NLP language model;
judging whether the information to be processed comprises necessary filling information or not by adopting a pre-trained classifier based on a machine learning algorithm, wherein the necessary filling information at least comprises a bill number and an information source;
carrying out authenticity verification on the freight bill number by adopting a regular expression;
when the information to be processed does not comprise the necessary filling information or the order number does not pass the authenticity verification, the order data to be processed does not pass the verification, and an information error instruction is returned.
5. The method for processing external data according to claim 1, wherein when the to-be-processed logistics order data passes the verification, obtaining update information corresponding to the to-be-processed logistics order data and fed back by the approval node, and generating an initial logistics data set according to the update information, specifically comprising:
when the to-be-processed logistics order data passes the verification, acquiring the service type of the to-be-processed order data;
Confirming approval nodes corresponding to the to-be-processed order data according to the service types, wherein the approval nodes comprise a filling node and at least one updating node;
and acquiring updating information fed back by the updating node, and generating an initial logistics data set according to the updating information, wherein the initial logistics data set comprises a plurality of initial logistics information.
6. The external data processing method according to claim 1, wherein the normalizing the initial logistics data set to obtain a plurality of standard logistics information comprises:
confirming the information type which is included in the initial logistics data set and corresponds to the initial logistics information;
a KMP algorithm is adopted, and a preset standard conversion table is searched according to the information type so as to obtain a unit conversion function corresponding to the information type;
and carrying out unit conversion processing on the plurality of initial logistics information one by adopting a corresponding unit conversion function to obtain a plurality of standard logistics information.
7. The external data processing method according to claim 1, wherein the encrypting process is performed on the plurality of standard logistics information one by one to obtain a plurality of encrypted logistics information, and the method specifically comprises:
generating an RSA key corresponding to the standard logistics information by using an encryption library, and storing the generated RSA key in a PEM format;
Adopting an RSA encryption algorithm and the generated RSA secret key to encrypt the standard logistics information based on a cipher block link mode to obtain a plurality of encrypted logistics information;
and when the preset updating time is reached, updating the RSA key corresponding to the standard logistics information and storing the RSA key.
8. An external data processing apparatus, comprising:
the acquisition module is used for acquiring historical standard order data and constructing a data storage mapping relation according to the historical standard order data;
the verification module is used for acquiring the to-be-processed logistics order data and carrying out verification processing on the to-be-processed logistics order data;
the updating module is used for acquiring updating information which corresponds to the to-be-processed logistics order data and is fed back by the approval node when the to-be-processed logistics order data passes the verification, and generating an initial logistics data set according to the updating information;
the processing module is used for carrying out standardized processing on the initial logistics data set to obtain a plurality of standard logistics information;
the encryption module is used for encrypting the plurality of standard logistics information one by one to obtain a plurality of encrypted logistics information;
and the storage module is used for storing the plurality of encrypted logistics information into the corresponding database based on the constructed data storage mapping relation.
9. An external data processing apparatus, characterized in that the external data processing apparatus comprises: a memory and at least one processor, the memory having instructions stored therein;
at least one of the processors invokes the instructions in the memory to cause the external data processing apparatus to perform the steps of the external data processing method according to any of claims 1-7.
10. A computer readable storage medium having instructions stored thereon, which when executed by a processor, implement the steps of the external data processing method according to any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311685591.6A CN117726253A (en) | 2023-12-08 | 2023-12-08 | External data processing method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311685591.6A CN117726253A (en) | 2023-12-08 | 2023-12-08 | External data processing method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117726253A true CN117726253A (en) | 2024-03-19 |
Family
ID=90209855
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311685591.6A Pending CN117726253A (en) | 2023-12-08 | 2023-12-08 | External data processing method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117726253A (en) |
-
2023
- 2023-12-08 CN CN202311685591.6A patent/CN117726253A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10318932B2 (en) | Payment card processing system with structure preserving encryption | |
| US8938067B2 (en) | Format preserving encryption methods for data strings with constraints | |
| KR20180115768A (en) | Encryption method and system for secure extraction of data from a block chain | |
| US20240184919A1 (en) | Batch tokenization service | |
| CN110932859B (en) | User information processing method, device and equipment and readable storage medium | |
| CN108153858A (en) | Information-pushing method, device, storage medium and computer equipment | |
| CN109815051A (en) | The data processing method and system of block chain | |
| CN112039986A (en) | System and method for realizing information interaction among departments of enterprise | |
| CN110378796A (en) | Vehicle unique identification generation method, device, computer equipment and medium | |
| CN105553980A (en) | Safety fingerprint identification system and method based on cloud computing | |
| CN111798254A (en) | Tracing code generation method, device, equipment and readable storage medium | |
| CN111800387A (en) | Intelligent encryption transmission system for computer information data | |
| CN113191121A (en) | Express bill number generation method, device, equipment and storage medium | |
| US11362806B2 (en) | System and methods for recording codes in a distributed environment | |
| CN116757849B (en) | Asset management system and method based on block chain | |
| CN117726253A (en) | External data processing method, device, equipment and storage medium | |
| CN112069182A (en) | Batch case reporting method, device, equipment and storage medium | |
| CN114866271B (en) | Electronic certificate generation method, device, equipment and storage medium | |
| CN115758432A (en) | Omnibearing data encryption method and system based on machine learning algorithm | |
| CN113935874A (en) | District chain-based book management system for studying income | |
| Rao | Operation mode of electric business logistics based on the application of two-dimensional code technology | |
| Shi et al. | AUDITEM: toward an automated and efficient data integrity verification model using blockchain | |
| CN113421090B (en) | Method, device, equipment and storage medium for issuing electronic evidence | |
| CN118411231B (en) | Batch meal delivery management method and system based on virtual basket | |
| KR20120047720A (en) | Method and system of managing data transmission |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication |