CN113326281A - Logistics order data processing method, device, equipment and storage medium - Google Patents

Logistics order data processing method, device, equipment and storage medium Download PDF

Info

Publication number
CN113326281A
CN113326281A CN202110526493.2A CN202110526493A CN113326281A CN 113326281 A CN113326281 A CN 113326281A CN 202110526493 A CN202110526493 A CN 202110526493A CN 113326281 A CN113326281 A CN 113326281A
Authority
CN
China
Prior art keywords
order data
query
data
database
level
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110526493.2A
Other languages
Chinese (zh)
Inventor
杨周龙
李波涛
毛立贤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Dongpu Software Co Ltd
Original Assignee
Dongpu Software Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Dongpu Software Co Ltd filed Critical Dongpu Software Co Ltd
Priority to CN202110526493.2A priority Critical patent/CN113326281A/en
Publication of CN113326281A publication Critical patent/CN113326281A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/242—Query formulation
    • G06F16/2428—Query predicate definition using graphical user interfaces, including menus and forms
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2228—Indexing structures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2455—Query execution
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G06F21/602—Providing cryptographic facilities or services
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
    • G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
    • G06F21/6227—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database where protection concerns the structure of data, e.g. records, types, queries
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/08—Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
    • G06Q10/083—Shipping

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Software Systems (AREA)
  • Business, Economics & Management (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Computer Security & Cryptography (AREA)
  • Computer Hardware Design (AREA)
  • Economics (AREA)
  • Bioethics (AREA)
  • Operations Research (AREA)
  • Tourism & Hospitality (AREA)
  • General Business, Economics & Management (AREA)
  • Strategic Management (AREA)
  • Quality & Reliability (AREA)
  • Development Economics (AREA)
  • Marketing (AREA)
  • Human Resources & Organizations (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Computing Systems (AREA)
  • Human Computer Interaction (AREA)
  • Mathematical Physics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The invention relates to the field of logistics management, and provides a method, a device, equipment and a storage medium for processing logistics order data, which are used for solving the problem that high-concurrency and high-performance order data service cannot be provided. The method for processing the logistics order data comprises the following steps: acquiring logistics order data to be processed, and comparing and analyzing the logistics order data to be processed with preset level storage conditions to obtain a target storage level, wherein the level storage conditions comprise level order creation time and service requirements; acquiring a level database corresponding to a target storage level, and storing logistics order data to be processed into the level database; carrying out life cycle monitoring and grade database updating on logistics order data in the grade database to obtain data to be inquired; and when the query request is received, retrieving the data to be queried based on the query scene according to the query request to obtain the target logistics order data.

Description

Logistics order data processing method, device, equipment and storage medium
Technical Field
The present invention relates to the field of logistics management, and in particular, to a method, an apparatus, a device, and a storage medium for processing logistics order data.
Background
As the data volume of the logistics order data is larger and larger as time goes on, a part of the logistics order data in a data table is only used as archived historical data and the access frequency of the logistics order data is gradually reduced, and the data volume of the part of the historical data is very large, so that the logistics order data needs to be effectively managed.
At present, management of logistics order data is generally realized by constructing different multidimensional storage tables, storing logistics order data through the multidimensional storage tables and inquiring the logistics order data through the multidimensional storage tables. However, in the above-described method, since the storage pressure of the database is large, the flexibility of query between the multidimensional storage tables is low, the second query rate is low, and the processing speed is slow, it is not possible to provide a highly concurrent and high-performance order data service.
Disclosure of Invention
The invention provides a method, a device, equipment and a storage medium for processing logistics order data, which are used for solving the problem that order data service with high concurrency and high performance cannot be provided.
The first aspect of the present invention provides a method for processing logistics order data, including:
acquiring logistics order data to be processed, and comparing and analyzing the logistics order data to be processed with preset level storage conditions to obtain a target storage level, wherein the level storage conditions comprise level order creation time and service requirements;
acquiring a level database corresponding to the target storage level, and storing the logistics order data to be processed into the level database;
carrying out life cycle monitoring and grade database updating on the logistics order data in the grade database to obtain data to be inquired;
and when a query request is received, retrieving the data to be queried based on a query scene according to the query request to obtain target logistics order data.
Optionally, in a first implementation manner of the first aspect of the present invention, the obtaining a level database corresponding to the target storage level and storing the to-be-processed logistics order data in the level database includes:
retrieving a preset block chain data source based on the target storage level to obtain a corresponding level database, wherein the level database comprises a primary database, a secondary database and a tertiary database;
encrypting the logistics order data to be processed based on authorization information to obtain encrypted order data; partition table creation based on months is carried out on the encrypted order data to obtain processed encrypted order data, and the processed encrypted order data are written into the primary database;
acquiring the order number and the order main category of the encrypted order data, creating a target data table based on the order number and the order main category, and inserting the encrypted order data into the third-level database through the target data table;
and performing index fragmentation processing, association processing and secondary database writing on the encrypted order data.
Optionally, in a second implementation manner of the first aspect of the present invention, the performing index fragmentation processing, association processing and second-level database writing on the encrypted order data includes:
establishing and processing the encrypted order data based on a primary index and a secondary index to obtain a target index;
according to the target index, carrying out fragmentation processing on the encrypted order data to obtain order data to be stored;
establishing an associated foreign key of the target index and a target data table in the tertiary database, and performing key value-based data structure conversion on the order data to be stored to obtain target storage data;
and storing the target index establishing the associated foreign key and the target storage data into the secondary database.
Optionally, in a third implementation manner of the first aspect of the present invention, the performing life cycle monitoring and level database updating on the logistics order data in the level database to obtain data to be queried includes:
carrying out life cycle detection on the logistics order data in the level database based on a preset life cycle management and control mechanism to obtain data to be removed;
updating the data of the level database based on migration through the data to be removed to obtain an updated level database;
creating a virtual data view and a database mirror image file of the logistics order data in the updated level database;
and determining the virtual data view and the database mirror image file as data to be inquired.
Optionally, in a fourth implementation manner of the first aspect of the present invention, when receiving a query request, performing query-scenario-based retrieval on the data to be queried according to the query request to obtain target logistics order data, including:
when a query request triggered based on a virtual data view in the data to be queried is received, analyzing the query request to obtain query key information, wherein the query key information comprises query condition information and a query scene, and the data to be queried comprises the virtual data view and a database mirror image file;
and acquiring a query strategy corresponding to the query scene, and querying the database mirror image file through the query condition information and the query strategy to obtain target logistics order data, wherein the query strategy comprises a time attribute query strategy, a multidimensional query strategy and/or a key value query strategy.
Optionally, in a fifth implementation manner of the first aspect of the present invention, the obtaining a query policy corresponding to the query scenario, and querying the database image file through the query condition information and the query policy to obtain target logistics order data includes:
when the query scene is a time attribute query scene, acquiring a time attribute query strategy corresponding to the time attribute query scene, and performing time attribute-based query on the database mirror image file based on the query condition information through the time attribute query strategy to obtain target logistics order data;
when the query scene is a multidimensional query scene, acquiring a multidimensional query strategy corresponding to the multidimensional query scene, creating a multidimensional index of the query condition information through the multidimensional query strategy, and retrieving the database image file through the multidimensional index to obtain target logistics order data;
and when the query scene is a key value query scene, acquiring a key value query strategy corresponding to the key value query scene, creating a target key value of the query condition information through the key value query strategy, and performing priority-based level database key value query on the database mirror image file through the target key value to obtain target logistics order data.
Optionally, in a sixth implementation manner of the first aspect of the present invention, the obtaining logistics order data to be processed and comparing and analyzing the logistics order data to be processed with a preset level storage condition to obtain a target storage level includes:
acquiring authorization information, verifying a preset top-speed chain through the authorization information, and if the verification is passed, extracting initial logistics order data from a database corresponding to the top-speed chain;
carrying out security detection, integrity detection and data conversion on the initial logistics order data to obtain to-be-processed logistics order data;
creating multiple threads according to preset level storage conditions, performing parallel judgment and analysis on the logistics order data to be processed through the multiple threads to obtain judgment and analysis results, and determining a target storage level according to the judgment and analysis results.
A second aspect of the present invention provides a device for processing logistics order data, including:
the analysis module is used for acquiring logistics order data to be processed and comparing and analyzing the logistics order data to be processed with preset level storage conditions to obtain a target storage level, wherein the level storage conditions comprise level order creation time and service requirements;
the storage module is used for acquiring a level database corresponding to the target storage level and storing the logistics order data to be processed into the level database;
the monitoring updating module is used for carrying out life cycle monitoring and level database updating on the logistics order data in the level database to obtain data to be inquired;
and the query module is used for retrieving the data to be queried based on a query scene according to the query request when receiving the query request so as to obtain target logistics order data.
Optionally, in a first implementation manner of the second aspect of the present invention, the storage module includes:
the retrieval unit is used for retrieving a preset block chain data source based on the target storage level to obtain a corresponding level database, wherein the level database comprises a primary database, a secondary database and a tertiary database;
the encryption unit is used for encrypting the logistics order data to be processed based on the authorization information to obtain encrypted order data;
the creating unit is used for creating a partition table based on months for the encrypted order data to obtain processed encrypted order data and writing the processed encrypted order data into the primary database;
the inserting unit is used for acquiring the order number and the order main category of the encrypted order data, creating a target data table based on the order number and the order main category, and inserting the encrypted order data into the third-level database through the target data table;
and the writing unit is used for performing index fragmentation processing, association processing and secondary database writing on the encrypted order data.
Optionally, in a second implementation manner of the second aspect of the present invention, the writing unit is specifically configured to:
establishing and processing the encrypted order data based on a primary index and a secondary index to obtain a target index;
according to the target index, carrying out fragmentation processing on the encrypted order data to obtain order data to be stored;
establishing an associated foreign key of the target index and a target data table in the tertiary database, and performing key value-based data structure conversion on the order data to be stored to obtain target storage data;
and storing the target index establishing the associated foreign key and the target storage data into the secondary database.
Optionally, in a third implementation manner of the second aspect of the present invention, the monitoring update module is specifically configured to:
carrying out life cycle detection on the logistics order data in the level database based on a preset life cycle management and control mechanism to obtain data to be removed;
updating the data of the level database based on migration through the data to be removed to obtain an updated level database;
creating a virtual data view and a database mirror image file of the logistics order data in the updated level database;
and determining the virtual data view and the database mirror image file as data to be inquired.
Optionally, in a fourth implementation manner of the second aspect of the present invention, the query module includes:
the query unit is used for analyzing the query request to obtain query key information when receiving the query request triggered based on the virtual data view in the data to be queried, wherein the query key information comprises query condition information and a query scene, and the data to be queried comprises the virtual data view and a database mirror image file;
and the query unit is used for acquiring a query strategy corresponding to the query scene, and querying the database mirror image file through the query condition information and the query strategy to obtain target logistics order data, wherein the query strategy comprises a time attribute query strategy, a multidimensional query strategy and/or a key value query strategy.
Optionally, in a fifth implementation manner of the second aspect of the present invention, the querying unit is specifically configured to:
when the query scene is a time attribute query scene, acquiring a time attribute query strategy corresponding to the time attribute query scene, and performing time attribute-based query on the database mirror image file based on the query condition information through the time attribute query strategy to obtain target logistics order data;
when the query scene is a multidimensional query scene, acquiring a multidimensional query strategy corresponding to the multidimensional query scene, creating a multidimensional index of the query condition information through the multidimensional query strategy, and retrieving the database image file through the multidimensional index to obtain target logistics order data;
and when the query scene is a key value query scene, acquiring a key value query strategy corresponding to the key value query scene, creating a target key value of the query condition information through the key value query strategy, and performing priority-based level database key value query on the database mirror image file through the target key value to obtain target logistics order data.
Optionally, in a sixth implementation manner of the second aspect of the present invention, the analysis module is specifically configured to:
acquiring authorization information, verifying a preset top-speed chain through the authorization information, and if the verification is passed, extracting initial logistics order data from a database corresponding to the top-speed chain;
carrying out security detection, integrity detection and data conversion on the initial logistics order data to obtain to-be-processed logistics order data;
creating multiple threads according to preset level storage conditions, performing parallel judgment and analysis on the logistics order data to be processed through the multiple threads to obtain judgment and analysis results, and determining a target storage level according to the judgment and analysis results.
A third aspect of the present invention provides a processing apparatus for logistics order data, including: a memory and at least one processor, the memory having instructions stored therein; the at least one processor calls the instructions in the memory to enable the processing equipment of the logistics order data to execute the processing method of the logistics order data.
A fourth aspect of the present invention provides a computer-readable storage medium having stored therein instructions, which, when run on a computer, cause the computer to execute the above-described method for processing logistics order data.
According to the technical scheme, logistics order data to be processed are obtained, the logistics order data to be processed are compared with preset level storage conditions and analyzed, and a target storage level is obtained, wherein the level storage conditions comprise level order creation time and service requirements; acquiring a level database corresponding to the target storage level, and storing the logistics order data to be processed into the level database; carrying out life cycle monitoring and grade database updating on the logistics order data in the grade database to obtain data to be inquired; and when a query request is received, retrieving the data to be queried based on a query scene according to the query request to obtain target logistics order data. In the embodiment of the invention, the logistics order data to be processed are subjected to multi-level storage, level database updating and retrieval based on the query scene, so that the consistency and real-time performance of the logistics order data can be maintained, the high-concurrency high-performance query requirement of the logistics order data can be met, the storage capacity of the logistics order data can be enhanced, the storage pressure of each level database is reduced, the query flexibility, the second query rate and the processing speed of the logistics order data are improved, the efficiency and the accuracy of subsequent data query of each level database are improved, high-concurrency high-performance order data service is provided, and the problem that the high-concurrency high-performance order data service cannot be provided is solved.
Drawings
Fig. 1 is a schematic diagram of an embodiment of a method for processing logistics order data according to an embodiment of the invention;
fig. 2 is a schematic diagram of another embodiment of the method for processing logistics order data according to the embodiment of the invention;
fig. 3 is a schematic diagram of an embodiment of a processing device for logistics order data according to the embodiment of the invention;
fig. 4 is a schematic diagram of another embodiment of the processing device of the logistics order data in the embodiment of the invention;
fig. 5 is a schematic diagram of an embodiment of a device for processing logistics order data according to an embodiment of the present invention.
Detailed Description
The embodiment of the invention provides a method, a device, equipment and a storage medium for processing logistics order data, which solve the problem that order data service with high concurrency and high performance cannot be provided.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims, as well as in the drawings, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Furthermore, the terms "comprises," "comprising," or "having," and any variations thereof, are intended to cover non-exclusive inclusions, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
For convenience of understanding, a detailed flow of an embodiment of the present invention is described below, and referring to fig. 1, an embodiment of a method for processing logistics order data according to an embodiment of the present invention includes:
101. and acquiring logistics order data to be processed, and comparing and analyzing the logistics order data to be processed with preset level storage conditions to obtain a target storage level, wherein the level storage conditions comprise level order creation time and service requirements.
It is to be understood that the executing entity of the present invention may be a processing device of logistics order data, and may also be a terminal or a server, which is not limited herein. The embodiment of the present invention is described by taking a server as an execution subject.
The logistics order data can include, but is not limited to, order creation time, freight order number, consignment type and logistics site information, and the logistics site information can include, but is not limited to, geographical information and site type of the logistics site.
The level order creating time level in the level storage condition is order creating time corresponding to each storage level, the order creating time comprises hot time, warm time and cold time, and the hot time, the warm time and the cold time have an inclusion relation; the hot time is a preset hot moment and/or a preset hot period, for example: the hot time is a preset hot time 2021.04.02, or the hot time is a preset hot time period 2021.04.02-2021.06.02; the warm time is a period in which a preset hot time of the hot time (or an initial time of a preset hot period of the hot time) is taken as an initial point, and a first preset time is taken as an interval, and the first preset time is longer than the preset hot period of the hot time, for example: the first preset time is three months, and the temperature time is 2021.04.02-2021.07.02; the cold time is a period in which a preset hot time of the hot time (or an initial time of a preset hot period of the hot time) is taken as an initial point and a second preset time is taken as an interval, and the second preset time is longer than the first preset time, for example: the second preset time is one year, and the cold time is 2021.04.02-2022.04.02.
The service requirements in the level storage condition may include, but are not limited to, order placement service requirements and order scheduling service requirements. The level storage condition includes one or more than one level of storage condition, and the judgment and analysis sequence of the storage conditions of multiple levels in the level storage condition is not limited, that is, the judgment and analysis sequence may be a sequential processing sequence or a parallel processing sequence, and the parallel processing sequence may be implemented by multithreading.
The server acquires the logistics order data to be processed by receiving the logistics order data to be processed sent by each logistics branch or logistics headquarters; judging whether the logistics order data to be processed meet preset level storage conditions or not; if so, determining the corresponding storage level meeting the preset level storage condition as a target storage level; if not, storing the logistics order data to be processed through a preset storage logic, or storing the logistics order data to be processed to a preset storage space.
102. And acquiring a level database corresponding to the target storage level, and storing the logistics order data to be processed to the level database.
The number of target storage levels includes one or more than one, level databases of different target storage levels are different, and storage objects and storage data volumes of the level databases are different, for example: the target storage level comprises a primary storage level, a secondary storage level and a tertiary storage level, the data storage amount of the primary storage level, the secondary storage level and the tertiary storage level is sequentially increased, the life cycle length of the data of the primary storage level, the secondary storage level and the tertiary storage level is sequentially increased, a level database corresponding to the primary storage level is a relational mysql database, namely a primary database, the level database corresponding to the secondary storage is an index storage database, namely a secondary database, the index storage database comprises a full text search engine elastic search database and a class remote dictionary service redis storage system pika database, a hierarchical relationship or an association relationship may exist between the elastic search and pika, the system comprises a level database, a cold time database and a hot time database, wherein the level database is used for storing logistics order data of hot time and/or warm time, and the level database corresponding to the third level storage is a distributed column-type database hbase, namely a third level database, and is used for storing logistics order data of cold time. Each level database comprises a plurality of storage spaces, and the plurality of storage spaces respectively correspond to different regions, types of logistics outlets or logistics importance degrees and the like.
The level database may also be a blockchain-based database, i.e., each level database has a verifiable mechanism, a traceable mechanism, and a security mechanism for blockchains. The level database may also include sub-database clusters, such as: the full text search engine elastic search database comprises an elastic search cluster.
The server searches a preset data source through a corresponding relation between a target storage level and a level database which is created in advance and the target storage level to obtain a level database corresponding to the target storage level, or the server searches a preset database hash table through a hash value generated by the target storage level to obtain level database information, and calls the corresponding level database according to the level database information;
if the level database comprises a primary database, a secondary database and a tertiary database, establishing a partition table for the logistics order data to be processed to obtain a logistics order data partition table, and storing the logistics order data partition table into the primary database; distributing logistics order data stored in the primary database to the secondary database and the tertiary database in a partitioning manner through a preset distributed message system kafka and a message form and a message subscription service; performing index fragmentation processing and key-value pair creation on logistics order data to be processed through a secondary database to obtain index data and key-value pair data, and storing the index data and the key-value pair data; and performing data table configuration and data table creation on the logistics order data to be processed through the third-level database to obtain data table data, and storing the data table data so as to realize the purpose of storing the logistics order data to be processed into the level database.
103. And carrying out life cycle monitoring and level database updating on the logistics order data in the level database to obtain the data to be inquired.
For example, the server obtains a life cycle of the logistics order data in each level of database, where the life cycle is a time interval between the time when the logistics order data is stored in each level of database and the current time, or the life cycle is a comprehensive time between the order creation time of the logistics order data and the storage time in each level of database, that is, the comprehensive time is the order creation time + the storage time; judging whether the life cycle accords with the life cycle in a preset life cycle management and control mechanism, if so, stopping execution; if not, acquiring the inconsistent logistics order data, determining the inconsistent logistics order data as to-be-removed data, removing the to-be-removed data from the currently stored level database, migrating the to-be-removed data to the level database of which the to-be-removed data meets the target storage level so as to update the level database, and determining the logistics order data in the level database after the level database is updated as to-be-inquired data. The storage pressure of each level of database is relieved, and the efficiency and the accuracy of subsequent data query of each level of database are improved.
104. And when the query request is received, retrieving the data to be queried based on the query scene according to the query request to obtain the target logistics order data.
Wherein the query scenario is used to indicate a type of concurrent query. When a server receives a query request triggered by a user interface or a user terminal, analyzing the query request to obtain query key information, wherein the query key information comprises query condition information and a query scene, creating a query priority of a level database based on the query scene and the query condition information, and sequentially retrieving data to be queried in the level database according to the query priority to obtain target logistics order data; or, the server obtains a query policy corresponding to the query scenario, and queries the data to be queried in the level database through the query condition information and the query policy to obtain target logistics order data, where the query policy includes a time attribute query policy, a multidimensional query policy, and/or a key value query policy, and the specific execution process includes: when the query scene is a time attribute query scene, acquiring a time attribute query strategy corresponding to time attribute query, and performing time attribute-based query on data to be queried in the level database based on query condition information through the time attribute query strategy to obtain target logistics order data; when the query scene is a multidimensional query scene, acquiring a multidimensional query strategy corresponding to the multidimensional query scene, creating a multidimensional index of query condition information through the multidimensional query strategy, and retrieving data to be queried in a level database through the multidimensional index to obtain target logistics order data; when the query scene is a key value query scene, a key value query strategy corresponding to the key value query scene is obtained, a target key value of query condition information is established through the key value query strategy, and the data to be queried in the level database is subjected to priority-based level database key value query through the target key value to obtain target logistics order data. The query condition information includes at least one of time, user information, forwarding destination information, logistics processing flow state, and forwarding type. The query scenario is a query type set according to the number of the level databases and the association relationship.
In the embodiment of the invention, the logistics order data to be processed are subjected to multi-level storage, level database updating and retrieval based on the query scene, so that the consistency and real-time performance of the logistics order data can be maintained, the high-concurrency high-performance query requirement of the logistics order data can be met, the storage capacity of the logistics order data can be enhanced, the storage pressure of each level database is reduced, the query flexibility, the second query rate and the processing speed of the logistics order data are improved, the efficiency and the accuracy of subsequent data query of each level database are improved, high-concurrency high-performance order data service is provided, and the problem that the high-concurrency high-performance order data service cannot be provided is solved.
Referring to fig. 2, another embodiment of the method for processing logistics order data according to the embodiment of the invention includes:
201. and acquiring logistics order data to be processed, and comparing and analyzing the logistics order data to be processed with preset level storage conditions to obtain a target storage level, wherein the level storage conditions comprise level order creation time and service requirements.
Specifically, the server acquires authorization information, verifies a preset top-speed chain through the authorization information, and extracts initial logistics order data from a database corresponding to the top-speed chain if the verification is passed; carrying out security detection, integrity detection and data conversion on the initial logistics order data to obtain to-be-processed logistics order data; and creating multiple threads according to preset level storage conditions, performing parallel judgment and analysis on the logistics order data to be processed through the multiple threads to obtain judgment and analysis results, and determining a target storage level according to the judgment and analysis results.
The super-speed chain is a block chain supporting high concurrency and network storage requirements in a 5G environment, and a storage and storage infrastructure in the super-speed chain can provide dynamic, fine-grained and distributed network storage support. The server stores the logistics order data of each logistics network according to each logistics network through the top speed chain, and the server can efficiently acquire the logistics order data stored or sent by the terminal of each logistics network through the top speed chain.
The server acquires authorization information of data access of each logistics network point, verifies a preset top-speed chain through the authorization information, and returns a verification result and stops executing if the verification fails; if the verification is passed, extracting initial logistics order data from a database corresponding to the top speed chain, acquiring data encryption information of each logistics network, performing key pairing on the initial logistics order data of each logistics network through the data encryption information of each logistics network to realize security detection of the initial logistics order data, performing detection based on an abnormal value and a missing value on the initial logistics order data which passes the security detection, and performing data format conversion on the initial logistics order data which passes the detection to obtain to-be-processed logistics order data;
the server establishes multiple threads according to a preset level storage condition, the multiple threads comprise a main thread and multiple sub-threads of one level storage condition, one sub-thread corresponds to one level storage condition judgment logic, parallel judgment and analysis are carried out on logistics order data to be processed through the multiple threads to obtain a judgment and analysis result, if the judgment and analysis result is yes, the corresponding storage level is determined as a target storage level, and if the judgment and analysis result is not, the logistics order data to be processed are stored through the preset storage logic or the logistics order data to be processed are stored into a preset storage space.
By acquiring the initial logistics order data from the top-speed chain, performing security detection, integrity detection and data conversion on the initial logistics order data and performing parallel judgment and analysis on the logistics order data to be processed through multiple threads, the security, the cooperativity and the sharing of the logistics order data to be processed are improved, and the concurrent processing of the logistics order data to be processed is realized.
202. And acquiring a level database corresponding to the target storage level, and storing the logistics order data to be processed to the level database.
Specifically, the server retrieves a preset block chain data source based on a target storage level to obtain a corresponding level database, wherein the level database comprises a primary database, a secondary database and a tertiary database; carrying out encryption based on authorization information on logistics order data to be processed to obtain encrypted order data; partition table creation based on months is carried out on the encrypted order data to obtain processed encrypted order data, and the processed encrypted order data are written into a primary database; acquiring the freight order number and the order master category of the encrypted order data, creating a target data table based on the freight order number and the order master category, and inserting the encrypted order data into a three-level database through the target data table; and performing index fragmentation processing, association processing and secondary database writing on the encrypted order data.
For example, a preset blockchain data source is a data source based on a blockchain, the blockchain can be connected with each logistics branch and a logistics headquarters, a server acquires a level database (the level database comprises a primary database, a secondary database and a tertiary database, the primary database is a relational database, the secondary database is a full-text search engine elastic search database and a similar remote dictionary service redis storage system pika database, and the tertiary database is a distributed column database hbase) corresponding to a target storage level from the preset blockchain data source, acquires authorization information allowing access to the blockchain data source, generates a key according to the authorization information, and encrypts logistics order data to be processed through the key to obtain encrypted order data;
calling a relational database management system mysql corresponding to the relational database, acquiring target order creation time of the encrypted order data, generating a partition key according to the target order creation time, establishing a partition table of the encrypted order data according to months through the partition key to obtain processed encrypted order data, and writing the processed encrypted order data into the relational database (primary database);
the server calls a storage interface of a distributed type column database hbase (a third-level database) to obtain an order main category and an order number of encrypted order data to be processed, wherein the order main category can be one of an article type, a consignment type, a key customer type and the like, the life cycle of the encrypted order data is counted to obtain the life cycle of the order, the number of the order main category is used as the number of data tables, the order life cycle is used as a data table family, the order number is subjected to reversal processing to obtain a reversed order number, the reversed order number is used as a data table family of the encrypted order data, a data table is created for the encrypted order data according to the number of the data tables, the data table family and the data table family to obtain a target data table, and the encrypted order data is stored to the distributed type column database hbase through the target data table;
the server calls a full-text search engine elastic search corresponding to a full-text search engine elastic search database (a secondary database), index fragmentation processing is carried out on encrypted order data to obtain a target index, an associated foreign key between the target index and a target data table in a tertiary database is established, the target index for establishing the associated foreign key is stored in the full-text search engine elastic search database to generate a key value pair of the encrypted order data, and the key value pair is stored in a similar remote dictionary service redis storage system pika database;
or the server calls the full-text search engine elastic search corresponding to the full-text search engine elastic search database, obtains the data size of the encrypted order data, judges whether the data size is larger than a preset threshold value, if so, divides the logistics order data to be processed according to a preset proportion to obtain first order data and second order data, performs index fragmentation processing on the first order data to obtain a target index, establishes an associated foreign key between the target index and a target data table in a third-level database, respectively stores the first order data into the full-text search engine elastic search database, generates a key value pair of the second order data through a similar remote dictionary service redis storage system pika of the similar remote dictionary service redis storage system pika database, stores the key value pair into the similar remote dictionary service redis storage system pika database, and if not, performs index fragmentation processing on the encrypted order data, obtaining a target index, establishing an associated foreign key between the target index and a target data table in a third-level database, and storing the target index with the associated foreign key into a full-text search engine elastic search database;
the writing sequence of the first-level database, the second-level database and the third-level database can be synchronous, or can be performed according to a preset sequence, and is not limited herein, for example: the first-level database can be written in, then the third-level database can be written in, and finally the second-level database can be written in; or firstly writing into the third-level database, then writing into the first-level database, and finally writing into the second-level database; or firstly writing into the third-level database, then writing into the second-level database, and finally writing into the first-level database; and synchronously writing the target index in the second database and the target data table in the third database.
By encrypting and storing the logistics order data to be processed to the block chain-based level database, the consistency and the real-time performance of the logistics order data can be maintained, the high-concurrency high-performance query requirement of the logistics order data can be met, and the storage capacity, the safety and the data access sharing capacity of the logistics order data can be enhanced.
Specifically, the server establishes and processes encrypted order data based on a primary index and a secondary index to obtain a target index; according to the target index, carrying out fragmentation processing on the encrypted order data to obtain order data to be stored; establishing an associated foreign key of a target index and a target data table in a tertiary database, and performing key value-based data structure conversion on order data to be stored to obtain target storage data; and storing the target index and the target storage data for establishing the associated foreign key into a secondary database.
For example, the server calls a full-text search engine elastic search corresponding to a full-text search engine elastic search database (secondary database), creates a primary index of encrypted order data, performs field analysis on the encrypted order data according to preset service requirements to obtain a field to be processed, establishes a secondary index of the field to be processed based on the primary index, performs fragmentation processing on the encrypted order data according to the primary index and the secondary index to obtain order data to be stored, and performs Key/Value structure data conversion on the order data to be stored to obtain target storage data; dividing target storage data according to a preset storage proportion, and respectively storing the divided target storage data into storage spaces corresponding to the storage proportions in a remote dictionary service like remote storage system pika; and establishing a target index and an associated foreign key of a target data table in a third-level database, storing the target index with the associated foreign key into a full-text search engine elastic search database, gradually losing the heat of an index document along with the time, and flexibly deleting logistics order data corresponding to the cold time according to the service time.
203. And carrying out life cycle monitoring and level database updating on the logistics order data in the level database to obtain the data to be inquired.
Specifically, the server performs life cycle detection on logistics order data in a level database based on a preset life cycle management and control mechanism to obtain data to be removed; updating the data of the level database based on migration through the data to be removed to obtain an updated level database; creating a virtual data view and a database mirror image file of the logistics order data in the updated level database; and determining the virtual data view and the database mirror image file as data to be inquired.
The server acquires a life cycle of the logistics order data in each level database, wherein the life cycle is a time interval between the logistics order data stored in each level database and the current time, or the life cycle is a comprehensive time of the order creation time of the logistics order data and the storage time in each level database, namely the comprehensive time is the order creation time plus the storage time; judging whether the life cycle accords with the life cycle in a preset life cycle management and control mechanism, if so, stopping execution; if not, acquiring the inconsistent logistics order data, determining the data as to-be-removed data, removing the to-be-removed data from the currently stored level database, and migrating the to-be-removed data to a target database based on a preset migration mechanism, wherein the target database is a level database of which the to-be-removed data meets the to-be-migrated target storage level, and the steps are as follows: the data to be removed is logistics order data of which the hot time is changed into the cold time along with the passage of time, and the corresponding target database is a level database for storing the logistics order data of the cold time so as to realize data updating based on removal or migration of the level database, wherein the preset migration mechanism comprises a migration management and control mechanism and a migration early warning mechanism, the migration management and control mechanism is used for managing and controlling the migration start and the migration termination of the data to be removed, and the migration early warning mechanism is used for monitoring and managing the migration abnormity of the data to be removed;
the server acquires the global field names and the local field names of the logistics order data in the updated level database through a preset global query mapping rule, establishes the corresponding relation between the global field names and the local field names to obtain virtual data, writes the virtual data into a preset virtual view chart and generates key values to obtain a virtual data view, and renders the virtual data view to a preset display interface;
the method comprises the steps that a server obtains a storage address and a storage directory of logistics order data for creating a virtual data view and key values of virtual data corresponding to the virtual data view, calls a preset mirror image container, creates a mirror image file of the logistics order data based on a preset mirror image function, the storage address, the storage directory and the key values of the virtual data to obtain a database mirror image file, and monitors and controls the database mirror image file through a preset mirror image file management system; and the server determines the virtual data view and the database mirror image file as data to be inquired.
The method has the advantages that the logistics order data in the level databases are used for life cycle monitoring, level database updating, virtual data view and database mirror image file creation, storage pressure of the level databases is relieved, the problems of information island, standard non-uniformity and safety of the level databases are solved, real-time performance and convenience of management of the logistics order data in the level databases are achieved, access pressure of the logistics order data in the level databases is relieved, efficiency and accuracy of follow-up data query of the level databases are improved, and high-concurrency and high-performance order data service is achieved.
204. When a query request triggered based on a virtual data view in data to be queried is received, analyzing the query request to obtain query key information, wherein the query key information comprises query condition information and a query scene, and the data to be queried comprises the virtual data view and a database mirror image file.
When a server receives a query request triggered by a user interface or a display page based on a virtual data view in data to be queried, the query request is analyzed to obtain query Key information, the query Key information comprises query condition information and a query scene, the query condition information comprises at least one of time, user information, delivery destination information, a logistics processing flow state, a delivery type and the like, and the query scene can comprise a time attribute query scene, a multidimensional query scene and a Key Value (Key/Value) query scene. And starting the query of the database mirror image file based on the virtual data view and the query key information. The data to be queried comprises a virtual data view and a database mirror image file.
205. And acquiring a query strategy corresponding to the query scene, and querying the database mirror image file through query condition information and the query strategy to obtain target logistics order data, wherein the query strategy comprises a time attribute query strategy, a multi-dimensional query strategy and/or a key value query strategy.
Specifically, when the query scene is a time attribute query scene, the server acquires a time attribute query strategy corresponding to the time attribute query scene, and performs time attribute-based query on the database image file based on query condition information through the time attribute query strategy to obtain target logistics order data; when the query scene is a multidimensional query scene, the server acquires a multidimensional query strategy corresponding to the multidimensional query scene, creates a multidimensional index of query condition information through the multidimensional query strategy, and retrieves the database mirror image file through the multidimensional index to obtain target logistics order data; when the query scene is a key value query scene, the server acquires a key value query strategy corresponding to the key value query scene, creates a target key value of query condition information through the key value query strategy, and performs priority-based level database key value query on the database mirror image file through the target key value to obtain target logistics order data.
For example, the level database comprises a relational mysql database (primary database), a full text search engine elastic search database, a remote dictionary service like remote storage system pika database (secondary database) and a distributed column database hbase (tertiary database), when the query scene is a time attribute query scene, time attribute parameters hot _ only, war _ only, cold _ only or time range are extracted from the query condition information through a time attribute query strategy, database mirror files corresponding to the primary database and the secondary database respectively are retrieved through the time attribute parameters hot _ only to obtain target logistics order data, or database mirror files corresponding to the secondary database are retrieved through the time attribute parameters war _ only to obtain target logistics order data, or the tertiary database is retrieved through the time attribute parameters cold _ only to obtain target logistics order data, or through the time attribute parameter TimeRange, carrying out multithreading-based parallel query on database mirror image files respectively corresponding to the primary database, the secondary database and the tertiary database to obtain target logistics order data;
when the query scene is a multi-dimensional query scene, calling a full-text search engine elastic search corresponding to a full-text search engine elastic search database through a key value query strategy, performing multi-layer row index creation or multi-layer index object-based index on query condition information based on a preset query priority to obtain a multi-dimensional index, and retrieving a database mirror image file corresponding to the full-text search engine elastic search database through the multi-dimensional index to obtain target logistics order data;
when the query scene is a Key Value query scene, generating Key/Value of query condition information, namely a target Key Value, by a Key Value query strategy, retrieving a database image file corresponding to a remote dictionary service redis storage system pika database based on preset hot time priority and the Key/Value to obtain a query result, if the query result is yes, directly extracting corresponding target logistics order data from the remote dictionary service redis storage system pika database by the database image file, if the query result is not, obtaining a corresponding relation between a query data column of the query condition information and a row Key rowkey by the Key Value query strategy, calling a full-text search engine elasticsearch engine corresponding to the full-text search engine elasticsearch database, creating an index of the corresponding relation, and obtaining a row Key row corresponding to the order data in the full-text search engine elasticsearch database based on a multidimensional query condition (namely query condition information), wherein the row Key row is a row Key row corresponding to the full-text search engine elasticsearch database And in the table, data table query is performed on the database mirror image file corresponding to the distributed columnar database hbase through the row key rowkey list to obtain corresponding target logistics order data, wherein the index based on the row key rowkey is stored in the distributed columnar database hbase, and the corresponding target logistics order data can be efficiently obtained from the database mirror image file corresponding to the distributed columnar database hbase through the row key rowkey list and the index.
In the embodiment of the invention, the logistics order data to be processed are subjected to multi-level storage, level database updating and retrieval based on the query scene, so that the consistency and real-time performance of the logistics order data can be maintained, the high-concurrency high-performance query requirement of the logistics order data can be met, the storage capacity of the logistics order data can be enhanced, the storage pressure of each level database is reduced, the query flexibility, the second query rate and the processing speed of the logistics order data are improved, the efficiency and the accuracy of subsequent data query of each level database are improved, high-concurrency high-performance order data service is provided, and the problem that the high-concurrency high-performance order data service cannot be provided is solved.
With reference to fig. 3, the method for processing logistics order data in the embodiment of the present invention is described above, and a device for processing logistics order data in the embodiment of the present invention is described below, where an embodiment of the device for processing logistics order data in the embodiment of the present invention includes:
the analysis module 301 is configured to obtain logistics order data to be processed, and compare and analyze the logistics order data to be processed with preset level storage conditions to obtain a target storage level, where the level storage conditions include level order creation time and service requirements;
a storage module 302, configured to obtain a level database corresponding to a target storage level, and store logistics order data to be processed in the level database;
the monitoring updating module 303 is configured to perform life cycle monitoring and level database updating on the logistics order data in the level database to obtain data to be queried;
and the query module 304 is configured to, when receiving a query request, perform query-scene-based retrieval on the data to be queried according to the query request to obtain target logistics order data.
The function implementation of each module in the processing apparatus for logistics order data corresponds to each step in the processing method embodiment of logistics order data, and the function and implementation process thereof are not described in detail here.
In the embodiment of the invention, the logistics order data to be processed are subjected to multi-level storage, level database updating and retrieval based on the query scene, so that the consistency and real-time performance of the logistics order data can be maintained, the high-concurrency high-performance query requirement of the logistics order data can be met, the storage capacity of the logistics order data can be enhanced, the storage pressure of each level database is reduced, the query flexibility, the second query rate and the processing speed of the logistics order data are improved, the efficiency and the accuracy of subsequent data query of each level database are improved, high-concurrency high-performance order data service is provided, and the problem that the high-concurrency high-performance order data service cannot be provided is solved.
Referring to fig. 4, another embodiment of the device for processing logistics order data according to the embodiment of the present invention includes:
the analysis module 301 is configured to obtain logistics order data to be processed, and compare and analyze the logistics order data to be processed with preset level storage conditions to obtain a target storage level, where the level storage conditions include level order creation time and service requirements;
a storage module 302, configured to obtain a level database corresponding to a target storage level, and store logistics order data to be processed in the level database;
the monitoring updating module 303 is configured to perform life cycle monitoring and level database updating on the logistics order data in the level database to obtain data to be queried;
the query module 304 is configured to, when receiving a query request, perform query-scene-based retrieval on data to be queried according to the query request to obtain target logistics order data;
the query module 304 specifically includes:
the analyzing unit 3041 is configured to, when receiving a query request triggered based on a virtual data view in data to be queried, analyze the query request to obtain query key information, where the query key information includes query condition information and a query scenario, and the data to be queried includes the virtual data view and a database mirror image file;
the query unit 3042 is configured to obtain a query policy corresponding to a query scenario, and query the database image file through the query condition information and the query policy to obtain target logistics order data, where the query policy includes a time attribute query policy, a multidimensional query policy, and/or a key value query policy.
Optionally, the storage module 302 includes:
a retrieval unit 3021, configured to retrieve a preset block chain data source based on a target storage level to obtain a corresponding level database, where the level database includes a primary database, a secondary database, and a tertiary database;
an encryption unit 3022, configured to encrypt the to-be-processed logistics order data based on the authorization information to obtain encrypted order data;
a creating unit 3023, configured to create a partition table based on months for the encrypted order data, to obtain processed encrypted order data, and write the processed encrypted order data into the primary database;
an inserting unit 3024, configured to obtain the order number and the order owner type of the encrypted order data, create a target data table based on the order number and the order owner type, and insert the encrypted order data into the third-level database through the target data table;
and the writing unit 3025 is configured to perform index fragmentation processing, association processing, and secondary database writing on the encrypted order data.
Optionally, the writing unit 3025 may be further specifically configured to:
establishing and processing the encrypted order data based on a primary index and a secondary index to obtain a target index;
according to the target index, carrying out fragmentation processing on the encrypted order data to obtain order data to be stored;
establishing an associated foreign key of a target index and a target data table in a tertiary database, and performing key value-based data structure conversion on order data to be stored to obtain target storage data;
and storing the target index and the target storage data for establishing the associated foreign key into a secondary database.
Optionally, the monitoring update module 303 may be further specifically configured to:
carrying out life cycle detection on logistics order data in the level database based on a preset life cycle management and control mechanism to obtain data to be removed;
updating the data of the level database based on migration through the data to be removed to obtain an updated level database;
creating a virtual data view and a database mirror image file of the logistics order data in the updated level database;
and determining the virtual data view and the database mirror image file as data to be inquired.
Optionally, the querying unit 3042 may be further specifically configured to:
when the query scene is a time attribute query scene, acquiring a time attribute query strategy corresponding to the time attribute query scene, and performing time attribute-based query on the database image file based on query condition information through the time attribute query strategy to obtain target logistics order data;
when the query scene is a multidimensional query scene, acquiring a multidimensional query strategy corresponding to the multidimensional query scene, creating a multidimensional index of query condition information through the multidimensional query strategy, and retrieving a database mirror image file through the multidimensional index to obtain target logistics order data;
when the query scene is a key value query scene, a key value query strategy corresponding to the key value query scene is obtained, a target key value of query condition information is established through the key value query strategy, and priority-based level database key value query is performed on the database mirror image file through the target key value to obtain target logistics order data.
Optionally, the analysis module 301 may be further specifically configured to:
acquiring authorization information, verifying a preset top-speed chain through the authorization information, and if the verification is passed, extracting initial logistics order data from a database corresponding to the top-speed chain;
carrying out security detection, integrity detection and data conversion on the initial logistics order data to obtain to-be-processed logistics order data;
and creating multiple threads according to preset level storage conditions, performing parallel judgment and analysis on the logistics order data to be processed through the multiple threads to obtain judgment and analysis results, and determining a target storage level according to the judgment and analysis results.
The function implementation of each module and each unit in the processing device of the logistics order data corresponds to each step in the embodiment of the processing method of the logistics order data, and the function and implementation process are not described in detail herein.
In the embodiment of the invention, the logistics order data to be processed are subjected to multi-level storage, level database updating and retrieval based on the query scene, so that the consistency and real-time performance of the logistics order data can be maintained, the high-concurrency high-performance query requirement of the logistics order data can be met, the storage capacity of the logistics order data can be enhanced, the storage pressure of each level database is reduced, the query flexibility, the second query rate and the processing speed of the logistics order data are improved, the efficiency and the accuracy of subsequent data query of each level database are improved, high-concurrency high-performance order data service is provided, and the problem that the high-concurrency high-performance order data service cannot be provided is solved.
Fig. 3 and 4 describe the processing device of the logistics order data in the embodiment of the present invention in detail from the perspective of the modular functional entity, and the processing device of the logistics order data in the embodiment of the present invention is described in detail from the perspective of hardware processing.
Fig. 5 is a schematic structural diagram of a processing device for logistics order data according to an embodiment of the present invention, where the processing device 500 for logistics order data can generate relatively large differences due to different configurations or performances, and may include one or more processors (CPUs) 510 (e.g., one or more processors) and a memory 520, and one or more storage media 530 (e.g., one or more mass storage devices) for storing applications 533 or data 532. Memory 520 and storage media 530 may be, among other things, transient or persistent storage. The program stored in the storage medium 530 may include one or more modules (not shown), each of which may include a series of instruction operations in the processing device 500 for logistics order data. Further, the processor 510 may be configured to communicate with the storage medium 530, and execute a series of instruction operations in the storage medium 530 on the processing device 500 of the logistics order data.
The logistics order data processing apparatus 500 may further include one or more power supplies 540, one or more wired or wireless network interfaces 550, one or more input/output interfaces 560, and/or one or more operating systems 531, such as Windows server, Mac OS X, Unix, Linux, FreeBSD, and the like. Those skilled in the art will appreciate that the configuration of the processing device of the physical distribution order data shown in fig. 5 does not constitute a limitation of the processing device of the physical distribution order data, and may include more or less components than those shown, or combine some components, or arrange different components.
The present application further provides a processing device for logistics order data, including: a memory having instructions stored therein and at least one processor, the memory and the at least one processor interconnected by a line; the at least one processor calls the instructions in the memory to enable the processing equipment of the logistics order data to execute the steps in the processing method of the logistics order data. The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium, and which may also be a volatile computer-readable storage medium, having stored therein instructions, which, when executed on a computer, cause the computer to perform the steps of the method for processing logistics order data.
Further, the computer-readable storage medium may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function, and the like; the storage data area may store data created according to the use of the blockchain node, and the like.
The block chain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
The above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims (10)

1. A processing method of logistics order data is characterized in that the processing method of the logistics order data comprises the following steps:
acquiring logistics order data to be processed, and comparing and analyzing the logistics order data to be processed with preset level storage conditions to obtain a target storage level, wherein the level storage conditions comprise level order creation time and service requirements;
acquiring a level database corresponding to the target storage level, and storing the logistics order data to be processed into the level database;
carrying out life cycle monitoring and grade database updating on the logistics order data in the grade database to obtain data to be inquired;
and when a query request is received, retrieving the data to be queried based on a query scene according to the query request to obtain target logistics order data.
2. The method for processing logistics order data according to claim 1, wherein the obtaining of the level database corresponding to the target storage level and the storing of the logistics order data to be processed into the level database comprises:
retrieving a preset block chain data source based on the target storage level to obtain a corresponding level database, wherein the level database comprises a primary database, a secondary database and a tertiary database;
encrypting the logistics order data to be processed based on authorization information to obtain encrypted order data;
partition table creation based on months is carried out on the encrypted order data to obtain processed encrypted order data, and the processed encrypted order data are written into the primary database;
acquiring the order number and the order main category of the encrypted order data, creating a target data table based on the order number and the order main category, and inserting the encrypted order data into the third-level database through the target data table;
and performing index fragmentation processing, association processing and secondary database writing on the encrypted order data.
3. The method for processing logistics order data according to claim 2, wherein the performing index fragmentation processing, association processing and secondary database writing on the encrypted order data includes:
establishing and processing the encrypted order data based on a primary index and a secondary index to obtain a target index;
according to the target index, carrying out fragmentation processing on the encrypted order data to obtain order data to be stored;
establishing an associated foreign key of the target index and a target data table in the tertiary database, and performing key value-based data structure conversion on the order data to be stored to obtain target storage data;
and storing the target index establishing the associated foreign key and the target storage data into the secondary database.
4. The method for processing logistics order data according to claim 1, wherein the performing lifecycle monitoring and level database updating on the logistics order data in the level database to obtain the data to be queried comprises:
carrying out life cycle detection on the logistics order data in the level database based on a preset life cycle management and control mechanism to obtain data to be removed;
updating the data of the level database based on migration through the data to be removed to obtain an updated level database;
creating a virtual data view and a database mirror image file of the logistics order data in the updated level database;
and determining the virtual data view and the database mirror image file as data to be inquired.
5. The method for processing logistics order data according to claim 1, wherein when receiving a query request, performing query-scenario-based retrieval on the data to be queried according to the query request to obtain target logistics order data, comprises:
when a query request triggered based on a virtual data view in the data to be queried is received, analyzing the query request to obtain query key information, wherein the query key information comprises query condition information and a query scene, and the data to be queried comprises the virtual data view and a database mirror image file;
and acquiring a query strategy corresponding to the query scene, and querying the database mirror image file through the query condition information and the query strategy to obtain target logistics order data, wherein the query strategy comprises a time attribute query strategy, a multidimensional query strategy and/or a key value query strategy.
6. The method for processing logistics order data according to claim 5, wherein the obtaining of the query policy corresponding to the query scenario and the querying of the database image file through the query condition information and the query policy to obtain the target logistics order data comprises:
when the query scene is a time attribute query scene, acquiring a time attribute query strategy corresponding to the time attribute query scene, and performing time attribute-based query on the database mirror image file based on the query condition information through the time attribute query strategy to obtain target logistics order data;
when the query scene is a multidimensional query scene, acquiring a multidimensional query strategy corresponding to the multidimensional query scene, creating a multidimensional index of the query condition information through the multidimensional query strategy, and retrieving the database image file through the multidimensional index to obtain target logistics order data;
and when the query scene is a key value query scene, acquiring a key value query strategy corresponding to the key value query scene, creating a target key value of the query condition information through the key value query strategy, and performing priority-based level database key value query on the database mirror image file through the target key value to obtain target logistics order data.
7. The method for processing logistics order data according to any one of claims 1 to 6, wherein the obtaining of the logistics order data to be processed and the comparison and analysis of the logistics order data to be processed with preset level storage conditions to obtain a target storage level comprises:
acquiring authorization information, verifying a preset top-speed chain through the authorization information, and if the verification is passed, extracting initial logistics order data from a database corresponding to the top-speed chain;
carrying out security detection, integrity detection and data conversion on the initial logistics order data to obtain to-be-processed logistics order data;
creating multiple threads according to preset level storage conditions, performing parallel judgment and analysis on the logistics order data to be processed through the multiple threads to obtain judgment and analysis results, and determining a target storage level according to the judgment and analysis results.
8. A processing apparatus of logistics order data, characterized in that the processing apparatus of logistics order data comprises:
the analysis module is used for acquiring logistics order data to be processed and comparing and analyzing the logistics order data to be processed with preset level storage conditions to obtain a target storage level, wherein the level storage conditions comprise level order creation time and service requirements;
the storage module is used for acquiring a level database corresponding to the target storage level and storing the logistics order data to be processed into the level database;
the monitoring updating module is used for carrying out life cycle monitoring and level database updating on the logistics order data in the level database to obtain data to be inquired;
and the query module is used for retrieving the data to be queried based on a query scene according to the query request when receiving the query request so as to obtain target logistics order data.
9. A processing apparatus of logistics order data, characterized in that the processing apparatus of logistics order data comprises: a memory and at least one processor, the memory having instructions stored therein;
the at least one processor invokes the instructions in the memory to cause the processing device of the logistics order data to execute the method of processing logistics order data of any one of claims 1-7.
10. A computer-readable storage medium having instructions stored thereon, wherein the instructions, when executed by a processor, implement a method for processing logistics order data according to any one of claims 1 to 7.
CN202110526493.2A 2021-05-14 2021-05-14 Logistics order data processing method, device, equipment and storage medium Pending CN113326281A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110526493.2A CN113326281A (en) 2021-05-14 2021-05-14 Logistics order data processing method, device, equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110526493.2A CN113326281A (en) 2021-05-14 2021-05-14 Logistics order data processing method, device, equipment and storage medium

Publications (1)

Publication Number Publication Date
CN113326281A true CN113326281A (en) 2021-08-31

Family

ID=77415603

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110526493.2A Pending CN113326281A (en) 2021-05-14 2021-05-14 Logistics order data processing method, device, equipment and storage medium

Country Status (1)

Country Link
CN (1) CN113326281A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114331584A (en) * 2021-11-12 2022-04-12 中国邮电器材集团有限公司 Order data storage method, device, equipment and storage medium
CN114881743A (en) * 2022-06-02 2022-08-09 广东乔润物联网科技有限公司 Intelligent logistics order management system

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114331584A (en) * 2021-11-12 2022-04-12 中国邮电器材集团有限公司 Order data storage method, device, equipment and storage medium
CN114881743A (en) * 2022-06-02 2022-08-09 广东乔润物联网科技有限公司 Intelligent logistics order management system

Similar Documents

Publication Publication Date Title
US9177025B2 (en) Hash-join in parallel computation environments
US20060041533A1 (en) Encrypted table indexes and searching encrypted tables
EP3251033B1 (en) Hybrid data distribution in a massively parallel processing architecture
CN113326281A (en) Logistics order data processing method, device, equipment and storage medium
TW201415262A (en) Construction of inverted index system, data processing method and device based on Lucene
CN112969996A (en) Tracking intermediate changes in database data
CN104063376A (en) Multi-dimensional grouping operation method and system
US20120290615A1 (en) Switching algorithms during a run time computation
CN104239377A (en) Platform-crossing data retrieval method and device
CN104871153A (en) System and method for flexible distributed massively parallel processing (mpp) database
CN112650759A (en) Data query method and device, computer equipment and storage medium
US9268952B2 (en) Scalable precomputation system for host-opaque processing of encrypted databases
CN112632058A (en) Track determination method, device and equipment and storage medium
CN115729965A (en) Information stream processing method, device, stream server and storage medium
Dong et al. GAT: A unified GPU-accelerated framework for processing batch trajectory queries
Xiao A Spark based computing framework for spatial data
Kondylakis et al. Enabling joins over cassandra NoSQL databases
EP4030312A1 (en) Method and apparatus for querying data, computing device, and storage medium
CN114185898A (en) Data query method and device and terminal equipment
CN116414801A (en) Data migration method, device, computer equipment and storage medium
CN112835873A (en) Power grid regulation and control heterogeneous system service access method, system, equipment and medium
EP2469424B1 (en) Hash-join in parallel computation environments
CN113127717A (en) Key retrieval method and system
Xiao A big spatial data processing framework applying to national geographic conditions monitoring
Micheli et al. Efficient Multi-User Indexing for Secure Keyword Search.

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination