CN113496374A - Data processing method and device - Google Patents
Data processing method and device Download PDFInfo
- Publication number
- CN113496374A CN113496374A CN202010270338.4A CN202010270338A CN113496374A CN 113496374 A CN113496374 A CN 113496374A CN 202010270338 A CN202010270338 A CN 202010270338A CN 113496374 A CN113496374 A CN 113496374A
- Authority
- CN
- China
- Prior art keywords
- data
- flow data
- inventory
- user
- business
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/08—Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
- G06Q10/087—Inventory or stock management, e.g. order filling, procurement or balancing against orders
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/10—Office automation; Time management
Landscapes
- Business, Economics & Management (AREA)
- Engineering & Computer Science (AREA)
- Entrepreneurship & Innovation (AREA)
- Strategic Management (AREA)
- Human Resources & Organizations (AREA)
- Economics (AREA)
- Marketing (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- Physics & Mathematics (AREA)
- General Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Development Economics (AREA)
- Finance (AREA)
- Accounting & Taxation (AREA)
- Data Mining & Analysis (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The embodiment of the disclosure discloses a data processing method and device. One embodiment of the method comprises: respectively acquiring inventory flow data and business flow data of a user to be processed from a warehouse system of a warehouse and a user system of the user who resides in the warehouse in real time; performing data alignment on the inventory flow data and the service flow data in response to the fact that the time stamp of the inventory flow data is matched with the time stamp of the service flow data; comparing the stock flow data and the service flow data after data alignment; and generating difference data between the inventory flow data and the business flow data of the user to be processed based on the comparison result. According to the embodiment, the inventory flow data of the warehouse system and the business flow data of the user system are subjected to data alignment, so that data comparison is more convenient and faster, and the generation efficiency of difference data is improved.
Description
Technical Field
The embodiment of the disclosure relates to the technical field of computers, in particular to a data processing method and device.
Background
With the rapid development of the e-commerce industry, the logistics industry and the like, the demand of warehouse storage is increasing. Therefore, large warehouses are increasing, such as self-built warehouses of large e-commerce platforms. Large warehouses are widely distributed, and thus warehouses may be leased to other users, for example, a large utility platform may lease a self-established warehouse to a merchant who hosts the utility platform. When a user uses the warehouse, the user often needs to extract the inventory flow change data of the items stored in the warehouse by the user from the warehouse regularly. Comparing the extracted stock flow data with the business flow change of the user per se to obtain difference data of the two parties, wherein the difference data can verify whether the goods in the warehouse are normally circulated.
In the related technology, the difference data of the warehouse platform and the user are often obtained by the manual report form comparison mode, and the manual data comparison efficiency is low. Or the warehouse platform and the user can also obtain difference data of the two parties by adopting an off-line data extraction and direct comparison mode, and the data comparison efficiency is low.
Disclosure of Invention
The embodiment of the disclosure provides a data processing method and device.
In a first aspect, an embodiment of the present disclosure provides a data processing method, where the method includes: respectively acquiring inventory flow data and business flow data of a user to be processed from a warehouse system of a warehouse and a user system of the user who resides in the warehouse in real time; performing data alignment on the inventory flow data and the service flow data in response to the fact that the time stamp of the inventory flow data is matched with the time stamp of the service flow data; comparing the stock flow data and the service flow data after data alignment; and generating difference data between the inventory flow data and the business flow data of the user to be processed based on the comparison result.
In some embodiments, data aligning the inventory flow data and the traffic flow data in response to determining that the time stamp of the inventory flow data and the time stamp of the traffic flow data match comprises: determining time nodes of a plurality of data comparison; aiming at a time node in a plurality of time nodes, determining a target time period formed by the time node and an adjacent time node; in response to determining that the obtained time stamp of the inventory flow data and the time stamp of the business flow data are within the target time period, determining that the time stamp of the inventory flow data is matched with the time stamp of the business flow data; and performing data alignment on the acquired inventory flow data and the acquired business flow data.
In some embodiments, comparing the data-aligned inventory flow data to the business flow data includes: determining the sum of the data quantity of the stock flow data and the service flow data after data alignment; and comparing the stock flow data and the service flow data after data alignment by adopting a Hash algorithm and/or a set operation mode of a data set based on the sum of the data quantity, wherein the data set comprises a stock flow data set and a service flow data set, the stock flow data set comprises the stock flow data after data alignment, and the service flow data set comprises the service flow data after data alignment.
In some embodiments, comparing the stock flow data and the business flow data after data alignment by using a hash algorithm and/or a set operation manner of a data set based on the sum of the data volumes includes: in response to the fact that the sum of the data amounts is smaller than a preset threshold value, comparing the stock flow data and the service flow data after data alignment by adopting a Hash algorithm; and in response to the fact that the sum of the data amounts is larger than or equal to the preset threshold value, comparing the stock flow data and the service flow data after data alignment by adopting a data set aggregation operation mode.
In some embodiments, comparing the stock flow data and the business flow data after data alignment by using a set operation mode of a data set includes: according to the time sequence of the time stamps of the inventory flow data and the time stamps of the business flow data, respectively dividing the inventory flow data and the business flow data after data alignment into a plurality of inventory flow data sets and business flow data sets corresponding to the inventory flow data sets; and aiming at the inventory flow data sets in the plurality of inventory flow data sets, carrying out difference set operation and intersection operation on the inventory flow data sets and the corresponding business flow data sets.
In some embodiments, the acquiring inventory flow data and business flow data of the user to be processed from the warehouse system of the warehouse and the user system of the user who is resident in the warehouse in real time respectively comprises: the method comprises the steps that inventory flow data of a plurality of users are obtained from a warehouse system in real time, and business flow data of users to be processed are obtained from a user system in real time; acquiring preset identification information of each user to be processed; and acquiring the inventory flow data of each user to be processed from the acquired inventory flow data of the plurality of users based on the acquired preset identification information.
In some embodiments, generating difference data between the inventory and business pipeline data of the pending user based on the comparison comprises: generating to-be-verified difference data of the to-be-processed user based on the comparison result; acquiring historical difference data of a user to be processed in a preset historical time period; and determining the to-be-verified difference data as the difference data of the to-be-processed user in response to determining that the to-be-verified difference data does not exist in the acquired historical difference data.
In a second aspect, an embodiment of the present disclosure provides a data processing apparatus, including: the acquisition unit is configured to acquire the inventory flow data and the business flow data of the user to be processed in real time from a warehouse system of the warehouse and a user system of the user who resides in the warehouse respectively; the data alignment unit is configured to perform data alignment on the inventory flow data and the business flow data in response to determining that the time stamp of the inventory flow data is matched with the time stamp of the business flow data; the comparison unit is configured to compare the stock flow data and the service flow data after the data alignment; and the generating unit is configured to generate difference data between the inventory flow data and the business flow data of the user to be processed based on the comparison result.
In some embodiments, the data alignment unit is further configured to: determining time nodes of a plurality of data comparison; aiming at a time node in a plurality of time nodes, determining a target time period formed by the time node and an adjacent time node; in response to determining that the obtained time stamp of the inventory flow data and the time stamp of the business flow data are within the target time period, determining that the time stamp of the inventory flow data is matched with the time stamp of the business flow data; and performing data alignment on the acquired inventory flow data and the acquired business flow data.
In some embodiments, the comparison unit comprises: the determining module is configured to determine the sum of the data quantity of the stock flow data and the business flow data after data alignment; and the data comparison module is configured to compare the stock flow data and the service flow data after data alignment by adopting a Hash algorithm and/or a set operation mode of a data set based on the sum of data quantity, wherein the data set comprises a stock flow data set and a service flow data set, the stock flow data set comprises the stock flow data after data alignment, and the service flow data set comprises the service flow data after data alignment.
In some embodiments, the data comparison module is further configured to: in response to the fact that the sum of the data amounts is smaller than a preset threshold value, comparing the stock flow data and the service flow data after data alignment by adopting a Hash algorithm; and in response to the fact that the sum of the data amounts is larger than or equal to the preset threshold value, comparing the stock flow data and the service flow data after data alignment by adopting a data set aggregation operation mode.
In some embodiments, the data comparison module is further configured to: according to the time sequence of the time stamps of the inventory flow data and the time stamps of the business flow data, respectively dividing the inventory flow data and the business flow data after data alignment into a plurality of inventory flow data sets and business flow data sets corresponding to the inventory flow data sets; and aiming at the inventory flow data sets in the plurality of inventory flow data sets, carrying out difference set operation and intersection operation on the inventory flow data sets and the corresponding business flow data sets.
In some embodiments, the obtaining unit is further configured to: the method comprises the steps that inventory flow data of a plurality of users are obtained from a warehouse system in real time, and business flow data of users to be processed are obtained from a user system in real time; acquiring preset identification information of each user to be processed; and acquiring the inventory flow data of each user to be processed from the acquired inventory flow data of the plurality of users based on the acquired preset identification information.
In some embodiments, the generating unit is further configured to: generating to-be-verified difference data of the to-be-processed user based on the comparison result; acquiring historical difference data of a user to be processed in a preset historical time period; and determining the to-be-verified difference data as the difference data of the to-be-processed user in response to determining that the to-be-verified difference data does not exist in the acquired historical difference data.
The data processing method and the data processing device provided by the embodiment of the disclosure can respectively obtain the stock flow data and the business flow data of a user to be processed from a warehouse system of the warehouse and a user system of the user who resides in the warehouse in real time, then in response to the fact that the time stamp of the stock flow data is matched with the time stamp of the business flow data, the stock flow data and the business flow data can be aligned, then the stock flow data and the business flow data after the data alignment are compared, finally based on the comparison result, the difference data between the stock flow data and the business flow data of the user to be processed can be generated, the realization mode aligns the stock flow data and the flow data of the same time period through a time stamp matching mode, thereby automatically comparing the stock flow data and the warehouse flow data after the data alignment, and the comparison between the stock flow data and the warehouse flow data after the data alignment is more convenient, and the data comparison efficiency is improved.
Drawings
Other features, objects and advantages of the disclosure will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:
FIG. 1 is an exemplary system architecture diagram in which some embodiments of the present disclosure may be applied;
FIG. 2 is a flow diagram for one embodiment of a data processing method according to the present disclosure;
FIG. 3 is a flow diagram of yet another embodiment of a data processing method according to the present disclosure;
FIG. 4 is a schematic block diagram of one embodiment of a data processing apparatus according to the present disclosure;
FIG. 5 is a schematic structural diagram of an electronic device suitable for use in implementing embodiments of the present disclosure.
Detailed Description
The present disclosure is described in further detail below with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.
It should be noted that, in the present disclosure, the embodiments and features of the embodiments may be combined with each other without conflict. The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
Fig. 1 shows an exemplary system architecture 100 of a data processing method or data processing apparatus to which embodiments of the present disclosure may be applied.
As shown in fig. 1, the system architecture 100 may include a warehouse system 101, a user system 102, a network 103, and a server 104. The network 103 is the medium used to provide the communication link between the warehouse system 101 and the server 104, and the network 103 is the medium used to provide the communication link between the user system 102 and the server 104. Network 103 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
The warehouse may provide a place for storage of items for users, such as merchants, that reside on the warehouse platform. The warehouse system 101 may be used to manage the circulation of items in the warehouse and generate and store inventory flow data generated by the circulation of items in the warehouse. Specifically, the warehouse system 101 may store various information, such as inventory flow data including article warehousing information, article ex-warehousing information, and inventory information. Likewise, a user residing in a warehouse may store items to the warehouse. A user who is resident in the warehouse may store and manage business flow data generated by the user through the user system 102, such as the number of items stored to the warehouse. It can be understood that the warehouse may also be a self-built warehouse of the e-commerce platform, and at this time, the user may be a merchant who is resident on the e-commerce platform, and the merchant may use the self-built warehouse of the e-commerce platform after being resident on the e-commerce platform.
The server 104 may be a server that provides various services, such as a background server that manages inventory and business flow data in the warehouse system 101 and the user systems 102. The background server may analyze and perform other processing on the inventory flow data acquired from the warehouse system and the business flow data acquired from the user system, and feed back a processing result (difference data between the inventory flow data and the business flow data) to the warehouse system and/or the user system.
It should be noted that the data processing method provided by the embodiment of the present disclosure may be executed by the server 105. Accordingly, the data processing apparatus may be provided in the server 105.
The server may be hardware or software. When the server is hardware, it may be implemented as a distributed server cluster formed by multiple servers, or may be implemented as a single server. When the server is software, it may be implemented as multiple pieces of software or software modules, for example, to provide distributed services, or as a single piece of software or software module. And is not particularly limited herein.
It should be understood that the number of user systems, networks, and servers in fig. 1 is merely illustrative. There may be any number of user systems, networks, and servers, as desired for an implementation.
With continued reference to FIG. 2, a flow 200 of one embodiment of a data processing method according to the present disclosure is shown. The data processing method comprises the following steps:
In this embodiment, an execution main body of the data processing method (for example, a server shown in fig. 1) may obtain the stock flow data of the user to be processed in real time from the warehouse system of the warehouse and obtain the business flow data of the user to be processed in real time from the user system through a wired connection manner or a wireless connection manner. Here, the warehouse platform may include a plurality of users entering the warehouse, and a user system in which each user entering the warehouse is located may count the business flow data generated by itself, for example, when user 1 stores 100 articles a in the warehouse, the system of user 1 may generate the corresponding business data. The inventory flow data in the warehouse system can be inventory data generated by warehouse entering and warehousing, returned goods and order deletion and the like. The user to be processed may be a user to be subjected to comparison between inventory flow data and business flow data among a plurality of users who are resident in the warehouse, and the user to be processed may include one user or a plurality of users, which is not limited uniquely herein. It can be understood that the execution main body acquires the inventory streamline data and the service streamline data in real time, so that the real-time comparison of the inventory streamline data and the service streamline data can be realized, the data synchronization effect is improved, and the data synchronization delay of a data change user and a warehouse is avoided.
Generally, after a user such as a merchant is resident in a warehouse platform, when the user uses the warehouse, the stock flow data of the user in the warehouse needs to be extracted, and the extracted data is compared with the business flow data of the user's own system, so as to verify whether the change data of the stock flow data of the warehouse has a problem. Therefore, in order to ensure that the items stored by the user can be normally circulated and operated in the warehouse, the inventory flow data of the warehouse system and the business flow data of the user system need to be compared to obtain the difference data between the two.
It should be noted that the wireless connection means may include, but is not limited to, a 3G/4G connection, a WiFi connection, a bluetooth connection, a WiMAX connection, a Zigbee connection, a uwb (ultra wideband) connection, and other wireless connection means now known or developed in the future.
In some optional implementation manners of this embodiment, the obtaining, in real time, inventory flow data and business flow data of a user to be processed from a warehouse system of the warehouse and a user system of a user who is resident in the warehouse respectively may include: the method comprises the steps that inventory flow data of a plurality of users are obtained from a warehouse system in real time, and business flow data of users to be processed are obtained from a user system in real time; acquiring preset identification information of each user to be processed; and acquiring the inventory flow data of each user to be processed from the acquired inventory flow data of the plurality of users based on the acquired preset identification information. The implementation mode can preset unique identification information for each user who is resident in the warehouse. Therefore, the execution subject can acquire the inventory flow data of the user to be processed from the warehouse system based on the identification information of the user to be processed. The realization mode can simply and quickly acquire the inventory flow data of the user to be processed, and improves the comparison efficiency of the inventory flow data and the business flow data of the user to be processed.
In some optional implementation manners of this embodiment, there may be a plurality of users to be processed, and at this time, the execution main body may process the inventory flow data and the business flow data of each user to be processed respectively based on the identification information of the user. Specifically, the stock flow data and the service flow data of different users to be processed can be divided into different hosts or servers, so that each host or server can process the stock flow data and the service flow data of each user to be processed respectively, data isolation processing for different users is realized, and the data processing efficiency is further improved.
In some optional implementations of this embodiment, different pieces of sub-identification information may be set for different services of the same user who is hosted in the repository. Therefore, for different services of the same user to be processed, the service flow data and the inventory flow data of the different services of the user to be processed can be respectively subjected to data alignment and other processing based on the sub-identification information. Specifically, the business flow data and the inventory flow data of different businesses of the same user to be processed are respectively processed in different hosts or servers. This implementation further improves the efficiency of data processing.
In this embodiment, based on the inventory flow data and the business flow data of the user to be processed acquired in step 201, the execution main body (for example, the server shown in fig. 1) may determine whether the timestamp of the acquired inventory flow data matches the timestamp of the business flow data. If the timestamps of the acquired inventory flow data and the timestamps of the business flow data match, it can be determined that the acquired inventory flow data and the business flow data are data generated for the same time period. The obtained stock flow data and the business flow data are data which can be compared, and difference data between the stock flow data and the business flow data can be obtained. It can be understood that, if the time stamp of the obtained inventory flow data and the time stamp of the business flow data do not match, it can be determined that the obtained inventory flow data and the business flow data are generated in different time periods, and at this time, the obtained inventory flow data and the business flow data do not have contrast, and the difference data between the two cannot be obtained.
In this embodiment, after determining that the obtained stock flow data and the business flow data are data that can be compared, the execution main body may align the stock flow data and the business flow data in various ways. As an example, the execution agent may convert the data format of the stock pipelining data to the data format of the business pipelining data, such that the stock pipelining data and the business pipelining data are aligned. Or, the execution main body may further convert the data format of the business pipeline data into the data format of the stock pipeline data, so that the stock pipeline data and the business pipeline data are aligned.
In some optional implementation manners of this embodiment, the execution main body may determine a format of data to be converted according to the format of the stock flow data and the format of the business flow data, and convert the stock flow data and the business flow data into the format of the data to be converted, thereby implementing alignment of the stock flow data and the business flow data. The realization mode can improve the accuracy of the converted stock flow data and the converted business flow data.
In some optional implementations of this embodiment, the performing data alignment on the inventory flow data and the business flow data in response to determining that the timestamp of the inventory flow data matches the timestamp of the business flow data may include: determining time nodes of a plurality of data comparison; aiming at a time node in a plurality of time nodes, determining a target time period formed by the time node and an adjacent time node; in response to determining that the obtained time stamp of the inventory flow data and the time stamp of the business flow data are within the target time period, determining that the time stamp of the inventory flow data is matched with the time stamp of the business flow data; and performing data alignment on the acquired inventory flow data and the acquired business flow data. This implementation may determine the difference data versus calibration time by constructing a check clock. Specifically, a time node of a plurality of data comparisons in a preset time period may be determined by checking a clock, and for a time node of the plurality of time nodes, a target time period formed by the time node and an adjacent time node is determined. Here, the time node and the target time period may provide a standard time for data alignment. It should be noted that, before selecting the time node and the target time segment, a reference time may be determined, so as to determine the time node and the target time segment with reference to the reference time, where the stock flow data and the service flow data of the reference time are relatively stable. The implementation mode can provide a standard target time period for data alignment, and further improves the accuracy of the aligned data. It is to be understood that the execution subject may also obtain the standard target time period for data alignment in other manners, and is not limited herein.
And step 203, comparing the stock flow data and the business flow data after data alignment.
In this embodiment, based on the inventory flow data and the business flow data after data alignment obtained in step 202, the execution main body may compare the inventory flow data and the business flow data after data alignment in various ways. By way of example, the inventory and business pipeline data may be compared in a direct data-by-data match.
And step 204, generating difference data between the inventory flow data and the business flow data of the user to be processed based on the comparison result.
In this embodiment, based on the comparison result obtained in step 203, the executing entity may perform data verification, integration, and the like on the obtained comparison result, so as to generate difference data between the inventory flow data and the business flow data of the user to be processed. It is understood that the execution body may also send the acquired difference data to a terminal device in a warehouse system, a user system, or the like, so that each person can acquire the difference data between the inventory flow data and the business flow data in time.
In some optional implementation manners of this embodiment, the generating difference data between the inventory flow data and the business flow data of the user to be processed based on the comparison result may include: generating to-be-verified difference data of the to-be-processed user based on the comparison result; acquiring historical difference data of a user to be processed in a preset historical time period; and determining the to-be-verified difference data as the difference data of the to-be-processed user in response to determining that the to-be-verified difference data does not exist in the acquired historical difference data. The implementation mode can avoid repeated difference data, and further improves the accuracy of the determined difference data.
The data processing method provided by the above embodiment of the present disclosure may obtain the stock flow data and the business flow data of the user to be processed in real time from the warehouse system of the warehouse and the user system of the user who resides in the warehouse, then, in response to determining that the timestamp of the stock flow data matches the timestamp of the business flow data, may perform data alignment on the stock flow data and the business flow data, then, compare the stock flow data and the business flow data after the data alignment, and finally, based on the comparison result, may generate difference data between the stock flow data and the business flow data of the user to be processed, and this implementation aligns the stock flow data and the flow data of the same time period by means of matching the timestamps, and may implement automatic comparison of the stock flow data and the warehouse flow data after the data alignment, and the comparison between the stock flow data and the warehouse flow data after the data alignment is more convenient, and the data comparison efficiency is improved.
With further reference to FIG. 3, a flow 300 of yet another embodiment of a data processing method is shown. The flow 300 of the data processing method includes the following steps:
In this embodiment, step 301 and step 302 are respectively the same as or similar to the specific implementation of step 201 and step 202 in the above embodiment, and are not described again here.
And step 303, determining the sum of the data volume of the stock flow data and the service flow data after data alignment.
In this embodiment, based on the data alignment result of the inventory flow data and the business flow data obtained in step 302, the execution main body may count the data amount of the inventory flow data and the business flow data after data alignment, so as to obtain the sum of the data amount of the inventory flow data and the business flow data after data alignment.
And 304, comparing the stock flow data and the service flow data after data alignment by adopting a Hash algorithm and/or a data set aggregation operation mode based on the sum of the data quantity.
In this embodiment, based on the sum of the data amounts obtained in step 303, the execution main body may process the stock flow data and the service flow data after data alignment in different manners, so as to compare the stock flow data and the service flow data after data alignment. The execution main body may process the stock flow data and the service flow data after data alignment by using a hash algorithm, or the execution main body may process the stock flow data and the service flow data after data alignment by using a collective operation method of a data set, or the execution main body may process the stock flow data and the service flow data after data alignment by using a combination method of a hash algorithm and a collective operation of a data set. It should be noted that the data set includes an inventory flow data set and a service flow data set, the inventory flow data set includes inventory flow data after data alignment, and the service flow data set includes service flow data after data alignment. The set operation of the data sets may be a set operation between an inventory flow data set and a business flow data set.
In some optional implementation manners of this embodiment, the execution main body may determine whether a sum of data amounts of the stock pipelining data and the business pipelining data after data alignment is smaller than a preset threshold. If the sum of the data amount is smaller than the preset threshold value, the execution main body can process the stock flow data and the service flow data after data alignment by adopting a hash algorithm. In this implementation manner, for the stock flow data and the service flow data with small data amount and aligned data, the execution main body calculates the stock flow data and the service flow data with aligned data by using a hash algorithm, so that the efficiency of data comparison can be further improved.
By way of example, if the data-aligned inventory flow data and business flow data are processed by a hash algorithm, and the hash calculation results of the inventory flow data and the business flow data are compared to be the same, it may be determined that there is no difference between the inventory flow data and the business flow data.
It can be understood that if the sum of the data amounts is greater than or equal to the preset threshold, the stock flow data and the business flow data after data alignment are processed in a set operation mode of a data set. For the stock flow data and the business flow data with large data volume and aligned data, the execution main body may perform intersection and difference operation on the stock flow data set including the stock flow data and the business flow data set including the business flow data. The difference set operation of the data sets can determine that only the data in the stock flow data set or the service flow data set exists, and the intersection operation of the data sets can find out the data which exists in both the stock flow data set and the service flow data set but has difference in target fields of the data. The data alignment result is processed by adopting a data set operation mode, so that the memory loss can be reduced, and the time consumption of data processing can be reduced.
In some optional implementation manners of this embodiment, when the sum of the data amounts is greater than or equal to a preset threshold, the inventory pipelining data and the business pipelining data after data alignment may be divided into a plurality of corresponding inventory pipelining data sets and business pipelining data sets, respectively. Specifically, the execution main body may divide the inventory flow data and the service flow data after data alignment into a plurality of inventory flow data sets and service flow data sets corresponding to the inventory flow data sets according to a time sequence of the time stamp of the inventory flow data and the time stamp of the service flow data. For example, the execution body may divide the data-aligned stock flow data and the service flow data into a plurality of stock flow data sets and service flow data sets corresponding to the stock flow data sets according to the target time period generated by the time node. It is to be understood that, the execution main body may also divide the inventory flow data and the business flow data after data alignment into a plurality of inventory flow data sets and business flow data sets corresponding to the inventory flow data sets, respectively, according to other time period division manners, which is not limited herein.
Further, for an inventory flow data set of the plurality of inventory flow data sets, the execution main body may perform difference set operation and intersection operation on the inventory flow data set and a corresponding business flow data set, so as to compare the inventory flow data and the business flow data after data alignment.
And 305, generating difference data between the inventory flow data and the business flow data of the user to be processed based on the comparison result.
In this embodiment, based on the comparison result obtained in step 304, the executing entity may perform data verification, integration, and the like on the obtained comparison result, so as to generate difference data between the inventory flow data and the business flow data of the user to be processed. It is understood that the execution body may also send the acquired difference data to a terminal device in a warehouse system, a user system, or the like, so that each person can acquire the difference data between the inventory flow data and the business flow data in time.
In some optional implementation manners of this embodiment, when the execution main body generates difference data by using the data processing method of the present disclosure, the execution main body may further collect a result of each data comparison, perform statistical analysis on the collected data, and record influences of factors such as data amount and time node selection in the data processing process on the data processing efficiency and accuracy. Further, the execution main body can reasonably distribute the computing resources according to the statistical analysis result.
As can be seen from fig. 3, compared with the embodiment corresponding to fig. 2, the process 300 of the data processing method in this embodiment may select to process the stock flow data and the service flow data after data alignment by using a hash algorithm and/or a set operation manner of a data set based on the sum of the data amounts of the stock flow data and the service flow data after data alignment, and then compare the processing results to obtain corresponding difference data. Therefore, the scheme described in this embodiment can process the stock flow data and the business flow data after data alignment by adopting different data processing modes according to the data volume, so that data comparison can be efficiently and accurately completed.
With further reference to fig. 4, as an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a data processing apparatus, which corresponds to the method embodiment shown in fig. 2, and which is particularly applicable to various electronic devices.
As shown in fig. 4, the data processing apparatus 400 of the present embodiment includes: an acquisition unit 401, a data alignment unit 402, a comparison and selection unit 403, and a generation unit 404. The obtaining unit 401 is configured to obtain, in real time, inventory flow data and business flow data of a user to be processed from a warehouse system of a warehouse and a user system of a user who is stationed in the warehouse, respectively; the data alignment unit 402 is configured to data align the inventory flow data and the business flow data in response to determining that the time stamp of the inventory flow data and the time stamp of the business flow data match; the comparing unit 403 is configured to compare the inventory pipelining data and the business pipelining data after data alignment; the generation unit 404 is configured to generate difference data between the stock flow data and the business flow data of the user to be processed based on the comparison result.
In some optional implementations of the present embodiment, the data alignment unit 402 is further configured to: determining time nodes of a plurality of data comparison; aiming at a time node in a plurality of time nodes, determining a target time period formed by the time node and an adjacent time node; in response to determining that the obtained time stamp of the inventory flow data and the time stamp of the business flow data are within the target time period, determining that the time stamp of the inventory flow data is matched with the time stamp of the business flow data; and performing data alignment on the acquired inventory flow data and the acquired business flow data.
In some optional implementations of this embodiment, the comparing unit 403 includes: the determining module is configured to determine the sum of the data quantity of the stock flow data and the business flow data after data alignment; and the data comparison module is configured to compare the stock flow data and the service flow data after data alignment by adopting a Hash algorithm and/or a set operation mode of a data set based on the sum of data quantity, wherein the data set comprises a stock flow data set and a service flow data set, the stock flow data set comprises the stock flow data after data alignment, and the service flow data set comprises the service flow data after data alignment.
In some optional implementations of this embodiment, the data comparison module is further configured to: in response to the fact that the sum of the data amounts is smaller than a preset threshold value, comparing the stock flow data and the service flow data after data alignment by adopting a Hash algorithm; and in response to the fact that the sum of the data amounts is larger than or equal to the preset threshold value, comparing the stock flow data and the service flow data after data alignment by adopting a data set aggregation operation mode.
In some optional implementations of this embodiment, the data comparison module is further configured to: according to the time sequence of the time stamps of the inventory flow data and the time stamps of the business flow data, respectively dividing the inventory flow data and the business flow data after data alignment into a plurality of inventory flow data sets and business flow data sets corresponding to the inventory flow data sets; and aiming at the inventory flow data sets in the plurality of inventory flow data sets, carrying out difference set operation and intersection operation on the inventory flow data sets and the corresponding business flow data sets.
In some optional implementations of this embodiment, the obtaining unit 401 is further configured to: the method comprises the steps that inventory flow data of a plurality of users are obtained from a warehouse system in real time, and business flow data of users to be processed are obtained from a user system in real time; acquiring preset identification information of each user to be processed; and acquiring the inventory flow data of each user to be processed from the acquired inventory flow data of the plurality of users based on the acquired preset identification information.
In some optional implementations of this embodiment, the generating unit 404 is further configured to: generating to-be-verified difference data of the to-be-processed user based on the comparison result; acquiring historical difference data of a user to be processed in a preset historical time period; and determining the to-be-verified difference data as the difference data of the to-be-processed user in response to determining that the to-be-verified difference data does not exist in the acquired historical difference data.
The units recited in the apparatus 400 correspond to the various steps in the method described with reference to fig. 2. Thus, the operations and features described above for the method are equally applicable to the apparatus 400 and the units included therein, and are not described in detail here.
Referring now to FIG. 5, a schematic diagram of an electronic device (e.g., the server of FIG. 1) 500 suitable for use in implementing embodiments of the present disclosure is shown. The server shown in fig. 5 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.
As shown in fig. 5, electronic device 500 may include a processing means (e.g., central processing unit, graphics processor, etc.) 501 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)502 or a program loaded from a storage means 508 into a Random Access Memory (RAM) 503. In the RAM503, various programs and data necessary for the operation of the electronic apparatus 500 are also stored. The processing device 501, the ROM 502, and the RAM503 are connected to each other through a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.
Generally, the following devices may be connected to the I/O interface 505: input devices 506 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 507 including, for example, a Liquid Crystal Display (LCD), speakers, vibrators, and the like; storage devices 508 including, for example, magnetic tape, hard disk, etc.; and a communication device 509. The communication means 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. While fig. 5 illustrates an electronic device 500 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided. Each block shown in fig. 5 may represent one device or may represent multiple devices as desired.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication means 509, or installed from the storage means 508, or installed from the ROM 502. The computer program, when executed by the processing device 501, performs the above-described functions defined in the methods of embodiments of the present disclosure. It should be noted that the computer readable medium described in the embodiments of the present disclosure may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In embodiments of the disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In embodiments of the present disclosure, however, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: respectively acquiring inventory flow data and business flow data of a user to be processed from a warehouse system of a warehouse and a user system of the user who resides in the warehouse in real time; performing data alignment on the inventory flow data and the service flow data in response to the fact that the time stamp of the inventory flow data is matched with the time stamp of the service flow data; comparing the stock flow data and the service flow data after data alignment; and generating difference data between the inventory flow data and the business flow data of the user to be processed based on the comparison result.
Computer program code for carrying out operations for embodiments of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present disclosure may be implemented by software or hardware. The described units may also be provided in a processor, and may be described as: a processor includes an acquisition unit, a data alignment unit, a comparison unit, and a generation unit. The names of these units do not in some cases constitute a limitation on the units themselves, and for example, the acquiring unit may also be described as a "unit that acquires the stock flow data and the business flow data of the user to be processed in real time from the warehouse system of the warehouse and the user system of the user who is stationed in the warehouse, respectively".
The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention in the embodiments of the present disclosure is not limited to the specific combination of the above-mentioned features, but also encompasses other embodiments in which any combination of the above-mentioned features or their equivalents is made without departing from the inventive concept as defined above. For example, the above features and (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure are mutually replaced to form the technical solution.
Claims (10)
1. A method of data processing, comprising:
respectively acquiring inventory flow data and business flow data of a user to be processed from a warehouse system of a warehouse and a user system of the user who resides in the warehouse in real time;
performing data alignment on the inventory flow data and the business flow data in response to determining that the time stamp of the inventory flow data is matched with the time stamp of the business flow data;
comparing the stock flow data and the service flow data after data alignment;
and generating difference data between the inventory flow data and the business flow data of the user to be processed based on the comparison result.
2. The method of claim 1, wherein the data aligning the inventory and traffic flow data in response to determining that the time stamps of the inventory and traffic flow data match comprises:
determining time nodes of a plurality of data comparison;
determining a target time period formed by a time node and an adjacent time node aiming at the time node in the plurality of time nodes;
in response to determining that the obtained time stamp of the inventory flow data and the time stamp of the business flow data are within the target time period, determining that the time stamps of the inventory flow data and the time stamps of the business flow data match;
and performing data alignment on the acquired inventory flow data and the acquired business flow data.
3. The method of claim 1, wherein said comparing said data-aligned inventory and business pipeline data comprises:
determining the sum of the data quantity of the stock flow data and the service flow data after data alignment;
and comparing the stock flow data and the service flow data after data alignment by adopting a Hash algorithm and/or a set operation mode of a data set based on the sum of the data volumes, wherein the data set comprises a stock flow data set and a service flow data set, the stock flow data set comprises the stock flow data after data alignment, and the service flow data set comprises the service flow data after data alignment.
4. The method according to claim 3, wherein comparing the stock flow data and the traffic flow data after data alignment by using a hash algorithm and/or a set operation of a data set based on the sum of the data volumes comprises:
in response to the fact that the sum of the data amounts is smaller than a preset threshold value, comparing the stock flow data and the service flow data after data alignment by adopting a Hash algorithm;
and in response to the fact that the sum of the data amounts is larger than or equal to the preset threshold value, comparing the stock flow data and the service flow data after data alignment by adopting a data set aggregation operation mode.
5. The method of claim 4, wherein comparing the stock flow data and the business flow data after data alignment by using a set operation of a data set comprises:
according to the time sequence of the time stamps of the inventory flow data and the time stamps of the business flow data, dividing the inventory flow data and the business flow data after data alignment into a plurality of inventory flow data sets and business flow data sets corresponding to the inventory flow data sets respectively;
and aiming at the inventory flow data sets in the plurality of inventory flow data sets, carrying out difference set operation and intersection operation on the inventory flow data sets and the corresponding business flow data sets.
6. The method of claim 1, wherein said obtaining inventory flow data and business flow data of a user to be processed in real time from a warehouse system of a warehouse and a user system of a user hosting the warehouse, respectively, comprises:
acquiring inventory flow data of a plurality of users from the warehouse system in real time, and acquiring business flow data of the users to be processed from the user system in real time;
acquiring preset identification information of each user to be processed;
and acquiring the inventory flow data of the users to be processed from the acquired inventory flow data of the users based on the acquired preset identification information.
7. The method of claim 1, wherein the generating difference data between the inventory and traffic flow data for the pending user based on the comparison comprises:
generating to-be-verified difference data of the to-be-processed user based on the comparison result;
acquiring historical difference data of the user to be processed in a preset historical time period;
and in response to determining that the to-be-verified difference data does not exist in the acquired historical difference data, determining the to-be-verified difference data as the difference data of the to-be-processed user.
8. A data processing apparatus comprising:
the system comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is configured to acquire inventory flow data and business flow data of a user to be processed in real time from a warehouse system of a warehouse and a user system of the user who resides in the warehouse respectively;
a data alignment unit configured to perform data alignment on the inventory flow data and the business flow data in response to determining that the time stamp of the inventory flow data and the time stamp of the business flow data match;
the comparison unit is configured to compare the stock flow data and the service flow data after data alignment;
a generating unit configured to generate difference data between the inventory flow data and the business flow data of the user to be processed based on a comparison result.
9. An electronic device, comprising:
one or more processors;
a storage device having one or more programs stored thereon,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-7.
10. A computer-readable medium, on which a computer program is stored, wherein the program, when executed by a processor, implements the method of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010270338.4A CN113496374A (en) | 2020-04-08 | 2020-04-08 | Data processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010270338.4A CN113496374A (en) | 2020-04-08 | 2020-04-08 | Data processing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113496374A true CN113496374A (en) | 2021-10-12 |
Family
ID=77994754
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010270338.4A Pending CN113496374A (en) | 2020-04-08 | 2020-04-08 | Data processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113496374A (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102480702A (en) * | 2010-11-24 | 2012-05-30 | 腾讯科技（深圳）有限公司 | Short message intercepting method and system |
| US20160247108A1 (en) * | 2015-02-20 | 2016-08-25 | Oracle International Corporation | Inventory-based warehouse allocation for retail items |
| CN106447246A (en) * | 2015-08-06 | 2017-02-22 | 阿里巴巴集团控股有限公司 | Inventory data account checking method and inventory data account checking device |
| CN107862567A (en) * | 2017-10-18 | 2018-03-30 | 上海瀚之友信息技术服务有限公司 | A kind of order reconciliation method |
| CN108133352A (en) * | 2018-02-09 | 2018-06-08 | 东莞嘉泰钟表有限公司 | A kind of practical method made an inventory with account inventory variance in quick processing warehouse |
| CN108960691A (en) * | 2017-05-18 | 2018-12-07 | 北京京东尚科信息技术有限公司 | The method and apparatus of determination article inventory for server system |
| CN109949940A (en) * | 2019-03-11 | 2019-06-28 | 金力 | A kind of big data digging system based on diabetes data library |
| CN110322192A (en) * | 2019-05-30 | 2019-10-11 | 苏宁云计算有限公司 | Account checking method and system for the physical holding of stock of electric business platform and logic inventory |
-
2020
- 2020-04-08 CN CN202010270338.4A patent/CN113496374A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102480702A (en) * | 2010-11-24 | 2012-05-30 | 腾讯科技（深圳）有限公司 | Short message intercepting method and system |
| US20160247108A1 (en) * | 2015-02-20 | 2016-08-25 | Oracle International Corporation | Inventory-based warehouse allocation for retail items |
| CN106447246A (en) * | 2015-08-06 | 2017-02-22 | 阿里巴巴集团控股有限公司 | Inventory data account checking method and inventory data account checking device |
| CN108960691A (en) * | 2017-05-18 | 2018-12-07 | 北京京东尚科信息技术有限公司 | The method and apparatus of determination article inventory for server system |
| CN107862567A (en) * | 2017-10-18 | 2018-03-30 | 上海瀚之友信息技术服务有限公司 | A kind of order reconciliation method |
| CN108133352A (en) * | 2018-02-09 | 2018-06-08 | 东莞嘉泰钟表有限公司 | A kind of practical method made an inventory with account inventory variance in quick processing warehouse |
| CN109949940A (en) * | 2019-03-11 | 2019-06-28 | 金力 | A kind of big data digging system based on diabetes data library |
| CN110322192A (en) * | 2019-05-30 | 2019-10-11 | 苏宁云计算有限公司 | Account checking method and system for the physical holding of stock of electric business platform and logic inventory |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107944481B (en) | Method and apparatus for generating information | |
| CN109766127B (en) | Method for updating application version information | |
| CN111427971B (en) | Business modeling method, device, system and medium for computer system | |
| CN108810047B (en) | Method and device for determining information push accuracy rate and server | |
| CN111177433B (en) | Method and apparatus for parallel processing of information | |
| CN110737655B (en) | Method and device for reporting data | |
| CN111324786A (en) | Method and device for processing consultation problem information | |
| CN112445866A (en) | Data processing method and device, computer readable medium and electronic equipment | |
| CN109933508B (en) | Method and apparatus for transmitting information | |
| CN111160410A (en) | Object detection method and device | |
| CN112561606B (en) | Shelf placement method and device based on user behaviors, electronic equipment and medium | |
| CN110928594A (en) | Service development method and platform | |
| CN107291923B (en) | Information processing method and device | |
| CN112598337A (en) | Article-oriented vehicle control method, apparatus, device and computer readable medium | |
| CN111324470B (en) | Method and device for generating information | |
| CN111385150A (en) | Method and device for acquiring information | |
| CN111488386A (en) | Data query method and device | |
| CN110730109A (en) | Method and apparatus for generating information | |
| CN112346882A (en) | Method and apparatus for transmitting information | |
| CN113496374A (en) | Data processing method and device | |
| CN115330540A (en) | Method and device for processing transaction data | |
| CN112507676B (en) | Method and device for generating energy report, electronic equipment and computer readable medium | |
| CN110084298B (en) | Method and device for detecting image similarity | |
| CN114428815A (en) | Data storage method and device, electronic equipment and computer readable medium | |
| CN113722315A (en) | Data generation method and device, electronic equipment and computer readable medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |