CN111309722A - Data processing method and device - Google Patents
Data processing method and device Download PDFInfo
- Publication number
- CN111309722A CN111309722A CN201811509959.2A CN201811509959A CN111309722A CN 111309722 A CN111309722 A CN 111309722A CN 201811509959 A CN201811509959 A CN 201811509959A CN 111309722 A CN111309722 A CN 111309722A
- Authority
- CN
- China
- Prior art keywords
- dimension
- data
- service
- calling
- dimensions
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 14
- 238000004519 manufacturing process Methods 0.000 claims abstract description 84
- 238000012545 processing Methods 0.000 claims abstract description 53
- 238000000034 method Methods 0.000 claims abstract description 36
- 238000004458 analytical method Methods 0.000 claims abstract description 22
- 230000008569 process Effects 0.000 claims description 19
- 238000003860 storage Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 9
- 238000010606 normalization Methods 0.000 abstract description 4
- 230000009286 beneficial effect Effects 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 13
- 238000011144 upstream manufacturing Methods 0.000 description 9
- 238000004891 communication Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 238000013075 data extraction Methods 0.000 description 3
- 238000007726 management method Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 235000003140 Panax quinquefolius Nutrition 0.000 description 1
- 240000005373 Panax quinquefolius Species 0.000 description 1
- 238000009825 accumulation Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000001427 coherent effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000002354 daily effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000004141 dimensional analysis Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000002035 prolonged effect Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0633—Workflow analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/08—Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
- G06Q10/087—Inventory or stock management, e.g. order filling, procurement or balancing against orders
Landscapes
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Engineering & Computer Science (AREA)
- Economics (AREA)
- Entrepreneurship & Innovation (AREA)
- Strategic Management (AREA)
- Quality & Reliability (AREA)
- Theoretical Computer Science (AREA)
- Marketing (AREA)
- Operations Research (AREA)
- Development Economics (AREA)
- General Physics & Mathematics (AREA)
- Tourism & Hospitality (AREA)
- Physics & Mathematics (AREA)
- General Business, Economics & Management (AREA)
- Accounting & Taxation (AREA)
- Finance (AREA)
- Educational Administration (AREA)
- Game Theory and Decision Science (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a data processing method and device, and relates to the technical field of computers. One embodiment of the method comprises: acquiring production data, and analyzing the data dimension of the production data; the data dimension is the smallest independent unit with detachable production data; combining the analyzed data dimensions according to a preset combination mode to obtain service dimensions; and determining dimension data corresponding to the data dimension in the production data to generate business data corresponding to the business dimension. The implementation method utilizes the processing rule of the service to automatically complete the analysis and processing of the related data, and can seamlessly cut into each data processing system; the whole system not only reduces the resource space occupancy rate of each service link and saves the time cost for processing data, but also is beneficial to the unification and normalization of service messages and shortens the waiting time of service parties for service requirements.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a data processing method and apparatus.
Background
In the current data processing process, various messages are generally required to be accessed according to business requirements. However, the data sources are distributed in various links of the production system, and are not uniform and coherent, and the timeliness and reusability of the messages become factors which are not negligible in the data processing process and are all factors to be considered in the business analysis.
Taking warehouse logistics as an example, the data flow is performed on the upstream and downstream of the production system by taking the freight note as the center, and each service line needs to extract relevant data from the mass data to perform analysis, allocation, accounting and the like.
However, each production link has business differences, and the dimensions of data to be processed and displayed are different, but more or less, the data extraction requirements are crossed. For example, orders and waybills generated every day basically go through the processes of sorting, transporting, distributing and the like.
In the process of implementing the invention, the inventor finds that the prior art has at least the following problems:
1) in the existing data processing, processing logic needs to be independently written aiming at each business link, and an independent data storage space is applied, so that resource waste is caused;
2) data under the same data source needs to be repeatedly processed into dimensions required by different service parties, so that the operation period of the whole service is prolonged; and the interaction times with the upstream service data are more, the repeated access of the data exists, more resources are occupied, and the operation cost of an enterprise is increased.
Disclosure of Invention
In view of this, embodiments of the present invention provide a data processing method and apparatus, which can at least solve the problem in the prior art that different business links have different dimensional requirements on the same data, and processing logic needs to be separately developed, resulting in low timeliness and reusability of data.
To achieve the above object, according to an aspect of an embodiment of the present invention, there is provided a data processing method including: acquiring production data, and analyzing the data dimension of the production data; wherein the data dimension is a smallest independent unit from which the production data can be split; combining the analyzed data dimensions according to a preset combination mode to obtain service dimensions; and determining dimension data corresponding to the data dimension in the production data to generate business data corresponding to the business dimension.
Optionally, the analyzing the data dimension of the production data includes: and determining the production type of the production data, and taking the production dimension of the production type as the data dimension of the production data.
Optionally, the combining the analyzed data dimensions according to a predetermined combination manner to obtain the service dimensions includes:
combining the analyzed data dimensions pairwise to obtain a first-level service dimension;
determining a first remaining data dimension after the primary service dimension is removed, and combining the primary service dimension and the first remaining data dimension to obtain a secondary service dimension;
determining a second remaining data dimension after the second service dimension is removed, and combining the second service dimension and the second remaining data dimension to obtain a third-level service dimension;
the above process is repeated until the number of remaining data dimensions is zero.
Optionally, after determining the dimension data corresponding to the data dimension in the production data to generate the business data corresponding to the business dimension, the method further includes: acquiring a calling dimension in a data calling request, determining a service dimension corresponding to the calling dimension according to a data dimension in the calling dimension, extracting service data corresponding to the determined service dimension, and outputting the service data.
Optionally, the determining, according to the data dimension in the calling dimension, a service dimension corresponding to the calling dimension includes: and determining corresponding service dimension levels according to the number of the data dimensions in the calling dimensions, and extracting the service dimensions corresponding to the data dimensions in the calling dimensions from the determined service dimension levels.
Optionally, the data dimension has a dimension identifier;
the combining the analyzed data dimensions according to a predetermined combination mode to obtain service dimensions, further comprising: generating a service identifier of the service dimension according to the dimension identifier of the data dimension in the service dimension;
the determining, according to the data dimension in the calling dimension, a service dimension corresponding to the calling dimension, further includes: and generating a corresponding calling identifier according to the dimension identifier of the data dimension in the calling dimension, determining a service identifier corresponding to the calling identifier, and taking the service dimension corresponding to the determined service identifier as the service dimension corresponding to the calling dimension.
Optionally, the method further includes: and acquiring the calling dimension in the data calling request, extracting corresponding dimension data according to the data dimension in the calling dimension, and performing combined output.
To achieve the above object, according to another aspect of the embodiments of the present invention, there is provided a data processing apparatus including:
the data dimension analysis module is used for acquiring production data and analyzing the data dimension of the production data; wherein the data dimension is a smallest independent unit from which the production data can be split;
the service dimension generation module is used for combining the analyzed data dimensions according to a preset combination mode to obtain service dimensions;
and the business data generation module is used for determining dimension data corresponding to the data dimension in the production data so as to generate business data corresponding to the business dimension.
Optionally, the data dimension analysis module is configured to: and determining the production type of the production data, and taking the production dimension of the production type as the data dimension of the production data.
Optionally, the service dimension generating module is configured to:
combining the analyzed data dimensions pairwise to obtain a first-level service dimension;
determining a first remaining data dimension after the primary service dimension is removed, and combining the primary service dimension and the first remaining data dimension to obtain a secondary service dimension;
determining a second remaining data dimension after the second service dimension is removed, and combining the second service dimension and the second remaining data dimension to obtain a third-level service dimension;
the above process is repeated until the number of remaining data dimensions is zero.
Optionally, the system further includes a data calling module, configured to: obtaining a calling dimension in a data calling request, determining a service dimension corresponding to the calling dimension according to the data dimension in the calling dimension, extracting service data corresponding to the determined service dimension, and outputting the service data.
Optionally, the data calling module is configured to: and determining corresponding service dimension levels according to the number of the data dimensions in the calling dimensions, and extracting the service dimensions corresponding to the data dimensions in the calling dimensions from the determined service dimension levels.
Optionally, the data dimension has a dimension identifier;
the service dimension generation module is further configured to: generating a service identifier of the service dimension according to the dimension identifier of the data dimension in the service dimension;
the data calling module is further configured to: and generating a corresponding calling identifier according to the dimension identifier of the data dimension in the calling dimension, determining a service identifier corresponding to the calling identifier, and taking the service dimension corresponding to the determined service identifier as the service dimension corresponding to the calling dimension.
Optionally, the apparatus further includes a data combining module, configured to: and acquiring the calling dimension in the data calling request, extracting corresponding dimension data according to the data dimension in the calling dimension, and performing combined output.
To achieve the above object, according to still another aspect of embodiments of the present invention, there is provided a data processing electronic device.
The electronic device of the embodiment of the invention comprises: one or more processors; a storage device, configured to store one or more programs, which when executed by the one or more processors, cause the one or more processors to implement any of the data processing methods described above.
To achieve the above object, according to still another aspect of embodiments of the present invention, there is provided a computer-readable medium on which a computer program is stored, the program implementing any of the data processing methods described above when executed by a processor.
According to the scheme provided by the invention, one embodiment of the invention has the following advantages or beneficial effects: by utilizing the processing rule of the service, the analysis and processing of the related data can be automatically completed, and the seamless cut-in can be realized into each data processing system; the whole system not only reduces the resource space occupancy rate of each service link and saves the time cost for processing data, but also is beneficial to the unification and normalization of service messages and shortens the waiting time of service parties for service requirements.
Further effects of the above-mentioned non-conventional alternatives will be described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a schematic main flow chart of a data processing method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a data creation process provided by an embodiment of the invention;
FIG. 3 is a schematic diagram of a data processing flow provided by an embodiment of the present invention;
FIG. 4 is a schematic diagram of a specific data creation process provided by an embodiment of the present invention;
FIG. 5 is a schematic diagram of a process for searching data according to data dimensions according to an embodiment of the present invention;
fig. 6 is a schematic view of an overall service flow provided by an embodiment of the present invention;
FIG. 7 is a schematic diagram of the main blocks of a data processing apparatus according to an embodiment of the present invention;
FIG. 8 is an exemplary system architecture diagram in which embodiments of the present invention may be employed;
FIG. 9 is a schematic block diagram of a computer system suitable for use with a mobile device or server implementing an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, in which various details of embodiments of the invention are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
It should be noted that the embodiment of the present invention is applicable to the field of real-time big data business processing, and the present invention is mainly described by taking a "waybill" in warehouse logistics as an example.
Referring to fig. 1, a main flowchart of a data processing method according to an embodiment of the present invention is shown, which includes the following steps:
s101: acquiring production data, and analyzing the data dimension of the production data; the data dimension is the smallest independent unit with detachable production data;
s102: combining the analyzed data dimensions according to a preset combination mode to obtain service dimensions;
s103: and determining dimension data corresponding to the data dimension in the production data to generate business data corresponding to the business dimension.
The invention divides the data processing process into three stages: data modeling, data processing, and data creation. And finishing data development by taking the three phases as main lines and taking an actual business mode as a reference.
In the above embodiment, for step S101, corresponding to the data modeling stage, see fig. 2 specifically, the purpose is to determine the data dimension of the production data. The upstream production data is data directly produced by the upstream system, such as waybill data.
Data modeling is the beginning of modularity, aiming to base subsequent dimensional analysis of production data. For individual production data, it can be split into multiple independent pieces of information from the data dimension.
Wherein the data angle is related to the data source, and the message data of the same type is usually in the same pattern. The data dimension is an independent parameter, which can be understood as data of a minimum unit, and can be obtained by data angle division. For example, the waybill information basically includes the waybill number, the package number, the weight, the price, the volume, various times ("creation time", "sort time", "ex-warehouse time"), and the like, and these waybill number, package number, weight, price, volume, time, and the like are data dimensions.
In addition, the angles/dimensions considered by different business parties and different business links are different. Taking merchant information and transportation information as an example, considering how many commodities are sold and the unit price of each commodity from the perspective of a merchant; from the perspective of the transportation side, how many transport vehicles, how many trips each vehicle can run, how many containers can be loaded in each time, etc. are considered.
Therefore, for different traffic patterns, what the minimum unit data to be extracted is needs to be considered separately, and the corresponding configuration will be different.
For step S102, it corresponds to a data processing stage, which is the core of business processing, and is specifically shown in fig. 3.
After the data modeling is completed, the business dimension needs to be processed subsequently according to the actual concerned angles of different business parties on the basis of the determined data dimension.
The business dimension here is not fixed and is usually associated with the needs of the business party. Taking the waybill as an example:
①, extracting from the perspective of the merchant, and combining the two data dimensions of the waybill number and the commodity price to determine the sales volume of the merchant;
②, from the transportation perspective, data dimensions of license plate number, departure place and destination are extracted and combined to determine purchasing power of different areas;
③ it can be changed into household registration data, which can be used for ranking men and women in different provinces, and the population age distribution in different cities.
The business dimensions are ranked, and are specifically determined according to the number of data dimensions to be combined, for example, a level business dimension includes two data dimensions. The whole service dimension and the data dimension form a tree structure, and the multi-layer property of the big data is reflected.
For example, license plate number, departure place, destination, transported waybill and the like are extracted from the transportation data, and waybill number, package number, weight, volume, price and the like are extracted from waybill information, and all the extracted data are dimension data;
① a data dimension of license plate number (data dimension 1), waybill (data dimension 2), price (data dimension 3), destination (data dimension 4);
②, two data dimensions, namely, counting the number of waybills transported by a certain vehicle (business dimension 11), the total price of the waybills transported by a certain vehicle (business dimension 12), the number of vehicles transported to the same destination (business dimension 13), the number of waybills transported by the same destination (business dimension 14), the total price of the waybills transported by the same destination (business dimension 15) and the like;
③, determining the total price of the freight bill transported by the vehicles with the same destination according to the service dimensions 12 and 13 (service dimension 21);
……
⑩ determine the purchasing power of each region and make a ranking (business dimension N1).
For the combination of data dimensions, it is not limited to two-by-two combination, but it is intended to be arbitrary (many). Similar to the idea of artificial intelligence, the program analyzes the service dimension which is possibly needed to be provided according to the dimension of the basic data, the basic data can be combined into a first-level service dimension at will, and then the first-level service dimension or the first-level and the basic are combined into a second-level service dimension, or the basic + the first-level and the second-level service dimensions are combined into a third-level service dimension in any combination mode. The specific implementation mode can be set by workers.
For step S103, corresponding to the data creation phase, it is mainly applied to the subsequent data call. The business dimension depends on business data support, and the business data can be regarded as statistics of production data, for example, the business dimension 11, the shipment volume of the vehicle 1 is 100, and the shipment volume of the vehicle 2 is 200.
The acquired service data can be assembled to generate a data set or stored in a database, so that subsequent calling is facilitated.
In the previous steps, the service data required by the service is basically processed, and then, corresponding data can be directly extracted from the generated service data according to the calling requirement without combining dimension data, which is specifically shown in fig. 4.
Furthermore, in order to improve the accuracy of extracting dimension data, an identifier can be established for the data dimension in the data creation stage.
1) First is the identification of production data. The data identifier is an abstract representation of production data that distinguishes between different upstream production data, each data having a data identifier.
The data identification primarily contains the source of production of the data, the meaning expressed, the dimensions of the data contained, and the like. The method is mainly divided into two parts:
① the classification of data in the production processing system is fixed, such as merchant information and transportation information, and the corresponding labels can be set as "merchant" and "transportation", or as "merchant code" (such as biz-vz00121659) and "transportation code" (such as trans-Beijing NA1126 or trans-mu525) created when the data is actually produced if not fixed.
Note that, although the order number may be used as the data identifier, the order number is mainly embodied in the logistics system. The data identification should not be limited to order numbers, as the present invention is not limited to applicable logistics systems.
②, the data generated by the business is not fixed, for example, the waybill is labeled "Y", the merchant is labeled "S", it can also be determined according to the data dimension in the production data, for example, the production data contains time, destination, price, etc., the generated label is ssj.
And combining the two parts of data into a data identifier which can be recognized by a processing system for extracting data in the subsequent step.
2) Data dimension — dimension data, can be distinguished from other dimension data using the data identification of the corresponding production data. Or the data identification of the production data and the dimension identification of the data dimension are combined to obtain the unique data identification.
For the subsequent received data call request, the identification of the dimensional data to be extracted can be determined according to the data dimensions contained in the data call request, and the data extraction precision is further improved.
3) The service identifiers can be combined by the identifiers of the data dimensions contained in the service identifiers, or by character strings required for extracting parts from the service identifiers, so as to produce identifiers independent of other service dimensions.
And subsequent data calling, namely performing operations such as service dimension identification and service data extraction according to the identification.
According to the calling dimension in the data request, extracting dimension data related to the calling dimension from the determined production data and combining the dimension data, which is specifically shown in fig. 5.
For example, the waybill 1-destination Beijing-price is 1000 yuan (or other currency units), and the waybill 2-destination Beijing-price is 500 yuan. If the required query destination is the total price of the waybill in Beijing and the included data dimensions are 'destination' and 'price', then 1000+500 is directly extracted as 1500 yuan.
But as the angle at which the data is considered by the business party changes, the determined data dimensions change. Thus, in some business scenarios, the needs of the business party may not be satisfied at one time from the data modeling.
For example, the number of packages with a price value of over 1000 yuan (or other currency units) in the commodities sold by the merchant on a certain day is determined, wherein the operation place is Beijing, Shenyang (transfer place) -Heilongjiang. The data dimension includes date, operation place and price, but if the dimensions including volume, weight and the like are considered, the original data dimension is not, the dimensions of volume, weight and the like need to be added on the basis of the original data dimension, data splitting is carried out on the original production data again, and the process is repeated until the required business data is obtained.
Or, taking an intersection of the acquired data, taking transit (beijing-sheng yang (transit place) -black longjiang) as an example, the package is sent from beijing to black longjiang, but needs to be transited in sheng yang. And (3) taking intersection of the acquired business data Beijing-Shenyang and Shenyang-Heilongjiang, which is the data combination.
Therefore, it is necessary to model a plurality of data or to perform a re-processing combination on the processed business data until the business requirement is satisfied. And as time increases and data is accumulated, data in each industry is gradually comprehensive, and the requirement for subsequent service expansion becomes comprehensive.
The design of the invention is not embedded code writing, but independent of the data provider, relying on middleware to provide data, such as MQ, binlog, etc. Therefore, the message processing system can be seamlessly switched in, and the message processing system has a cross-platform property (java language itself, and the introduction of middleware can embody the cross-platform).
In addition, the messages are layered according to the services, modular processing is provided for basic data, and then the service data are flexibly combined according to service requirements, so that the configuration is flexible, modification and maintenance of the accessed messages are facilitated, modularization, uniqueness and uniformity are realized, and the unified management standard of a company is facilitated.
The method provided by the embodiment of the invention provides a concept of unified message processing and data processing, utilizes the processing rule of the service to automatically complete the analysis and processing of the related data, and can be seamlessly switched into each data processing system. Therefore, the resource space occupancy rate of each service link is reduced, the time cost for processing data is saved, the unification and normalization of service messages are facilitated, and the waiting time of a service party for the data is shortened.
For the overall business process of the present invention, see fig. 6 for illustration:
1) upstream production data, data generated by an upstream system. The upstream traffic collision data is included in the upstream production data, and is directly extractable without requiring complicated processing. This part of the data
2) The interception processor may be understood as a screening means, such as AOP, or other implementation. The purpose of interception is usually to add an identifier to the production data, for example, data 1-viewable by the user, data 2-viewable by insiders, and data 3-viewable by the merchant.
3) A basic logic processing module for storing basic information of data, such as vehicle information (license plate number, departure place, destination, and freight notes transported), freight note information (note number, parcel number, weight, volume, and price);
4) the preset analysis module extracts data dimensions such as single numbers, prices and the like from the basic logic processing module, performs combined statistics according to multiple types to obtain service dimensions and corresponding service data, and the calculation angle is gradually increased along with the change of the service or the accumulation of time; for example, daily sales volume and sales amount of the merchant are calculated.
Referring specifically to fig. 1 and 4, it can be seen that the main process of the overall invention is completed in the preset analysis module.
5) The business analysis module is mainly used for storing business data required by a business party, for example, the business data are sorted according to order quantity and sorted according to regional hotspots. The service data in the preset analysis module can be combined according to the personalized requirements of the service party; for example, how many cars, freight volume, total value, etc. are run in a certain warehouse or a certain region on a certain day, and the hot spot information of the region is obtained by sorting the data according to the individual volume or the value.
It should be noted that not all data need to be subjected to all processing links or modules in fig. 5, most data (data dimensions, which are included in the preset analysis module) may be directly extracted from the preset analysis module, and some requirements (data dimensions, which are not included in the preset analysis module) need to be implemented by the extended service analysis module.
Referring to fig. 7, a schematic diagram of main modules of a data processing apparatus 700 according to an embodiment of the present invention is shown, including:
a data dimension analysis module 701, configured to obtain production data and analyze a data dimension of the production data; wherein the data dimension is a smallest independent unit from which the production data can be split;
a business dimension generating module 702, configured to combine the analyzed data dimensions according to a predetermined combination manner to obtain a business dimension;
a business data generating module 703, configured to determine dimension data corresponding to a data dimension in the production data, so as to generate business data corresponding to the business dimension.
In the implementation apparatus of the present invention, the data dimension analysis module 701 is configured to: and determining the production type of the production data, and taking the production dimension of the production type as the data dimension of the production data.
In the device for implementing the present invention, the service dimension generating module 702 is configured to:
combining the analyzed data dimensions pairwise to obtain a first-level service dimension;
determining a first remaining data dimension after the primary service dimension is removed, and combining the primary service dimension and the first remaining data dimension to obtain a secondary service dimension;
determining a second remaining data dimension after the second service dimension is removed, and combining the second service dimension and the second remaining data dimension to obtain a third-level service dimension;
the above process is repeated until the number of remaining data dimensions is zero.
The device for implementing the present invention further includes a data calling module 704 (not shown in the figure), configured to: acquiring a calling dimension in a data calling request, determining a service dimension corresponding to the calling dimension according to a data dimension in the calling dimension, extracting service data corresponding to the determined service dimension, and outputting the service data.
In the device for implementing the present invention, the data call module 704 is configured to: and determining corresponding service dimension levels according to the number of the data dimensions in the calling dimensions, and extracting the service dimensions corresponding to the data dimensions in the calling dimensions from the determined service dimension levels.
In the implementation device of the invention, the data dimension has a dimension identifier;
the business dimension generating module 702 is further configured to: generating a service identifier of the service dimension according to the dimension identifier of the data dimension in the service dimension;
the data calling module 704 is further configured to: and generating a corresponding calling identifier according to the dimension identifier of the data dimension in the calling dimension, determining a service identifier corresponding to the calling identifier, and taking the service dimension corresponding to the determined service identifier as the service dimension corresponding to the calling dimension.
The apparatus further comprises a data combining module 705 (not shown) for: and acquiring the calling dimension in the data calling request, extracting corresponding dimension data according to the data dimension in the calling dimension, and performing combined output.
In addition, the specific implementation of the data processing apparatus in the embodiment of the present invention has been described in detail in the above data processing method, and therefore, the repeated description is not repeated here.
Fig. 8 shows an exemplary system architecture 800 of a data processing method or data processing apparatus to which embodiments of the present invention may be applied.
As shown in fig. 8, the system architecture 800 may include terminal devices 801, 802, 803, a network 804, and a server 805 (by way of example only). The network 804 serves to provide a medium for communication links between the terminal devices 801, 802, 803 and the server 805. Network 804 may include various types of connections, such as wire, wireless communication links, or fiber optic cables, to name a few.
A user may use the terminal devices 801, 802, 803 to interact with a server 805 over a network 804 to receive or send messages or the like. Various communication client applications may be installed on the terminal devices 801, 802, 803.
The terminal devices 801, 802, 803 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.
The server 805 may be a server that provides various services, such as a back-office management server (for example only) that supports shopping-like websites browsed by users using the terminal devices 801, 802, 803. The background management server can analyze and process the received data such as the product information inquiry request and feed back the processing result to the terminal equipment.
It should be noted that the data processing method provided by the embodiment of the present invention is generally executed by the server 805, and accordingly, the data processing apparatus is generally disposed in the server 805.
It should be understood that the number of terminal devices, networks, and servers in fig. 8 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 9, shown is a block diagram of a computer system 900 suitable for use with a terminal device implementing an embodiment of the present invention. The terminal device shown in fig. 9 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.
As shown in fig. 9, the computer system 900 includes a Central Processing Unit (CPU)901 that can perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)902 or a program loaded from a storage section 908 into a Random Access Memory (RAM) 903. In the RAM 903, various programs and data necessary for the operation of the system 900 are also stored. The CPU 901, ROM 902, and RAM 903 are connected to each other via a bus 904. An input/output (I/O) interface 905 is also connected to bus 904.
The following components are connected to the I/O interface 905: an input portion 906 including a keyboard, a mouse, and the like; an output section 907 including components such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 908 including a hard disk and the like; and a communication section 909 including a network interface card such as a LAN card, a modem, or the like. The communication section 909 performs communication processing via a network such as the internet. The drive 910 is also connected to the I/O interface 905 as necessary. A removable medium 911 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 910 as necessary, so that a computer program read out therefrom is mounted into the storage section 908 as necessary.
In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 909, and/or installed from the removable medium 911. The above-described functions defined in the system of the present invention are executed when the computer program is executed by a Central Processing Unit (CPU) 901.
It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present invention may be implemented by software or hardware. The described modules may also be provided in a processor, which may be described as: a processor includes a data dimension analysis module, a business dimension generation module, and a business data generation module. Where the names of these modules do not in some cases constitute a limitation on the modules themselves, for example, a data dimension analysis module may also be described as a "module that analyzes data dimensions in production data".
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to comprise:
acquiring production data, and analyzing the data dimension of the production data; wherein the data dimension is a smallest independent unit from which the production data can be split; combining the analyzed data dimensions according to a preset combination mode to obtain service dimensions; and determining dimension data corresponding to the data dimension in the production data to generate business data corresponding to the business dimension.
According to the technical scheme of the embodiment of the invention, a unified message processing and data processing thought is provided, the analysis and processing of related data are automatically completed by utilizing the processing rule of the business, and the method can be seamlessly switched into each data processing system. Therefore, the resource space occupancy rate of each service link is reduced, the time cost for processing data is saved, the unification and normalization of service messages are facilitated, and the waiting time of a service party for the data is shortened.
The above-described embodiments should not be construed as limiting the scope of the invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (16)
1. A data processing method, comprising:
acquiring production data, and analyzing the data dimension of the production data; wherein the data dimension is a smallest independent unit from which the production data can be split;
combining the analyzed data dimensions according to a preset combination mode to obtain service dimensions;
and determining dimension data corresponding to the data dimension in the production data to generate business data corresponding to the business dimension.
2. The method of claim 1, wherein the analyzing the data dimension of the production data comprises:
and determining the production type of the production data, and taking the production dimension of the production type as the data dimension of the production data.
3. The method of claim 1, wherein the combining the analyzed data dimensions according to a predetermined combination to obtain a business dimension comprises:
combining the analyzed data dimensions pairwise to obtain a first-level service dimension;
determining a first remaining data dimension after the primary service dimension is removed, and combining the primary service dimension and the first remaining data dimension to obtain a secondary service dimension;
determining a second remaining data dimension after the second service dimension is removed, and combining the second service dimension and the second remaining data dimension to obtain a third-level service dimension;
the above process is repeated until the number of remaining data dimensions is zero.
4. The method of claim 1, further comprising, after the determining dimension data corresponding to a data dimension in the production data to generate business data corresponding to the business dimension:
acquiring a calling dimension in a data calling request, determining a service dimension corresponding to the calling dimension according to a data dimension in the calling dimension, extracting service data corresponding to the determined service dimension, and outputting the service data.
5. The method of claim 4, wherein the determining a business dimension corresponding to the calling dimension according to a data dimension of the calling dimension comprises:
and determining corresponding service dimension levels according to the number of the data dimensions in the calling dimensions, and extracting the service dimensions corresponding to the data dimensions in the calling dimensions from the determined service dimension levels.
6. The method of claim 1 or 4, wherein the data dimension has a dimension identification;
the combining the analyzed data dimensions according to a predetermined combination mode to obtain service dimensions, further comprising:
generating a service identifier of the service dimension according to the dimension identifier of the data dimension in the service dimension;
the determining, according to the data dimension in the calling dimension, a service dimension corresponding to the calling dimension, further includes:
and generating a corresponding calling identifier according to the dimension identifier of the data dimension in the calling dimension, determining a service identifier corresponding to the calling identifier, and taking the service dimension corresponding to the determined service identifier as the service dimension corresponding to the calling dimension.
7. The method of claim 1, further comprising:
and acquiring the calling dimension in the data calling request, extracting corresponding dimension data according to the data dimension in the calling dimension, and performing combined output.
8. A data processing apparatus, comprising:
the data dimension analysis module is used for acquiring production data and analyzing the data dimension of the production data; wherein the data dimension is a smallest independent unit from which the production data can be split;
the service dimension generation module is used for combining the analyzed data dimensions according to a preset combination mode to obtain service dimensions;
and the business data generation module is used for determining dimension data corresponding to the data dimension in the production data so as to generate business data corresponding to the business dimension.
9. The apparatus of claim 8, wherein the data dimension analysis module is configured to:
and determining the production type of the production data, and taking the production dimension of the production type as the data dimension of the production data.
10. The apparatus of claim 8, wherein the business dimension generation module is configured to:
combining the analyzed data dimensions pairwise to obtain a first-level service dimension;
determining a first remaining data dimension after the primary service dimension is removed, and combining the primary service dimension and the first remaining data dimension to obtain a secondary service dimension;
determining a second remaining data dimension after the second service dimension is removed, and combining the second service dimension and the second remaining data dimension to obtain a third-level service dimension;
the above process is repeated until the number of remaining data dimensions is zero.
11. The apparatus of claim 8, further comprising a data call module to:
acquiring a calling dimension in a data calling request, determining a service dimension corresponding to the calling dimension according to a data dimension in the calling dimension, extracting service data corresponding to the determined service dimension, and outputting the service data.
12. The apparatus of claim 11, wherein the data call module is configured to:
and determining corresponding service dimension levels according to the number of the data dimensions in the calling dimensions, and extracting the service dimensions corresponding to the data dimensions in the calling dimensions from the determined service dimension levels.
13. The apparatus of claim 8 or 11, wherein the data dimension has a dimension identification;
the service dimension generation module is further configured to:
generating a service identifier of the service dimension according to the dimension identifier of the data dimension in the service dimension;
the data calling module is further configured to:
and generating a corresponding calling identifier according to the dimension identifier of the data dimension in the calling dimension, determining a service identifier corresponding to the calling identifier, and taking the service dimension corresponding to the determined service identifier as the service dimension corresponding to the calling dimension.
14. The apparatus of claim 8, further comprising a data combining module to:
and acquiring the calling dimension in the data calling request, extracting corresponding dimension data according to the data dimension in the calling dimension, and performing combined output.
15. An electronic device, comprising:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-7.
16. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811509959.2A CN111309722A (en) | 2018-12-11 | 2018-12-11 | Data processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811509959.2A CN111309722A (en) | 2018-12-11 | 2018-12-11 | Data processing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111309722A true CN111309722A (en) | 2020-06-19 |
Family
ID=71156096
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811509959.2A Pending CN111309722A (en) | 2018-12-11 | 2018-12-11 | Data processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111309722A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113034111A (en) * | 2021-03-31 | 2021-06-25 | 北京金山云网络技术有限公司 | Data export and import method and device |
| CN113986519A (en) * | 2021-12-29 | 2022-01-28 | 深圳市毕美科技有限公司 | Data scheduling processing method, device, equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7653742B1 (en) * | 2004-09-28 | 2010-01-26 | Entrust, Inc. | Defining and detecting network application business activities |
| CN107846435A (en) * | 2016-09-20 | 2018-03-27 | 中国移动通信有限公司研究院 | Business realizing system and method |
| CN108108426A (en) * | 2017-12-15 | 2018-06-01 | 杭州网蛙科技有限公司 | Understanding method, device and the electronic equipment that natural language is putd question to |
| CN108595179A (en) * | 2018-05-10 | 2018-09-28 | 北京小度信息科技有限公司 | Task creating method, device, electronic equipment and computer readable storage medium |
-
2018
- 2018-12-11 CN CN201811509959.2A patent/CN111309722A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7653742B1 (en) * | 2004-09-28 | 2010-01-26 | Entrust, Inc. | Defining and detecting network application business activities |
| CN107846435A (en) * | 2016-09-20 | 2018-03-27 | 中国移动通信有限公司研究院 | Business realizing system and method |
| CN108108426A (en) * | 2017-12-15 | 2018-06-01 | 杭州网蛙科技有限公司 | Understanding method, device and the electronic equipment that natural language is putd question to |
| CN108595179A (en) * | 2018-05-10 | 2018-09-28 | 北京小度信息科技有限公司 | Task creating method, device, electronic equipment and computer readable storage medium |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113034111A (en) * | 2021-03-31 | 2021-06-25 | 北京金山云网络技术有限公司 | Data export and import method and device |
| CN113986519A (en) * | 2021-12-29 | 2022-01-28 | 深圳市毕美科技有限公司 | Data scheduling processing method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106296059B (en) | Method and equipment for determining delivery network points | |
| CN110472207A (en) | List generation method and device | |
| Chang et al. | Innovative Bike‐Sharing in China: Solving Faulty Bike‐Sharing Recycling Problem | |
| CN110555640A (en) | Method and device for route planning | |
| CN107679103B (en) | Attribute analysis method and system for entity | |
| CN110798567A (en) | Short message classification display method and device, storage medium and electronic equipment | |
| CN113259144A (en) | Storage network planning method and device | |
| CN109831501A (en) | Information dispensing method and system based on block chain | |
| CN110309142A (en) | The method and apparatus of regulation management | |
| CN110858347A (en) | Method and device for logistics distribution and order distribution | |
| CN114579087A (en) | Method and device for designing power customer service middlings | |
| CN112819263A (en) | Method and device for dispatching customer service | |
| CN111309722A (en) | Data processing method and device | |
| CN113392093A (en) | Service processing method and device, electronic equipment and computer readable medium | |
| CN113538028A (en) | Advertisement putting method and device | |
| CN113326095A (en) | Commission data processing method and device | |
| CN113762674A (en) | Order distribution method and device | |
| CN115099865A (en) | Data processing method and device | |
| CN110321399A (en) | The method and apparatus for selecting address | |
| CN115526587A (en) | Method and device for generating customs message | |
| KR102169328B1 (en) | Open manufacturing service system for factory-free manufacturing | |
| Yuan | Intelligent logistics management application relying on the internet of things | |
| CN112988746A (en) | Transaction record partitioning method and device | |
| CN113393190A (en) | Storage information processing method and device, electronic equipment and readable medium | |
| CN111127077A (en) | Recommendation method and device based on stream computing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |