CN115391386A - Data processing method, device, equipment and storage medium - Google Patents

Data processing method, device, equipment and storage medium Download PDF

Info

Publication number
CN115391386A
CN115391386A CN202211116509.3A CN202211116509A CN115391386A CN 115391386 A CN115391386 A CN 115391386A CN 202211116509 A CN202211116509 A CN 202211116509A CN 115391386 A CN115391386 A CN 115391386A
Authority
CN
China
Prior art keywords
data
dimensions
hierarchy
dimension
dimensionality
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211116509.3A
Other languages
Chinese (zh)
Inventor
牛煜超
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ping An Bank Co Ltd
Original Assignee
Ping An Bank Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Bank Co Ltd filed Critical Ping An Bank Co Ltd
Priority to CN202211116509.3A priority Critical patent/CN115391386A/en
Publication of CN115391386A publication Critical patent/CN115391386A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2453—Query optimisation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21—Design, administration or maintenance of databases
    • G06F16/211—Schema design and management
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28—Databases characterised by their database models, e.g. relational or object models
    • G06F16/283—Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The embodiment of the invention discloses a data processing method, a data processing device, data processing equipment and a storage medium. The method comprises the following steps: processing each data through a preset data processing rule to construct a database; acquiring preset dimensions, and sequencing the dimensions, wherein each dimension comprises one or more levels; associating the dimensions with each data in the database, wherein each data corresponds to a hierarchy of dimensions; acquiring a data structure to be tested, wherein the data structure to be tested comprises one or more data; determining the dimensionality corresponding to the data, and searching whether the dimensionality of the hierarchy lacking the corresponding data exists in the dimensionality according to a preset hierarchy sequence; if so, completing the dimension of the hierarchy with the missing data. The method associates the data with the dimensionality, can find out the dimensionality of the hierarchy lacking the data according to the association relation, and completes the dimensionality data of all the hierarchies, thereby effectively reducing the workload of developers, and being convenient to maintain, clear to manage, simple and easy to use.

Description

Data processing method, device, equipment and storage medium
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a data processing method, apparatus, device, and storage medium.
Background
Hundreds of millions of data are generated in daily life all the time, and at present, when data display is carried out, multiple dimensions of the data are counted, and at least six dimensions are involved generally. The system code in the data field is highly abstract, so that a developer can obtain a full-dimensional query interface only by specifying a data structure, and can query corresponding data according to the full-dimensional query interface.
But the problems of the existing mode are as follows: the amount of data stored in a database is enormous, resulting in a lack of data of associated dimensions when querying the data. At this time, a developer is required to manually check data, and due to the huge data amount, manual checking consumes a lot of time, so that the efficiency is low, and manpower and time are wasted.
Disclosure of Invention
In view of this, the present invention provides a data processing method, apparatus, device and storage medium to overcome the deficiencies in the prior art, and aims to solve the current problems.
The invention provides the following technical scheme:
in a first aspect, an embodiment of the present disclosure provides a data processing method, where the method includes:
processing each data through a preset data processing rule to construct a database;
acquiring preset dimensions, and sequencing the dimensions, wherein each dimension comprises one or more levels;
associating the dimensions with respective data in the database, wherein each of the data corresponds to a hierarchy of dimensions;
acquiring a data structure to be tested, wherein the data structure to be tested comprises one or more data;
determining dimensions corresponding to the data, and searching whether dimensions of hierarchies lacking the corresponding data exist in the dimensions according to a preset hierarchy sequence;
if so, completing the dimension of the hierarchy lacking the data.
Further, before the step of processing each data according to the preset data processing rule and constructing the database, the method further includes:
generating a dimension corresponding to the data, wherein the dimension is used for defining the meaning of the data;
defining levels of said dimensions, each of said dimensions comprising one or more levels.
Further, the obtaining a preset dimension and sorting the dimensions, wherein each dimension includes one or more levels, includes:
obtaining the dimensionality of each preset level;
and sorting the dimensions according to the hierarchical order from low to high.
Further, the determining dimensions corresponding to the data, and searching whether there is a dimension lacking a level of the corresponding data in the dimensions according to a preset level sequence includes:
determining a dimension corresponding to the data;
and searching whether the dimensionality of the hierarchy lacking the corresponding data exists in the dimensionality layer by layer upwards according to the hierarchy sequence from low to high.
Further, if yes, completing the dimension of the hierarchy lacking the data, including:
if yes, determining the dimensionality of the hierarchy lacking the data, and completing the dimensionality according to pre-associated data and a preset rule.
Further, if the data is missing, after completing the dimensions of the missing data, the method further includes:
embedding points in the dimensionality of the hierarchy lacking the data to generate a plurality of abnormal embedded points;
and uploading the abnormal buried points to a data processing server.
In a second aspect, an embodiment of the present disclosure provides a data processing apparatus, including:
the data processing module is used for processing each data through a preset data processing rule to construct a database;
the dimension acquiring module is used for acquiring preset dimensions and sequencing the dimensions, wherein each dimension comprises one or more levels;
the association module is used for associating the dimension with each data in the database, wherein each data corresponds to a hierarchy of dimensions;
the data structure acquisition module is used for acquiring a data structure to be tested, wherein the data structure to be tested comprises one or more data;
the searching module is used for determining the dimensionality corresponding to the data and searching whether the dimensionality of the hierarchy lacking the corresponding data exists in the dimensionality according to a preset hierarchy sequence;
and the completion module is used for completing the dimensionality of the hierarchy lacking the data if the hierarchy lacks the data.
Further, the apparatus further comprises:
the dimension generation module is used for generating a dimension corresponding to the data, wherein the dimension is used for defining the meaning of the data;
a hierarchy definition module to define a hierarchy of the dimensions, each of the dimensions comprising one or more hierarchies.
In a third aspect, the present disclosure provides a computer device, including a memory and a processor, where the memory stores a computer program, and the processor implements the steps of the data processing method in the first aspect when executing the computer program.
In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, where a computer program is stored, and the computer program, when executed by a processor, implements the steps of the data processing method in the first aspect.
The embodiment of the application has the following advantages:
according to the data processing method provided by the embodiment of the application, each data is processed through a preset data processing rule, and a database is built; acquiring preset dimensions, and sequencing the dimensions, wherein each dimension comprises one or more levels; associating the dimensions with respective data in the database, wherein each of the data corresponds to a hierarchy of dimensions; acquiring a data structure to be tested, wherein the data structure to be tested comprises one or more data; determining the dimensionality corresponding to the data, and searching whether the dimensionality lacks the hierarchy of the corresponding data in the dimensionality according to a preset hierarchy sequence; if so, completing the dimension of the hierarchy lacking the data. Through the above, in the embodiment of the application, the data and the dimensionality are associated, the dimensionality of the hierarchy lacking the data can be found out according to the association relation, the data of the dimensionalities of all the hierarchies is completed, the workload of a developer is effectively reduced, and the method is convenient to maintain, clear to manage, simple and easy to use.
In order to make the aforementioned objects, features and advantages of the present invention more comprehensible and comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
Fig. 1 is a flowchart illustrating a data processing method provided in an embodiment of the present application;
fig. 2 is a schematic structural diagram illustrating a data processing apparatus according to an embodiment of the present application;
fig. 3 shows a hardware architecture diagram of a computer device provided in an embodiment of the present application.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention and are not to be construed as limiting the present invention.
It will be understood that when an element is referred to as being "secured to" another element, it can be directly on the other element or intervening elements may also be present. When an element is referred to as being "connected" to another element, it can be directly connected to the other element or intervening elements may also be present. In contrast, when an element is referred to as being "directly on" another element, there are no intervening elements present. The terms "vertical," "horizontal," "left," "right," and the like as used herein are for illustrative purposes only.
In the present invention, unless otherwise expressly stated or limited, the terms "mounted," "connected," "secured," and the like are to be construed broadly and can, for example, be fixedly connected, detachably connected, or integrally formed; can be mechanically or electrically connected; either directly or indirectly through intervening media, either internally or in any other relationship. The specific meanings of the above terms in the present invention can be understood according to specific situations by those of ordinary skill in the art.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the present invention, "a plurality" means two or more unless specifically defined otherwise.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description of the templates herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
Example 1
As shown in fig. 1, which is a flowchart of a data processing method in an embodiment of the present application, the data processing method provided in the embodiment of the present application includes the following steps:
step 101, processing each data through a preset data processing rule to construct a database.
In the embodiment of the present application, the data herein refers to relevant data of internal staff of a bank, for example, sales data of banking staff, sales data of bank groups, sales data of bank teams, sales data of bank branches, and the like. The data may include storage manners such as texts, tables, pictures, etc., and is not limited in particular here. Firstly, the acquired data is subjected to standardized processing according to a preset data processing rule, and repeated data, error data and the like in the data are subjected to standardized processing according to a preset rule, so that the data subsequently used for database construction is more regular, and the quality of the data is improved. After each data is processed, a database is built by using each processed data. The method for constructing the database may be determined according to the service requirement, and the embodiment of the present application is not limited herein.
According to the above embodiment, each data is processed using a preset data processing rule to remove invalid data in each data. And then constructing a database by using the processed data. Because each data in the database is processed data, each data in the database is more accurate, and correct data can be extracted from the database subsequently to complement the dimensionality of the hierarchy lacking the data.
Step 102, obtaining preset dimensions, and sorting the dimensions, wherein each dimension comprises one or more levels.
Specifically, preset dimensions of all levels are obtained, wherein each dimension comprises one or more levels, and the dimensions are sorted according to the order of the levels from low to high. For example, in the embodiment of the present application, the dimensions include a dimension of an operator level, a dimension of a group level, a dimension of a team level, and a dimension of a center-divided level, and after being sorted, the dimensions are: dimension of the operator level-dimension of the team level-dimension of the sub-center level.
By acquiring the preset dimensionality of each hierarchy and sequencing the dimensionalities according to the hierarchy sequence from low to high, data searching is conveniently carried out according to the hierarchy sequence subsequently, the dimensionality of the hierarchy lacking data is further searched, the data of the dimensionalities of all the hierarchies is supplemented, and the workload of developers is effectively reduced.
Step 103, associating the dimensions with each data in the database, wherein each data corresponds to a hierarchy of dimensions.
It is understood that the dimension is associated with each data in the database, for example, all data related to sales are associated with the dimension of "sales", wherein sales data of the salesman corresponds to the dimension of the salesman hierarchy, sales data of the team corresponds to the dimension of the team hierarchy, and sales data of the branch center corresponds to the dimension of the branch center hierarchy. It will be appreciated that the relationship between the several levels described above is that the business members are contained within small groups, that the small groups are contained within teams, and that the teams are contained within a hub.
According to the embodiment, the dimensions are associated with the data in the database in advance, then the dimensions of one hierarchy corresponding to each data are determined, the dimensions of the hierarchy lacking data can be found out conveniently according to the association relationship, the data of the dimensions of all the hierarchies are supplemented, and the workload of developers is effectively reduced.
Step 104, obtaining a data structure to be tested, wherein the data structure to be tested comprises one or more data.
Specifically, when a data structure test is required, a data structure to be tested is obtained, wherein the data structure to be tested includes one or more data, for example, the data structure is sales data of a team, a group or a salesman belonging to a branch center. The amount of data included in the data structure may be determined according to business requirements, and the embodiment of the present application is not limited herein.
And 105, determining the dimensionality corresponding to the data, and searching whether the dimensionality lacks the hierarchy of the corresponding data in the dimensionality according to a preset hierarchy sequence.
Preferably, the dimensions corresponding to the data are determined first, and whether the dimensions of the hierarchy lacking the corresponding data exist in the dimensions is searched layer by layer upwards according to the hierarchy sequence from low to high. For example, the data to be tested includes sales volume data of an operator, a dimension corresponding to the sales volume data of the operator is determined as a dimension of "sales volume", the dimension includes a dimension of an operator level, a dimension of a group level, a dimension of a team level and a dimension of a center-dividing level, and whether corresponding data are absent under the four levels of the dimensions is searched according to a level sequence from bottom to top.
It can be understood that the dimensionality of the hierarchy lacking data can be found out by sequentially searching upwards from the low hierarchy to the high hierarchy layer by layer, and the data of the dimensionalities of all the hierarchies is complemented, so that the workload of a developer is effectively reduced.
And step 106, if so, completing the dimensionality of the hierarchy lacking the corresponding data.
Further, if there is a dimension of a hierarchy lacking corresponding data, the dimension of the hierarchy lacking corresponding data is determined first, and then the dimension is completed according to pre-associated data and a preset rule. For example, if the dimension of the subgroup hierarchy lacks corresponding data after searching, the corresponding data is extracted from the database according to the data associated with the dimension of the subgroup hierarchy in advance and the preset rule for completion.
It can be understood that the above embodiment can complement the data of the dimensions of all the hierarchies, effectively reduce the workload of developers, and has the advantages of convenient maintenance, clear management, simplicity and easy use.
In an optional embodiment, before the data is processed through a preset data processing rule to construct the database, the method further includes:
step 107, generating a dimension corresponding to the data, wherein the dimension is used for defining the meaning of the data;
step 108, defining the levels of the dimensions, wherein each dimension comprises one or more levels.
In this embodiment, a dimension corresponding to the data is generated according to the data content, where the dimension is used to define the meaning of the data, and the name of the dimension may be determined according to actual requirements, for example, the dimension may be the sales volume. The levels of the dimension are defined, preferably, each dimension comprises one or more levels, and the number and the range of the levels corresponding to the dimension can be determined according to business requirements. For example, when the dimension is the sales volume, it may be determined that the sales volume corresponds to four levels, which are: a dimension of an attendant level, a dimension of a team level, a dimension of a hub level.
In an optional implementation manner, after completing the dimension of the hierarchy that lacks the data if any, the method further includes:
step 109, performing embedding on the dimensionality of the hierarchy lacking the data to generate a plurality of abnormal embedding points;
and step 110, uploading the abnormal buried points to a data processing server.
Further, when the fact that corresponding data are lacked under the dimensionality of a certain hierarchy is detected, the dimensionality of the hierarchy lacking the data is subjected to point burying, a plurality of abnormal buried points are generated, then the abnormal buried points are uploaded to a data processing server, and the data processing server analyzes and processes the data under the abnormal buried points. Wherein, the data processing server is used for further analyzing and processing the buried point data. Optionally, the data processing server may be a log server, the log server performs cleaning and conversion on the buried point data, and then generates a log according to the data under the abnormal buried point, and the generated log may be used for a developer to perform data analysis and the like.
The embodiment of the application has the following advantages:
according to the data processing method provided by the embodiment of the application, each data is processed through a preset data processing rule, and a database is built; acquiring preset dimensions, and sequencing the dimensions, wherein each dimension comprises one or more levels; associating the dimensions with respective data in the database, wherein each of the data corresponds to a hierarchy of dimensions; acquiring a data structure to be tested, wherein the data structure to be tested comprises one or more data; determining the dimensionality corresponding to the data, and searching whether the dimensionality lacks the hierarchy of the corresponding data in the dimensionality according to a preset hierarchy sequence; and if so, completing the dimension of the hierarchy lacking the data. According to the method and the device, the data and the dimensionality are associated, the dimensionality lacking the data hierarchy can be found out according to the association relation, the data of the dimensionalities of all the hierarchies is completed, the workload of a developer is effectively reduced, and the device and the method are convenient to maintain, clear to manage, simple and easy to use.
Example 2
As shown in fig. 2, which is a schematic structural diagram of a data processing apparatus 200 in the embodiment of the present application, the apparatus includes:
the data processing module 210 is configured to process each data according to a preset data processing rule to construct a database;
a dimension obtaining module 220, configured to obtain preset dimensions and sort the dimensions, where each dimension includes one or more levels;
an association module 230, configured to associate the dimensions with respective data in the database, where each data corresponds to a hierarchy of dimensions;
a data structure obtaining module 240, configured to obtain a data structure to be tested, where the data structure to be tested includes one or more data;
a searching module 250, configured to determine a dimension corresponding to the data, and search, according to a preset hierarchy order, whether a hierarchy dimension lacking corresponding data exists in the dimension;
a completion module 260, configured to complete the dimensions of the hierarchy lacking the data if any.
Optionally, the data processing apparatus may further include:
the dimension generation module is used for generating a dimension corresponding to the data, wherein the dimension is used for defining the meaning of the data;
a hierarchy definition module to define a hierarchy of the dimensions, each of the dimensions comprising one or more hierarchies.
Optionally, the data processing apparatus may further include:
the dimension acquisition submodule is used for acquiring the dimensions of each preset level;
and the dimension sorting module is used for sorting the dimensions according to the hierarchical order from low to high.
Optionally, the data processing apparatus may further include:
a determination module to determine a dimension corresponding to the data;
and the searching submodule is used for searching whether the dimensionality of the hierarchy lacking the corresponding data exists in the dimensionality layer by layer upwards according to the hierarchy sequence from low to high.
Optionally, the data processing apparatus may further include:
and the completion submodule is used for determining the dimensionality of the hierarchy lacking the data if the dimensionality is available and completing the dimensionality according to the pre-associated data and a preset rule.
Optionally, the data processing apparatus may further include:
the buried point generating module is used for performing buried point on the dimensionality of the hierarchy lacking the data to generate a plurality of abnormal buried points;
and the buried point uploading module is used for uploading the abnormal buried points to a data processing server.
The embodiment of the application has the following advantages:
the data processing device provided by the embodiment of the application can find out the dimensionality of the hierarchy lacking data according to the incidence relation by correlating the data and the dimensionality, and complement the data of the dimensionality of all the hierarchies, so that the workload of developers is effectively reduced, and the data processing device is convenient to maintain, clear to manage, simple and easy to use.
Example 3
Fig. 3 shows a hardware architecture diagram of a computer device provided in the present application, where the computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the data processing method according to embodiment 1 when executing the computer program.
In the present embodiment, the computer device 300 is a device capable of automatically performing numerical calculation and/or information processing in accordance with a command set or stored in advance. For example, the server may be a rack server, a blade server, a tower server, or a cabinet server (including an independent server or a server cluster composed of a plurality of servers). As shown in fig. 3, computer device 300 includes at least, but is not limited to: memory 310, processor 320, network interface 330 may be communicatively linked to each other via a system bus. Wherein:
the memory 310 includes at least one type of computer-readable storage medium including flash memory, hard disks, multimedia cards, card-type memory (e.g., SD or DX memory, etc.), random Access Memory (RAM), static Random Access Memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disks, optical disks, etc. In some embodiments, the storage 310 may be an internal storage module of the computer device 300, such as a hard disk or a memory of the computer device 300. In other embodiments, the memory 310 may also be an external storage device of the computer device 300, such as a plug-in hard disk provided on the computer device 300, a smart Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and so on. Of course, the memory 310 may also include both internal and external memory modules of the computer device 300. In this embodiment, the memory 310 is generally used for storing an operating system installed in the computer device 300 and various application software, such as program codes of a video playing method. In addition, the memory 310 may also be used to temporarily store various types of data that have been output or are to be output.
Processor 320 may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor, or other data Processing chip in some embodiments. The processor 320 generally serves to control the overall operation of the computer device 300, such as to perform control and processing related to data interaction or communication with the computer device 300. In this embodiment, the processor 320 is used to execute program codes stored in the memory 310 or process data.
The network interface 330 may include a wireless network interface or a wired network interface, the network interface 330 generally being used to establish a communication link between the computer device 300 and other computer devices. For example, the network interface 330 is used to connect the computer device 300 to an external terminal via a network, establish a data transmission channel and a communication link between the computer device 300 and the external terminal, and the like. The network may be an Intranet (Intranet), the Internet (Internet), a global system for Mobile communications (GSM), wideband Code Division Multiple Access (WCDMA), a 4G network, a 5G network, bluetooth (Bluetooth), wi-Fi, or other wireless or wired network.
It is noted that fig. 3 only shows a computer device with components 310-330, but it is understood that not all of the shown components are required to be implemented, and more or less components may be implemented instead.
In this embodiment, the data processing method stored in the memory 310 can be further divided into one or more program modules and executed by one or more processors (in this embodiment, the processor 320) to implement the present invention.
Example 4
The present embodiment also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the data processing method in the embodiments.
In this embodiment, the computer-readable storage medium includes a flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read Only Memory (ROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a Programmable Read Only Memory (PROM), a magnetic memory, a magnetic disk, an optical disk, and the like. In some embodiments, the computer readable storage medium may be an internal storage unit of the computer device, such as a hard disk or a memory of the computer device. In other embodiments, the computer readable storage medium may be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like provided on the computer device. Of course, the computer-readable storage medium may also include both internal and external storage units of the computer device. In this embodiment, the computer-readable storage medium is generally used for storing an operating system and various types of application software installed in the computer device. In addition, the computer-readable storage medium may also be used to temporarily store various types of data that have been output or are to be output.
In all examples shown and described herein, any particular value should be construed as exemplary only and not as a limitation, and thus other examples of example embodiments may have different values.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
The above examples are merely illustrative of several embodiments of the present invention, and the description thereof is more specific and detailed, but not to be construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention.

Claims (10)

1. A method of data processing, comprising:
processing each data through a preset data processing rule to construct a database;
acquiring preset dimensions, and sequencing the dimensions, wherein each dimension comprises one or more levels;
associating the dimensions with respective data in the database, wherein each of the data corresponds to a hierarchy of dimensions;
acquiring a data structure to be tested, wherein the data structure to be tested comprises one or more data;
determining dimensions corresponding to the data, and searching whether dimensions of hierarchies lacking the corresponding data exist in the dimensions according to a preset hierarchy sequence;
and if so, completing the dimension of the hierarchy lacking the data.
2. The data processing method of claim 1, wherein before the step of processing each data according to the preset data processing rule and building the database, the method further comprises:
generating a dimension corresponding to the data, wherein the dimension is used for defining the meaning of the data;
defining levels of the dimensions, each of the dimensions comprising one or more levels.
3. The data processing method of claim 1, wherein the obtaining of the preset dimensions and the sorting of the dimensions are performed, wherein each dimension comprises one or more levels, and comprises:
obtaining preset dimensionality of each level;
and sorting the dimensions according to a hierarchical order from low to high.
4. The data processing method of claim 1, wherein the determining dimensions corresponding to the data, and searching whether there is a dimension with a hierarchy lacking corresponding data in a preset hierarchy order in the dimensions comprises:
determining a dimension corresponding to the data;
and searching whether the dimensionality of the hierarchy lacking the corresponding data exists in the dimensionality layer by layer upwards according to the hierarchy sequence from low to high.
5. The data processing method of claim 1, wherein the complementing the dimensionality of the hierarchy that lacks the data, if any, comprises:
if yes, determining the dimensionality of the hierarchy lacking the data, and completing the dimensionality according to pre-associated data and a preset rule.
6. The data processing method of claim 1, wherein the completing the dimensions of the hierarchy lacking the data if any, further comprises:
embedding points in the dimensionality of the hierarchy lacking the data to generate a plurality of abnormal embedded points;
and uploading the abnormal buried points to a data processing server.
7. A data processing apparatus, characterized in that the apparatus comprises:
the data processing module is used for processing each data through a preset data processing rule to construct a database;
the dimension acquiring module is used for acquiring preset dimensions and sequencing the dimensions, wherein each dimension comprises one or more levels;
the association module is used for associating the dimension with each data in the database, wherein each data corresponds to a hierarchy of dimensions;
the data structure acquisition module is used for acquiring a data structure to be tested, wherein the data structure to be tested comprises one or more data;
the searching module is used for determining the dimensionality corresponding to the data and searching whether the dimensionality of the hierarchy lacking the corresponding data exists in the dimensionality according to a preset hierarchy sequence;
and the completion module is used for completing the dimensionality of the hierarchy lacking the data if the hierarchy lacks the data.
8. The data processing apparatus of claim 7, wherein the apparatus further comprises:
the dimension generation module is used for generating a dimension corresponding to the data, wherein the dimension is used for defining the meaning of the data;
a hierarchy definition module to define a hierarchy of the dimensions, each of the dimensions comprising one or more hierarchies.
9. A computer arrangement comprising a memory storing a computer program and a processor implementing the steps of the data processing method according to any one of claims 1-6 when the computer program is executed by the processor.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program which, when executed by a processor, carries out the steps of the data processing method of any one of claims 1 to 6.
CN202211116509.3A 2022-09-14 2022-09-14 Data processing method, device, equipment and storage medium Pending CN115391386A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211116509.3A CN115391386A (en) 2022-09-14 2022-09-14 Data processing method, device, equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211116509.3A CN115391386A (en) 2022-09-14 2022-09-14 Data processing method, device, equipment and storage medium

Publications (1)

Publication Number Publication Date
CN115391386A true CN115391386A (en) 2022-11-25

Family

ID=84126492

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211116509.3A Pending CN115391386A (en) 2022-09-14 2022-09-14 Data processing method, device, equipment and storage medium

Country Status (1)

Country Link
CN (1) CN115391386A (en)

Similar Documents

Publication Publication Date Title
CN110704231A (en) Fault processing method and device
CN112445875B (en) Data association and verification method and device, electronic equipment and storage medium
CN110825805B (en) Data visualization method and device
CN112231417A (en) Data classification method and device, electronic equipment and storage medium
CN115081025A (en) Sensitive data management method and device based on digital middlebox and electronic equipment
CN112328631A (en) Production fault analysis method and device, electronic equipment and storage medium
CN113722370A (en) Data management method, device, equipment and medium based on index analysis
CN111950623B (en) Data stability monitoring method, device, computer equipment and medium
CN116579671B (en) Performance assessment method, system, terminal and storage medium for automatically matching indexes
CN112052310A (en) Information acquisition method, device, equipment and storage medium based on big data
CN106845787A (en) A kind of data method for automatically exchanging and device
CN114841165B (en) User data analysis and display method and device, electronic equipment and storage medium
CN115391386A (en) Data processing method, device, equipment and storage medium
CN115878400A (en) Test method, test apparatus, computer device, storage medium, and program product
CN114564621A (en) Method, device and equipment for associating data and readable storage medium
CN114896230A (en) Application data real-time analysis distributed big data platform
CN115114297A (en) Data lightweight storage and search method and device, electronic equipment and storage medium
CN114862140A (en) Behavior analysis-based potential evaluation method, device, equipment and storage medium
CN113220551A (en) Index trend prediction and early warning method and device, electronic equipment and storage medium
CN113159118A (en) Logistics data index processing method, device, equipment and storage medium
CN113220726A (en) Data quality detection method and system
CN113806372B (en) New data information construction method, device, computer equipment and storage medium
CN111061632B (en) Automated test method and test system for report data
CN112131846A (en) Method and system for parameter transmission based on mapping relation, electronic device and storage medium
CN114997512A (en) Employee departure prediction method, device, equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination