CN117827902A - Service data processing method, device, computer equipment and storage medium - Google Patents
Service data processing method, device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN117827902A CN117827902A CN202311711540.6A CN202311711540A CN117827902A CN 117827902 A CN117827902 A CN 117827902A CN 202311711540 A CN202311711540 A CN 202311711540A CN 117827902 A CN117827902 A CN 117827902A
- Authority
- CN
- China
- Prior art keywords
- data
- information
- query instruction
- rule
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title abstract description 17
- 238000012216 screening Methods 0.000 claims abstract description 74
- 238000000034 method Methods 0.000 claims abstract description 51
- 238000013507 mapping Methods 0.000 claims abstract description 47
- 238000004590 computer program Methods 0.000 claims abstract description 23
- 238000012545 processing Methods 0.000 claims description 30
- 238000006243 chemical reaction Methods 0.000 claims description 10
- 238000001914 filtration Methods 0.000 claims description 8
- 230000009466 transformation Effects 0.000 claims description 6
- 238000010276 construction Methods 0.000 claims description 4
- 230000002829 reductive effect Effects 0.000 abstract description 7
- 230000008569 process Effects 0.000 description 18
- 238000012550 audit Methods 0.000 description 5
- 238000013500 data storage Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 230000000670 limiting effect Effects 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 230000019771 cognition Effects 0.000 description 2
- 238000002360 preparation method Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- OKTJSMMVPCPJKN-UHFFFAOYSA-N Carbon Chemical compound [C] OKTJSMMVPCPJKN-UHFFFAOYSA-N 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 229910021389 graphene Inorganic materials 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 230000036961 partial effect Effects 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present application relates to a business data processing method, apparatus, computer device, storage medium and computer program product. The method comprises the following steps: acquiring a query instruction of service data and data demand information carried in the query instruction; acquiring a target data table corresponding to the query instruction according to a mapping relation of a pre-stored service library table; splitting the data demand information to obtain a plurality of demand element information, and integrating the plurality of demand element information to obtain a data screening rule corresponding to the query instruction; and screening the target business data from the target data table based on the data screening rule. By adopting the method, the automatic configuration of the data query task can be realized, the time cost of developing rules is reduced, and the rule freedom of the service is increased; and the time cost and the error cost of manual operation can be effectively reduced, and the accuracy and the reliability of the data query result are improved.
Description
Technical Field
The present invention relates to the field of computer software technology, and in particular, to a service data processing method, apparatus, computer device, storage medium, and computer program product.
Background
In the big data age, the automatic data processing flow is a high-efficiency, accurate and quick data processing mode, so that the data processing efficiency and accuracy can be greatly improved, and the manual intervention and errors are reduced. For example, the cleaned and preprocessed data needs to be stored in a distributed storage system, HDFS employing Hadoop or cloud storage systems, which may provide efficient, scalable data storage and management functions.
However, in some specific working scenarios, it is still necessary to manually participate in the data processing process, and for example, the working process of financial accounting, which is required to complete the examination order according to the bill information submitted by the bill of lading person, is used. Document information requiring financial accounting attention includes: invoice information, amount, accounting subjects, descriptions, etc., most operations remain in the manual screening of part of the information and completion of data queries in the conventional workflow described above. Therefore, in the traditional workflow or method, the accuracy of data query is affected by subjective factors such as cognition and memory of a person to be examined, and certain instability is caused to the query result, namely the reliability of the final examination result is relatively poor.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a service data processing method, apparatus, computer device, computer readable storage medium, and computer program product that are capable of higher reliability.
In a first aspect, the present application provides a service data processing method. The method comprises the following steps:
acquiring a query instruction of service data and data demand information carried in the query instruction;
acquiring a target data table corresponding to the query instruction according to a mapping relation of a pre-stored service library table;
splitting the data demand information to obtain a plurality of demand element information, and integrating the plurality of demand element information to obtain a data screening rule corresponding to the query instruction;
and screening the target business data from the target data table based on the data screening rule.
In one embodiment, before the target data table corresponding to the query instruction is obtained according to the mapping relation of the pre-stored service library table, the method further includes:
acquiring data source information of historical service data and storing data table information of the service data;
and creating an enumeration type according to the data source information and the data table information, wherein the enumeration type is used for representing the mapping relation between the data source information and the data table information.
In one embodiment, the obtaining the target data table corresponding to the query instruction according to the mapping relation of the pre-stored service library table includes:
analyzing the query instruction to obtain data source constraint information of the target service data;
comparing and screening the data source constraint information with the data source information recorded in the enumeration type;
and when the data source information is the same as the data source constraint information, determining a target data table corresponding to the query instruction according to the data table information corresponding to the enumeration type of the data source information.
In one embodiment, the splitting the data requirement information to obtain a plurality of requirement element information, and integrating the plurality of requirement element information to obtain the data filtering rule corresponding to the query instruction includes:
and splitting the instruction text corresponding to the data demand information according to fields to obtain a plurality of initial elements, wherein the plurality of initial elements comprise: logical relationship characters, condition fields, and operators;
according to a preset rule grammar, carrying out grammar conversion on the plurality of initial elements to obtain a plurality of pieces of requirement element information;
And integrating the plurality of requirement element information according to grammar logic corresponding to the rule grammar to obtain a data screening rule.
In one embodiment, when the initial element is a logical relation character or an operator, the performing syntax transformation on the plurality of initial elements according to a preset rule syntax to obtain a plurality of requirement element information includes:
and according to the enumerated type mapping relation in the rule grammar, carrying out grammar conversion on a plurality of logical relation characters or a plurality of operators to obtain a plurality of element information.
In one embodiment, when the initial element is a condition field, the performing syntax transformation on the plurality of initial elements according to a preset rule syntax to obtain a plurality of requirement element information includes:
loading a byte code object corresponding to the entity type according to the entity type corresponding to the target data table;
and generating a plurality of attribute names of entity classes corresponding to the plurality of condition fields through the byte code object, and obtaining a plurality of element information according to the plurality of attribute names.
In one embodiment, the method further comprises: acquiring task information of data processing; taking the data screening rule as the task execution condition, and judging the task information; and if the task information meets the task execution conditions, executing the data processing task corresponding to the task information.
In a second aspect, the present application further provides a service data processing device. The device comprises:
the instruction acquisition module is used for acquiring a query instruction of service data and data demand information carried in the query instruction;
the data table query module is used for acquiring a target data table corresponding to the query instruction according to the mapping relation of the pre-stored service library table;
the rule construction module is used for splitting the data demand information to obtain a plurality of demand element information, and integrating the plurality of demand element information to obtain a data screening rule corresponding to the query instruction;
and the data screening module is used for screening the target service data from the target data table based on the data screening rule.
In a third aspect, the present application also provides a computer device. The computer device comprises a memory storing a computer program and a processor which when executing the computer program performs the steps of:
acquiring a query instruction of service data and data demand information carried in the query instruction;
acquiring a target data table corresponding to the query instruction according to a mapping relation of a pre-stored service library table;
Splitting the data demand information to obtain a plurality of demand element information, and integrating the plurality of demand element information to obtain a data screening rule corresponding to the query instruction;
and screening the target business data from the target data table based on the data screening rule.
In a fourth aspect, the present application also provides a computer-readable storage medium. The computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of:
acquiring a query instruction of service data and data demand information carried in the query instruction;
acquiring a target data table corresponding to the query instruction according to a mapping relation of a pre-stored service library table;
splitting the data demand information to obtain a plurality of demand element information, and integrating the plurality of demand element information to obtain a data screening rule corresponding to the query instruction;
and screening the target business data from the target data table based on the data screening rule.
In a fifth aspect, the present application also provides a computer program product. The computer program product comprises a computer program which, when executed by a processor, implements the steps of:
Acquiring a query instruction of service data and data demand information carried in the query instruction;
acquiring a target data table corresponding to the query instruction according to a mapping relation of a pre-stored service library table;
splitting the data demand information to obtain a plurality of demand element information, and integrating the plurality of demand element information to obtain a data screening rule corresponding to the query instruction;
and screening the target business data from the target data table based on the data screening rule.
The business data processing method, the business data processing device, the computer equipment, the storage medium and the computer program product are characterized in that firstly, a target data table is obtained through a screening mode of a mapping relation based on a query instruction of business data and data demand information carried in the query instruction; on the basis of obtaining a target data table, further splitting data demand information to obtain a plurality of demand element information, integrating the plurality of demand element information to obtain corresponding data screening rules, and finally screening according to the data screening rules to obtain target service data; based on the above process, the automatic configuration of the data query task can be realized, the time cost of developing rules is reduced, and the rule freedom of the service is increased; and the time cost and the error cost of manual operation can be effectively reduced, and the accuracy and the reliability of the data query result are improved.
Drawings
FIG. 1 is an application environment diagram of a business data processing method in one embodiment;
FIG. 2 is a flow chart of a business data processing method in one embodiment;
FIG. 3 is a flow diagram of the create enumeration type sub-step in one embodiment;
FIG. 4 is a flow chart of a business data processing method according to another embodiment;
FIG. 5 is a block diagram of a business data processing device in one embodiment;
fig. 6 is an internal structural diagram of a computer device in one embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.
In the traditional technical scheme, a manual data query mode is adopted, and the accuracy of a query result is judged to be influenced by subjective factors such as artificial subjective cognition and memory, so that a certain instability is caused to the query result; and the data query efficiency is too long in the time interval of partial receipts passing approval due to the arrangement difference of the query task time, so that the examination efficiency is affected.
In view of the above technical problems, the service data processing method provided in the embodiment of the present application may be applied to an application environment as shown in fig. 1. Wherein the terminal 102 communicates with the server 104 via a network. The data storage system may store data that the server 104 needs to process. The data storage system may be integrated on the server 104 or may be located on a cloud or other network server. The terminal 102 obtains a service data query instruction from the user through the interactive operation interface, and uploads the service data query instruction to the server 104. After receiving the service data query instruction, the server 104 obtains a target data table by a mapping relation screening mode based on the query instruction of the service data and the data demand information carried in the query instruction; on the basis of obtaining the target data table, further splitting the data demand information to obtain a plurality of demand element information, integrating the plurality of demand element information to obtain corresponding data screening rules, and finally screening according to the data screening rules to obtain target service data. Finally, the server 104 returns the target service data obtained by the query to the terminal 102 for visual display.
In the application environment shown in fig. 1, the terminal 102 may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, internet of things devices and portable wearable devices, where the internet of things devices may be smart speakers, smart televisions, smart air conditioners, smart vehicle devices, etc. The portable wearable device may be a smart watch, smart bracelet, headset, or the like. The server 104 may be implemented as a stand-alone server or as a server cluster of multiple servers.
In one embodiment, as shown in fig. 2, a service data processing method is provided, and the method is applied to the server 104 in fig. 1 for illustration, and includes the following steps:
step 202, acquiring a query instruction of service data and data demand information carried in the query instruction.
In an embodiment, business data refers to a collection of various data related to a business in an enterprise or organization; such data may include sales data, customer data, inventory data, financial data, and the like. The pre-stored service data refers to the historical service data which is stored and finished in the data storage system of the server before the data query instruction is sent to the server. The query instruction in the embodiment is an instruction for acquiring target content of service data; the data requirement information in the embodiment is information for describing and limiting the target content specified in the query instruction, including, but not limited to, time information of target service data required to be queried, and the like.
Illustratively, the data storage system or database of the server in the embodiment stores various service data generated in real time in various service scenarios, including, but not limited to, user information, device information, IP information, address information, order information, payment information, and the like. For example, in an embodiment, device information may be obtained from mobile terminal devices (mobile phones, computers, tablets, etc.); user information obtained from user registration, login, consumption, etc.; order information and payment information acquired from a payment platform. After the above-mentioned various service data are passed through the sorting and collecting process, they are uploaded to server, and stored in the data in the server. On the other hand, in a scenario of data audit or data query, for example, in a scenario of automatic audit based on service data, in order to complete an automatic audit task, in an embodiment, acquisition and query of target service data need to be performed, so that the automatic audit task can be advanced. Therefore, in the embodiment, the query instruction of the target business data is generated based on the configured automatic audit task, and the content to be queried is explicitly described and limited in the query instruction, for example, all order completion times in the target time period need to be queried, so that the foregoing descriptive content is stored in the query instruction as data requirement information and is sent to the server.
Step 204, obtaining a target data table corresponding to the query instruction according to the mapping relation of the pre-stored business library table.
In the embodiment, the mapping relation of the service library table is in an enumeration mapping mode, and describes the one-to-one corresponding association relation between the type of the service data and the library table of the service data; for example, the relevant data for the business order may be correspondingly stored in a business order database.
In particular, in the embodiment, different data sources can be connected by using a Spring dynamic multi-data source mode to acquire data tables of different systems, so that a configurable rule is changeable, and preparation is made for general service. More specifically, after receiving the query instruction in the foregoing step, the server in the embodiment analyzes the query instruction, analyzes the query instruction through a data processing analysis flow such as natural language processing analysis and identification and extraction of a key field, and obtains a data type of the queried service data or other attribute information capable of uniquely describing the service data from the query instruction list, and determines a database table to which the query instruction is uniquely directed according to a (unique) correspondence between the data type or other attribute information and the responsive database table.
And 206, splitting the data demand information to obtain a plurality of demand element information, and integrating the plurality of demand element information to obtain a data screening rule corresponding to the query instruction.
In an embodiment, the requirement element information is information content characterizing the requirement information in a manner of small granularity such as characters, text fields, or text phrases. Such as logical symbols and the type of data that the query outputs, etc. In the embodiment, the data filtering rule is a rule obtained by integrating the plurality of information requirements obtained by splitting according to a certain rule or logic, and the presentation mode of the rule can be instruction text described in a computer language, such as script degree of JAVA language or instruction statement of SQL.
In an exemplary embodiment, after determining a database table pointed by a query instruction based on a mapping relation of a preset service library table, processing data requirement information in the query instruction is required to form element information with smaller granularity, and then encoding or converting the element information obtained by splitting to form a rule or instruction (i.e., a data screening rule in the embodiment) of a responsive computing language, so that a server can screen and query all data contents in a target data table according to the obtained data screening rule.
And step 208, screening the target business data from the target data table based on the data screening rule.
In particular, in the embodiment, after the data screening rule is constructed through the foregoing steps, the server can screen all the service data recorded in the target data table according to the data screening rule, so as to obtain the target service data which can conform to the data screening rule.
In the service data processing method, firstly, a target data table is obtained through a screening mode of a mapping relation based on a query instruction of service data and data demand information carried in the query instruction; on the basis of obtaining a target data table, further splitting data demand information to obtain a plurality of demand element information, integrating the plurality of demand element information to obtain corresponding data screening rules, and finally screening according to the data screening rules to obtain target service data; based on the above process, the automatic configuration of the data query task can be realized, the time cost of developing rules is reduced, and the rule freedom of the service is increased; and the time cost and the error cost of manual operation can be effectively reduced, and the accuracy and the reliability of the data query result are improved.
In one embodiment, as shown in fig. 3, before obtaining the target data table corresponding to the query instruction according to the mapping relation of the pre-stored service library table, the method may further include the following steps:
step 302, obtaining data source information of historical service data and data table information of stored service data.
In an embodiment, the data source information is an acquisition source for characterizing the service data, and the data report information in an embodiment is specific identification information for characterizing a database or a data table storing the service data. More specifically, in the embodiment, any historical service data value is obtained, and the historical service data can be annotated according to the generation mode, the specific source, the acquisition mode, the acquisition time, the data type of the service data and the stored database name of the historical service data, so that the source information and the stored data table information of the data can be directly determined through the annotation information in the subsequent data processing process.
Step 304, creating enumeration types according to the data source information and the data table information, wherein the enumeration types are used for representing the mapping relation between the data source information and the data table information.
In an embodiment, an enumeration mapping is an operation that uses an enumeration type as a key, mapping values to other types. Specifically, selecting a Java implementation (software) environment in an embodiment, enumeration mapping may be implemented using the EnumMap class. Then, in the process of constructing the mapping relationship between the data table names and the pre-stored data tables, firstly, a package in which the EnumMap class is located needs to be imported, and then an enumeration type is defined as a key of the EnumMap, for example, in the embodiment, the source of service data can be used as a key value; the source of the service data in the embodiment may be determined according to a database table stored by different types of service data. Next, a new map may be created using EnumMap, key-value pairs may be added to the map using the put () method, and corresponding values, i.e., pre-stored data tables corresponding to key-values, may be obtained from the keys using the get () method.
In one embodiment, the process of obtaining the target data table corresponding to the query instruction according to the mapping relation of the pre-stored service library table in the method may further include the following steps:
analyzing and obtaining data source constraint information of target service data from the query instruction.
And step two, comparing and screening the data source constraint information and the data source information recorded in the mapping relation.
And thirdly, determining a target data table corresponding to the query instruction according to the pre-stored data table corresponding to the data source information in the mapping relation when the data source information is the same as the data source constraint information.
In an embodiment, the data source constraint information in the query instruction is source information for describing the target service data queried and obtained by the query instruction, that is, the source constraint information is used for constraining the source of the target service data. In addition, in the embodiment, the pre-stored data table is obtained by dividing and storing the historical service data according to a certain clustering rule; specifically, in an embodiment, the clustering rule may be a partitioning rule determined according to a correlation attribute of the service data table, for example, the service data is classified according to a service type, and stored in different data tables or databases respectively. More specifically, in the embodiment, the constraint information of the query instruction on the data source may be constraint on the data type of the service data.
The screening process of the target data table is performed by means of enumeration mapping in an embodiment, which may be by setting a plurality of fixed (key, value) relationship values; in the screening stage of the target database, the key value may be information according to the data source specified in the query instruction, so that the embodiment server may traverse all the fixed values to determine that the key of the transmitted key is consistent with the key of the fixed value, directly intercept the key, and directly read the value of the relationship, to obtain the unique identification information of the target database table, such as the name or ID of the target database table. In the embodiment, the database table pointed by the query instruction can be more efficiently and accurately determined by an enumeration mapping mode.
In an embodiment, in the method, the process of splitting the data requirement information to obtain a plurality of requirement element information and integrating the plurality of requirement element information to obtain the data screening rule corresponding to the query instruction may include the following steps:
step one, splitting an instruction text corresponding to data demand information according to fields to obtain a plurality of initial elements, wherein the plurality of initial elements comprise: logical relationship characters, condition fields, and operators.
And secondly, carrying out grammar conversion on the plurality of initial elements according to a preset rule grammar to obtain a plurality of pieces of requirement element information.
And thirdly, integrating the plurality of requirement element information according to grammar logic corresponding to the rule grammar to obtain a data screening rule.
In an embodiment, logical relationships are used to describe logical associations, such as and or, between text information. The field condition in the embodiment may be a description that defines data by attribute characteristics, or attribute names; such as the name of the attribute defined by the entity class. In the embodiment, the operator is an operational relation for describing the data. In the embodiment, in order that the target data table or database can accurately respond to the instruction statement, in the process of obtaining the data filtering rule according to the combination of the plurality of element information, necessary grammar compiling is needed, and the plurality of requirement element information is combined according to grammar logic corresponding to the data table or database.
In the process of constructing the data filtering rule, the requirements in the query instruction are firstly extracted to form the instruction text corresponding to the requirement information; and then splitting text fields with smaller granularity from the instruction text through a preset semantic analysis model to obtain a plurality of initial elements such as logical relation characters, condition fields, operators and the like. More specifically, in an embodiment, the process of splitting the fields may be implemented by way of a dictionary. After obtaining a plurality of initial elements, in the embodiment, corresponding grammar rules are determined according to a database stored in the target data table, and encoding of grammar conversion of the initial elements is obtained. Illustratively, in the embodiment, a plurality of initial elements are obtained for encoding based on the Wrappers grammar and are integrated and spliced; more specifically, in the stage of extracting the initial element, the relationship of "and the attribute of the condition field are: "name"; obtain=. Then, the wrappers are used for conditional combination to obtain the bottom SQL grammar, and name=? . Wherein? Values defined for the user are uploaded by the user from the visual interface. The embodiment introduces a necessary grammar conversion process, ensures that the data query task can be smoothly executed, and improves the quasimilitude of the data query result.
In one embodiment, when the initial element is a logical relation character or an operator, in the method, according to a preset rule grammar, performing grammar conversion on the plurality of initial elements to obtain a plurality of requirement element information, the method includes the following steps:
firstly, according to the enumerated type mapping relation in the rule grammar, carrying out grammar conversion on a plurality of logical relation characters or a plurality of operators to obtain a plurality of element information.
Illustratively, embodiments are directed to logical relationships and AND OR; mapping associations, and representations and relationships, or representations or relationships may be made by and/or using enumerated types. In an embodiment, for operators, hard transformations may be performed by enumeration classes, e.g., EQUALS into "=" symbols, NE into "+|! = "etc.
In one embodiment, when the initial element is a condition field, the method performs syntax transformation on the plurality of initial elements according to a preset rule syntax, and the process of obtaining the plurality of requirement element information includes the following steps:
step one, according to the entity type corresponding to the target data table, loading the byte code object corresponding to the entity type.
And step two, generating a plurality of attribute names of entity classes corresponding to the plurality of condition fields through the byte code object, and obtaining a plurality of element information according to the plurality of attribute names.
For example, for the condition field extracted in the embodiment, the embodiment needs to determine the attribute name of the entity class corresponding to the condition field by using the java reflection technology. Specifically, in the embodiment, the table name of the target data table is determined according to the previous steps; and mapping out the entity class of the corresponding table by enumeration according to the table name, loading the byte code object according to the name of the entity class, and acquiring the attribute name defined by the class through the byte code. More specifically, the class of the entity is compiled by java, and the file contains the structure, method, field and the like of the class. The reflection technology is to obtain metadata of the class through java. Lang. Class, and obtain the class by using the getDeclaredFields () method to parse all attribute names according to the byte code.
In one embodiment, the method provided herein further comprises the steps of:
step one, task information of data processing is obtained.
And step two, taking the data screening rule as a task execution condition, and judging task information.
And step three, executing the data processing task corresponding to the task information if the task information meets the task execution condition.
In an embodiment, the data processing task refers to subsequent processing required after obtaining the service data query result, for example, intercepting an order or automatically reviewing an order. Taking a task of intercepting orders as an example, in the embodiment, the server may intercept documents submitted by a user by using a rule engine. And capturing business rule data corresponding to the bill from a rule storage library, and converting the rule into a rule statement with judging conditions according to the data screening rule formed in the previous step. And counting whether the bill has passing conditions or not by using the count grammar of mysql. When the counted value is higher than 1, interception is not performed, and when the counted value is lower than 1, interception is performed.
According to fig. 4 of the specification, a complete implementation process of the service data processing method provided by the technical scheme of the application is described as follows:
and step one, mapping the relation of the business library table. Mapping to a business library table name using enumeration; and the data tables of different systems are acquired by connecting different data sources in a Spring dynamic multi-data source mode, so that the preparation for the business with changeable configurable rules and universality is achieved.
And secondly, splicing rule conditions, namely splicing SQL grammar according to conditions selected by a user. The specific splicing process may further specifically include the steps of:
and sub-step 1, extracting a logical relation and AND or. Mapping association is performed through and/or by using enumeration types, and represents and relation, or represents or relation.
And 2, extracting the condition field content. Embodiments may be implemented using java's reflection technology. According to the table names mapped out in the technical scheme 1 point of the invention, the entity class of the corresponding table is mapped out by enumeration according to the table names, the byte code object is loaded according to the names of the entity class, and the attribute names defined by the class are obtained through the byte codes. For example, the class is compiled by java, and the file contains the structure, method, field, etc. of the class. The reflection technology is to obtain metadata of the class through java. Lang. Class, and obtain the class by using the getDeclaredFields () method to parse all attribute names according to the byte code.
And 3, operator acquisition. The operator is hard translated by an enumeration class, e.g., EQUALS translates to =, NE translates to-! =etc.
And 4, performing conditional integration assembly on logic of the sub-step 1 and the sub-step 3 by utilizing a Wrappers grammar according to the result data obtained in the sub-step. E.g., substep 1 gets a "and" relationship; step 2, extracting to obtain a name attribute; and 3, extracting in the substep to obtain "=". Finally, the condition combination is carried out by applying writers to obtain a bottom SQL grammar statement, namely, and name=? . (wherein.
Thirdly, searching and inquiring all the historical service data according to the rules formed in the steps and the user-defined values uploaded by the user, and returning the obtained inquiry results to a visual interface of the terminal; and storing the rule content formed by the steps in a library by using Mysql.
Step four, converting the rule content stored in the step three into a rule sentence with judging conditions; constraint limits are placed on the data processing tasks. For example, intercepting documents submitted by users by using a rule engine; and capturing business rule data 1 corresponding to the bill from a rule storage library, and counting whether the bill has passing conditions by using the count grammar of mysql. When the counted value is higher than 1, interception is not performed, and when the counted value is lower than 1, interception is performed.
For another example, in an embodiment, rule condition data of the automatic review may be extracted from the rule repository to perform rule filtering. And if the data passes, automatically approving the data to mark the data as passing, otherwise, changing the data to not pass.
It should be understood that, although the steps in the flowcharts related to the above embodiments are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.
Based on the same inventive concept, the embodiment of the application also provides a service data processing device for implementing the service data processing method. The implementation of the solution provided by the device is similar to the implementation described in the above method, so the specific limitation in the embodiments of one or more service data processing devices provided below may refer to the limitation of the service data processing method in the above description, which is not repeated here.
In one embodiment, as shown in fig. 5, there is provided a service data processing apparatus 500, including: an instruction acquisition module 501, a data table query module 502, a rule construction module 503, and a data screening module 504, wherein:
the instruction acquisition module 501 is configured to acquire a query instruction of service data and data requirement information carried in the query instruction;
the data table query module 502 is configured to obtain a target data table corresponding to the query instruction according to a mapping relationship of the pre-stored service library table;
a rule construction module 503, configured to split the data requirement information to obtain a plurality of requirement element information, and integrate the plurality of requirement element information to obtain a data screening rule corresponding to the query instruction;
the data filtering module 504 is configured to obtain the target service data from the target data table based on the data filtering rule.
In one embodiment, the apparatus 500 further includes a map creation module for obtaining data source information of the historical service data and storing data table information of the service data; and creating an enumeration type according to the data source information and the data table information, wherein the enumeration type is used for representing the mapping relation between the data source information and the data table information.
In one embodiment, the mapping creation module is further configured to parse the query instruction to obtain data source constraint information of the target service data; comparing and screening the data source constraint information with the data source information recorded in the enumeration type; and when the data source information is the same as the data source constraint information, determining a target data table corresponding to the query instruction according to the data table information corresponding to the enumeration type of the data source information.
In one embodiment, the rule building module 503 is further configured to split the instruction text corresponding to the data requirement information according to fields, to obtain a plurality of initial elements, where the plurality of initial elements include: logical relationship characters, condition fields, and operators; according to a preset rule grammar, carrying out grammar conversion on a plurality of initial elements to obtain a plurality of requirement element information; and integrating the plurality of requirement element information according to grammar logic corresponding to the rule grammar to obtain a data screening rule.
In one embodiment, the rule building module 503 is further configured to, when the initial element is a logical relationship character or an operator, perform syntax transformation on a plurality of logical relationship characters or a plurality of operators according to a mapping relationship of an enumerated type in a rule syntax to obtain a plurality of element information.
In one embodiment, the rule building module 503 is further configured to, when the initial element is the condition field, load a bytecode object corresponding to the entity type according to the entity type corresponding to the target data table; and generating a plurality of attribute names of the entity class corresponding to the plurality of condition fields through the byte code object, and obtaining a plurality of element information according to the plurality of attribute names.
In one embodiment, the apparatus 500 further includes a task interception module for acquiring task information of the data processing; taking the data screening rule as a task execution condition, and judging task information; and if the task information meets the task execution conditions, executing the data processing task corresponding to the task information.
The various modules in the service data processing device described above may be implemented in whole or in part by software, hardware, or a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In one embodiment, a computer device is provided, which may be a server, the internal structure of which may be as shown in fig. 6. The computer device includes a processor, a memory, an Input/Output interface (I/O) and a communication interface. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface is connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is used for storing various business data. The input/output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a business data processing method.
It will be appreciated by those skilled in the art that the structure shown in fig. 6 is merely a block diagram of some of the structures associated with the present application and is not limiting of the computer device to which the present application may be applied, and that a particular computer device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided comprising a memory and a processor, the memory having stored therein a computer program, the processor when executing the computer program performing the steps of:
acquiring a query instruction of service data and data demand information carried in the query instruction;
acquiring a target data table corresponding to the query instruction according to a mapping relation of a pre-stored service library table;
splitting the data demand information to obtain a plurality of demand element information, and integrating the plurality of demand element information to obtain a data screening rule corresponding to the query instruction;
and screening the target business data from the target data table based on the data screening rule.
In one embodiment, a computer readable storage medium is provided having a computer program stored thereon, which when executed by a processor, performs the steps of:
Acquiring a query instruction of service data and data demand information carried in the query instruction;
acquiring a target data table corresponding to the query instruction according to a mapping relation of a pre-stored service library table;
splitting the data demand information to obtain a plurality of demand element information, and integrating the plurality of demand element information to obtain a data screening rule corresponding to the query instruction;
and screening the target business data from the target data table based on the data screening rule.
In one embodiment, a computer program product is provided comprising a computer program which, when executed by a processor, performs the steps of:
acquiring a query instruction of service data and data demand information carried in the query instruction;
acquiring a target data table corresponding to the query instruction according to a mapping relation of a pre-stored service library table;
splitting the data demand information to obtain a plurality of demand element information, and integrating the plurality of demand element information to obtain a data screening rule corresponding to the query instruction;
and screening the target business data from the target data table based on the data screening rule.
It should be noted that, the user information (including, but not limited to, user equipment information, user personal information, etc.) and the data (including, but not limited to, data for analysis, stored data, presented data, etc.) referred to in the present application are information and data authorized by the user or sufficiently authorized by each party, and the collection, use and processing of the related data are required to comply with the related laws and regulations and standards of the related countries and regions.
Those skilled in the art will appreciate that implementing all or part of the above-described methods in accordance with the embodiments may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in the various embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magnetic random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (Phase Change Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like. The databases referred to in the various embodiments provided herein may include at least one of relational databases and non-relational databases. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processors referred to in the embodiments provided herein may be general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic units, quantum computing-based data processing logic units, etc., without being limited thereto.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The foregoing examples represent only a few embodiments of the present application, which are described in more detail and are not thereby to be construed as limiting the scope of the present application. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application shall be subject to the appended claims.
Claims (10)
1. A method for processing service data, the method comprising:
acquiring a query instruction of service data and data demand information carried in the query instruction;
acquiring a target data table corresponding to the query instruction according to a mapping relation of a pre-stored service library table;
splitting the data demand information to obtain a plurality of demand element information, and integrating the plurality of demand element information to obtain a data screening rule corresponding to the query instruction;
And screening the target business data from the target data table based on the data screening rule.
2. The method according to claim 1, wherein before obtaining the target data table corresponding to the query instruction according to the mapping relation of the pre-stored service library table, the method further comprises:
acquiring data source information of historical service data and storing data table information of the service data;
and creating an enumeration type according to the data source information and the data table information, wherein the enumeration type is used for representing the mapping relation between the data source information and the data table information.
3. The method of claim 2, wherein the obtaining the target data table corresponding to the query instruction according to the mapping relation of the pre-stored service library table comprises:
analyzing the query instruction to obtain data source constraint information of the target service data;
comparing and screening the data source constraint information with the data source information recorded in the enumeration type;
and when the data source information is the same as the data source constraint information, determining a target data table corresponding to the query instruction according to the data table information corresponding to the enumeration type of the data source information.
4. The method of claim 1, wherein the splitting the data requirement information to obtain a plurality of requirement element information, and integrating the plurality of requirement element information to obtain the data filtering rule corresponding to the query instruction includes:
and splitting the instruction text corresponding to the data demand information according to fields to obtain a plurality of initial elements, wherein the plurality of initial elements comprise: logical relationship characters, condition fields, and operators;
according to a preset rule grammar, carrying out grammar conversion on the plurality of initial elements to obtain a plurality of pieces of requirement element information;
and integrating the plurality of requirement element information according to grammar logic corresponding to the rule grammar to obtain a data screening rule.
5. The method of claim 4, wherein when the initial element is a logical relationship character or an operator, the performing syntax transformation on the plurality of initial elements according to a preset rule syntax to obtain a plurality of requirement element information includes:
and according to the enumerated type mapping relation in the rule grammar, carrying out grammar conversion on a plurality of logical relation characters or a plurality of operators to obtain a plurality of element information.
6. Loading a byte code object corresponding to the entity type according to the entity type corresponding to the target data table;
and generating a plurality of attribute names of entity classes corresponding to the plurality of condition fields through the byte code object, and obtaining a plurality of element information according to the plurality of attribute names.
7. The method according to any one of claims 1-6, further comprising:
acquiring task information of data processing;
taking the data screening rule as a task execution condition, and judging the task information;
and if the task information meets the task execution conditions, executing the data processing task corresponding to the task information.
8. A traffic data processing apparatus, the apparatus comprising:
the instruction acquisition module is used for acquiring a query instruction of service data and data demand information carried in the query instruction;
the data table query module is used for acquiring a target data table corresponding to the query instruction according to the mapping relation of the pre-stored service library table;
the rule construction module is used for splitting the data demand information to obtain a plurality of demand element information, and integrating the plurality of demand element information to obtain a data screening rule corresponding to the query instruction;
And the data screening module is used for screening the target service data from the target data table based on the data screening rule.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any of claims 1 to 7 when the computer program is executed.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311711540.6A CN117827902A (en) | 2023-12-12 | 2023-12-12 | Service data processing method, device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311711540.6A CN117827902A (en) | 2023-12-12 | 2023-12-12 | Service data processing method, device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117827902A true CN117827902A (en) | 2024-04-05 |
Family
ID=90523546
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311711540.6A Pending CN117827902A (en) | 2023-12-12 | 2023-12-12 | Service data processing method, device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117827902A (en) |
-
2023
- 2023-12-12 CN CN202311711540.6A patent/CN117827902A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11748371B2 (en) | Systems and methods for searching for and translating real estate descriptions from diverse sources utilizing an operator-based product definition | |
| US9229971B2 (en) | Matching data based on numeric difference | |
| CN110795524B (en) | Main data mapping processing method and device, computer equipment and storage medium | |
| CN112287013B (en) | Data conversion method and adapter | |
| CN116737915B (en) | Semantic retrieval method, device, equipment and storage medium based on knowledge graph | |
| US10671631B2 (en) | Method, apparatus, and computer-readable medium for non-structured data profiling | |
| US20160188685A1 (en) | Fan identity data integration and unification | |
| CN112732763A (en) | Data aggregation method and device, electronic equipment and medium | |
| US11544669B2 (en) | Computing framework for compliance report generation | |
| CN116450890A (en) | Graph data processing method, device and system, electronic equipment and storage medium | |
| US8862609B2 (en) | Expanding high level queries | |
| CN107729330B (en) | Method and apparatus for acquiring data set | |
| CN113626558B (en) | Intelligent recommendation-based field standardization method and system | |
| CN109542890B (en) | Data modification method, device, computer equipment and storage medium | |
| CN114860737B (en) | Processing method, device, equipment and medium of teaching and research data | |
| CN115357625A (en) | Structured data comparison method and device, electronic equipment and storage medium | |
| CN112765197B (en) | Data query method, device, computer equipment and storage medium | |
| US11321340B1 (en) | Metadata extraction from big data sources | |
| CN114860819A (en) | Method, device, equipment and storage medium for constructing business intelligent system | |
| CN117827902A (en) | Service data processing method, device, computer equipment and storage medium | |
| US9471569B1 (en) | Integrating information sources to create context-specific documents | |
| Kalampokis et al. | Check for updates Towards Interoperable Open Statistical Data | |
| CN118200407A (en) | Method and device for generating message codes, computer equipment and readable storage medium | |
| CN113076317A (en) | Data processing method, device and equipment based on big data and readable storage medium | |
| CN117521610A (en) | Data processing method, computer device, and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |