CN117521610A

CN117521610A - Data processing method, computer device, and readable storage medium

Info

Publication number: CN117521610A
Application number: CN202311279508.5A
Authority: CN
Inventors: 王晶; 贺庆江; 方健力; 资武; 余鑫
Original assignee: Zhongke Yungu Technology Co Ltd
Current assignee: Zhongke Yungu Technology Co Ltd
Priority date: 2023-09-28
Filing date: 2023-09-28
Publication date: 2024-02-06

Abstract

The application discloses a data processing method, computer equipment and readable storage medium, comprising the following steps: acquiring JSON data to be processed; performing format conversion processing on the JSON data to be processed based on a preset rule configuration table to obtain target format data; the preset rule configuration table comprises data analysis logic for converting the JSON field into a field corresponding to the target format. Therefore, the target format data generation logic is configured and managed based on the preset rule configuration table, the workload of development and maintenance is reduced, the understanding and maintenance are easy, and the data conversion efficiency is improved.

Description

Data processing method, computer device, and readable storage medium

Technical Field

The present invention relates to the field of data processing technologies, and in particular, to a data processing method, a computer device, and a readable storage medium.

Background

The Excel report is a data form with good readability, and is convenient for users to review data. Typically, data originates from different organizations, systems or modules, and data standards in different environments are different, so that certain conversion and combination are required, and finally the data is displayed in a report format suitable for reading. Considering that the data source interface has been used in advance for production and that report requirements are typically set forth by users after the system has been in use for a period of time, it is necessary for developers to understand the existing data query interface, to process logic for each field in the report, and to complete the report generation logic in a hard-coded manner (i.e., program code).

The inventors found in the study that the above-described manner has the following problems: the code machinery is repeatedly piled up with value and assignment logic, and the butt joint of developers is difficult, so that the code quantity of development and maintenance is large, the understanding and maintenance are not easy, and meanwhile, the data conversion efficiency is low.

Disclosure of Invention

The invention aims to provide a data processing method, computer equipment and a computer readable storage medium, which can reduce the workload of development and maintenance, are easy to understand and maintain and improve the data conversion efficiency.

To achieve the above object:

in a first aspect, an embodiment of the present application provides a data processing method, where the method includes:

acquiring JSON data to be processed;

performing format conversion processing on the JSON data to be processed based on a preset rule configuration table to obtain target format data; the preset rule configuration table comprises data analysis logic for converting the JSON field into a field corresponding to the target format.

In an embodiment, the performing format conversion processing on the JSON data to be processed based on the preset rule configuration table to obtain target format data includes:

analyzing each source field expression in a preset rule configuration table to obtain a parameter object list corresponding to each source field expression; the source field expression comprises at least one JSON field, and the parameter object list comprises parameter objects corresponding to the JSON fields forming the corresponding source field expression;

Performing format conversion processing on the JSON data to be processed based on a parameter object list corresponding to each source field expression and a mapping rule in the preset rule configuration table to obtain target format data; the mapping rule includes a mapping field of each of the source field expressions in the target format data to be generated.

In an embodiment, the parsing each source field expression in the preset rule configuration table to obtain a parameter object list corresponding to each source field expression includes:

separating at least one JSON field in each source field expression into at least one parameter according to a first preset character contained in each source field expression in a preset rule configuration table;

creating a parameter object corresponding to each parameter based on the attribute of the at least one parameter; the attribute of the parameter object comprises a parameter value type and a parameter identifier;

and generating a parameter object list corresponding to each source field expression according to the parameter object corresponding to each parameter.

In an embodiment, the creating, based on the attribute of the at least one parameter, a parameter object corresponding to each parameter includes:

detecting whether a target parameter contains a second preset character, wherein the target parameter is any parameter;

If yes, determining the parameter value type of the target parameter as an array;

if not, determining the parameter value type of the target parameter as an object;

and creating a parameter object corresponding to the target parameter based on the parameter value type of the target parameter.

In an embodiment, the performing format conversion processing on the JSON data to be processed based on the parameter object list corresponding to each source field expression and the mapping rule in the preset rule configuration table to obtain target format data includes:

acquiring a value corresponding to each source field expression from JSON data to be processed based on a parameter object list corresponding to each source field expression;

determining mapping fields of the source field expressions in the target format data to be generated and positions of the mapping fields in the target format data to be generated according to mapping rules in the preset rule configuration table;

determining the value of each mapping field according to the value corresponding to each source field expression;

and generating target format data according to the values of the mapping fields and the positions of the mapping fields in the target format data to be generated.

In an embodiment, the obtaining, based on the parameter object list corresponding to each source field expression, the value corresponding to each source field expression from the JSON data to be processed includes at least one of:

responding to the parameter object list corresponding to the source field expression to only contain one parameter object, acquiring the value of a JSON field matched with the parameter object from the JSON message data to be processed, and determining the value of the JSON field matched with the parameter object as the value corresponding to the source field expression;

and traversing each parameter object according to a preset value rule to obtain the value of the JSON field matched by each parameter object, and determining the value corresponding to the source field expression according to the value of the JSON field matched by each parameter object.

In an embodiment, traversing each parameter object according to a preset value rule to obtain a value of a JSON field matched by each parameter object includes:

acquiring the value of the JSON field matched with the previous parameter object;

and determining the value of the JSON field matched with the next parameter object based on the value of the JSON field matched with the last parameter object and a preset value rule corresponding to the JSON field matched with the next parameter object.

In an embodiment, the determining the value of each mapping field according to the value corresponding to each source field expression includes:

determining a mapping value matched with a value corresponding to the target source field expression in the preset rule configuration table according to the value corresponding to the target source field expression; the target source field expression is any source field expression;

and determining the mapping value as a value of a target mapping field, wherein the target mapping field is a mapping field of the target source field expression in target format data to be generated.

In a second aspect, embodiments of the present application provide a computer device, comprising: a processor and a memory storing a computer program, which, when run by the processor, implements the steps of the data processing method described above.

In a third aspect, embodiments of the present application provide a computer-readable storage medium having stored therein a computer program which, when executed by a processor, implements the steps of the data processing method described above.

The embodiment of the application provides a data processing method, a computer device and a computer readable storage medium, wherein the method comprises the following steps: acquiring JSON data to be processed; performing format conversion processing on the JSON data to be processed based on a preset rule configuration table to obtain target format data; the preset rule configuration table comprises data analysis logic for converting the JSON field into a field corresponding to the target format. Therefore, the target format data generation logic is configured and managed based on the preset rule configuration table, codes are greatly reduced, and the value process is standardized and simplified, so that the workload of development and maintenance is reduced, the understanding and maintenance are easy, and the data conversion efficiency is improved.

Drawings

FIG. 1 is a schematic flow chart of a data processing method according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of an implementation architecture of a data processing method according to an embodiment of the present invention;

FIG. 3 is a schematic process diagram of a data processing method according to an embodiment of the present invention;

fig. 4 is a schematic structural diagram of a computer device according to an embodiment of the present invention.

Detailed Description

Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present application as detailed in the accompanying claims.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, the element defined by the phrase "comprising one … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element, and furthermore, elements having the same name in different embodiments of the present application may have the same meaning or may have different meanings, a particular meaning of which is to be determined by its interpretation in this particular embodiment or by further combining the context of this particular embodiment.

It should be understood that although the terms first, second, third, etc. may be used herein to describe various information, these information should not be limited by these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope herein. The word "if" as used herein may be interpreted as "at … …" or "at … …" or "responsive to a determination", depending on the context. Furthermore, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context indicates otherwise. It will be further understood that the terms "comprises," "comprising," "includes," and/or "including" specify the presence of stated features, steps, operations, elements, components, items, categories, and/or groups, but do not preclude the presence, presence or addition of one or more other features, steps, operations, elements, components, items, categories, and/or groups. The terms "or" and/or "as used herein are to be construed as inclusive, or meaning any one or any combination. Thus, "A, B or C" or "A, B and/or C" means "any of the following: a, A is as follows; b, a step of preparing a composite material; c, performing operation; a and B; a and C; b and C; A. b and C). An exception to this definition will occur only when a combination of elements, functions, steps or operations are in some way inherently mutually exclusive.

It should be understood that, although the steps in the flowcharts in the embodiments of the present application are shown in order as indicated by the arrows, these steps are not necessarily performed in order as indicated by the arrows. The steps are not strictly limited in order and may be performed in other orders, unless explicitly stated herein. Moreover, at least some of the steps in the figures may include multiple sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, the order of their execution not necessarily occurring in sequence, but may be performed alternately or alternately with other steps or at least a portion of the other steps or stages.

It should be noted that, in this document, step numbers such as S1 and S2 are used for the purpose of more clearly and briefly describing the corresponding contents, and not to constitute a substantial limitation on the sequence, and those skilled in the art may perform S2 first and then S1 when implementing the present invention, which are all within the scope of protection of the present application.

It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.

In the following description, suffixes such as "module", "component", or "unit" for representing elements are used only for facilitating the description of the present application, and are not of specific significance per se. Thus, "module," "component," or "unit" may be used in combination.

Referring to fig. 1, a data processing method provided in this embodiment of the present application may be implemented by a data processing apparatus provided in this embodiment of the present application, where the data processing apparatus may be implemented by software and/or hardware, and in this embodiment, an execution body of the data processing method is taken as an example of a server, a terminal, or other computer device, and the data processing method provided in this embodiment includes:

and S1, acquiring JSON data to be processed.

The JSON data to be processed is JSON data to be converted into a target format, and specifically may be JSON character strings or the like. Here, the source of JSON data may be an HTTP request (e.g., RESTful API of micro-services, etc.).

S2, performing format conversion processing on JSON data to be processed based on a preset rule configuration table to obtain target format data; the preset rule configuration table comprises data analysis logic for converting the JSON field into a field corresponding to the target format.

The target format may be set according to actual needs, including, but not limited to Excel, word, etc., which are not specifically limited herein. It will be appreciated that the JSON data to be processed consists of JSON fields, which may include in particular field names and/or corresponding values. For example, for JSON field "{" age ": '24' }, field name is age, and corresponding value is 24. The preset rule configuration table comprises data analysis logic for converting the JSON field into a field corresponding to the target format, and the data analysis logic can specifically comprise a value rule, a mapping rule and the like. The value rule is used for indicating how to acquire the values of the fields, and the mapping rule is used for indicating how to map the JSON fields into the fields in the target format data to be generated, the layout among the fields and the like.

Based on a parameter object list corresponding to each source field expression and a mapping rule in a preset rule configuration table, performing format conversion processing on the JSON data to be processed to obtain target format data; the mapping rule includes a mapping field of each source field expression in the target format data to be generated.

It can be understood that, since JSON data is represented by JSON fields (i.e., characters), and the target format data to be generated may not be represented by characters, if it is required to convert JSON data into target format data, a source field expression needs to be established in a preset rule configuration table, where each source field expression includes at least one JSON field, and according to the representation manner of each JSON field in the source field expression, the value type, such as an object or an array, of each JSON field can be known. Meanwhile, mapping fields of each source field expression in target format data to be generated and association relations among the mapping fields are also required to be established, so that field mapping, field value and the like are realized.

According to the mapping field of each source field expression in the target format data to be generated, information such as the position of each mapping field or the fields contained in each mapping field in the target format data can be obtained. For example, for mapping the field "basic information. Name", it can be known that the field "name" is an attribute under the field "basic information", the position of the field "name" is below or right of the position of the field "basic information" in the outer report, and so on.

Analyzing each source field expression in the preset rule configuration table to obtain a parameter object list corresponding to each source field expression, wherein the method comprises the following steps:

creating a parameter object corresponding to each parameter based on the attribute of at least one parameter; the parameter object comprises a parameter value type and a parameter identifier;

The first preset character may be set according to actual situation, for example, may be a character "", etc. In any source field expression, different JSON fields are distinguished by the first preset character, so that at least one JSON field in each source field expression can be separated into at least one parameter according to the first preset character contained in each source field expression. For example, taking the source field expression as "baseInfo. Sex" as an example, the JSON field in the source field expression may be partitioned into two parameters, "baseInfo" and "sex", respectively. Then, a parameter object corresponding to each parameter can be created based on the attribute of at least one parameter, and the parameter object comprises a parameter value type and a parameter identification. Further, a parameter object list corresponding to each source field expression is generated according to the parameter object corresponding to each parameter.

Here, the parameter value type and the parameter identifier may be obtained by an attribute of the parameter, for example, may be obtained according to a representation manner of the parameter itself. The parameter identification may be the parameter itself, and may specifically be represented by a parameter name. The parameter value type is used for representing whether the parameter is an array or an object, and can be obtained specifically according to whether the parameter contains preset characters or not. Therefore, the format conversion processing is more convenient for the JSON data to be processed by acquiring the parameter object list corresponding to each source field expression, and the data conversion efficiency is further improved.

detecting whether the target parameter contains a second preset character or not, wherein the target parameter is any parameter;

The second preset character may be set according to actual needs, for example, may be a character "[" ] ", etc. Specifically, whether the target parameter contains a second preset character or not is checked, when the target parameter contains the second preset character, the parameter value type of the target parameter is determined to be an array, and when the target parameter does not contain the second preset character, the parameter value type of the target parameter is determined to be an object, and then a parameter object corresponding to the target parameter is created based on the parameter value type and the parameter identification of the target parameter. For example, a JsonFiledInfo object may be predefined to represent a parameter object that contains two attributes: taking the target parameter as an example, when the parameter value type of the target parameter is an array (i.e., array), the JsonFiledInfo object of the target parameter may be marked as JsonFiledInfo (type=array, field=age), and when the parameter value type of the target parameter is an object (i.e., object), the JsonFiledInfo object of the target parameter may be marked as JsonFiledInfo (type=object, field=age). Therefore, the parameter object corresponding to the target parameter can be quickly and accurately created, and the data conversion efficiency is further improved.

In an embodiment, the performing format conversion processing on JSON data to be processed based on the parameter object list corresponding to each source field expression and the mapping rule in the preset rule configuration table to obtain target format data includes:

acquiring values corresponding to the source field expressions from the JSON data to be processed based on the parameter object list corresponding to the source field expressions;

according to a mapping rule in a preset rule configuration table, determining mapping fields of each source field expression in target format data to be generated and positions of each mapping field in the target format data to be generated;

and generating the target format data according to the values of the mapping fields and the positions of the mapping fields in the target format data to be generated.

The parameter object list corresponding to each source field expression contains the parameter objects corresponding to each JSON field constituting the corresponding source field expression, and the parameter objects include the parameter value type and the parameter identifier, so that the value corresponding to each source field expression can be obtained from the JSON data to be processed based on the parameter object list corresponding to each source field expression.

It will be appreciated that, since the mapping field of each of the source field expressions in the target format data to be generated is specified in the mapping rule, the mapping field of each of the source field expressions in the target format data to be generated can be directly determined. For example, taking the case that the source field expression is "baseinfo.customername", if the mapping field corresponding to the source field expression "baseinfo.customername" set in the preset rule configuration table is "basic information. Name", it may be determined that the mapping field corresponding to the source field expression "baseinfo.customername" is "basic information. Name". It should be noted that, since the mapping field may be formed by two or more fields, the same field may exist between any two mapping fields, and there is an association relationship between fields included in the same mapping field, so that the position of each mapping field in the target format data to be generated may be determined according to the above information. Further, the value of the mapping field may be regarded as the value of a field contained in the mapping field. For example, the mapping field "basic information, name" includes two fields of "basic information" and "name", the mapping field "basic information, gender" includes two fields of "basic information" and "gender", because the field "basic information" is the upper concept of the field "name" and "gender" and both mapping fields contain the field "basic information", the field "basic information" can be regarded as an attribute, the field "name" and "gender" can be regarded as an element under the field "basic information", at this time, the position of the mapping field "basic information, name" in the target format data to be generated can be regarded as the position of the field "basic information" and the field "name" in the target format data to be generated, respectively, and the value of the mapping field "basic information, name" can be regarded as the value of the field "name". Therefore, the mapping fields of the source field expressions in the target format data to be generated and the positions of the mapping fields in the target format data to be generated are determined, and then the target format data is generated according to the values of the mapping fields and the positions of the mapping fields in the target format data to be generated, so that the value process is standardized and simplified, and the maintainability is further improved.

responding to the parameter object list corresponding to the source field expression to only contain one parameter object, acquiring the value of the JSON field matched with the parameter object from the JSON message data to be processed, and determining the value of the JSON field matched with the parameter object as the value corresponding to the source field expression;

Optionally, when the parameter object list corresponding to the source field expression only includes one parameter object, the value of the JSON field matched with the parameter object may be directly obtained from the JSON data to be processed, and the value of the JSON field matched with the parameter object may be determined as the value corresponding to the source field expression. The obtaining of the value of the JSON field matched with the parameter object from the JSON data to be processed may be determining the value of the JSON field matched with the parameter identifier in the JSON message data to be processed according to the parameter identifier in the parameter object. For example, taking the source field expression as "id" as an example, assuming that the corresponding parameter object list includes only the parameter object "JsonFiledInfo (type=object, field=id)", 0210067196 may be directly determined as the value corresponding to the source field expression if "id:"0210067196 "is recorded in JSON data to be processed.

Optionally, when the parameter object list corresponding to the source field expression includes at least two parameter objects, traversing each parameter object according to a preset value rule to obtain a value of a JSON field matched by each parameter object, and further determining a value corresponding to the source field expression according to the value of the JSON field matched by each parameter object. The preset value rule may be set according to actual needs, and includes directly taking a single value, directly taking multiple values, taking a single value according to a rule, taking a group value according to a rule, and the like, which is not limited herein. For example, for the fields of path1.Path2.Value or path1[0]. Path2[1]. Value, the single value may be directly taken according to a fixed path. Therefore, the method can accurately and rapidly acquire the value corresponding to each source field expression from the JSON data to be processed, and further improves the data conversion efficiency.

In an embodiment, traversing each parameter object according to a preset value rule to obtain a value of a JSON field matched with each parameter object, including:

It can be understood that, because the parameter identifiers in the parameter objects are related to each other in the parameter object list corresponding to the source field expression, after the value of the JSON field matched with the previous parameter object is obtained, the value of the JSON field matched with the next parameter object can be determined based on the value of the JSON field matched with the previous parameter object, that is, the value of the JSON field matched with the next parameter object is obtained from the value of the JSON field matched with the previous parameter object. For example, it is assumed that a parameter object list corresponding to the source field expression "addressList [ addresstype=id ]. Providengenname" includes three parameter objects, which are JsonFiledInfo (type=object, field=addresslist), jsonFiledInfo (type=array, field=addresstype=id), and JsonFiledInfo (type=object, field=providenname), respectively, if the value of the JSON field matched by the parameter object JsonFiledInfo (type=object, field=addresslist) is "addressList": the values of JSON fields matched by the second parameter object JSON field are { address type ": id", "priority Name": AA "," priority Name ": a", "districtName": B region "," detail ": north of C street" }, { "address type": office "," priority Name ": a region", "priority Name": C street side "}, { a city", "priority Name": B region "," priority ": D subway station" }, and the third parameter object JSON file info (type=array, field=address type=id) are the values of JSON fields matched by { "address type": id "," priority Name ": priority", "priority Name": a city "," priority Name ": a region", "priority": C street side "}, and the third parameter object JSON file info=jet=north of JSON field. Therefore, the value of the JSON field matched with each parameter object can be rapidly and accurately determined, and the data conversion efficiency is further improved.

determining a mapping value matched with a value corresponding to the target source field expression in a preset rule configuration table according to the value corresponding to the target source field expression; the target source field expression is any source field expression;

It can be appreciated that, due to different conversion requirements, the value corresponding to the source field expression obtained from the JSON data to be processed may not be directly added to the target format data to be generated, but needs to be added after the display style or the display format is changed correspondingly, for example, the value corresponding to the source field expression needs to be replaced by the data with the same meaning or special characters so as to be added to the target format data to be generated. For example, assuming that the target source field expression is "baseinfo. Sex", the value corresponding to the target source field expression is "M", if the mapping value matched by the value "M" corresponding to the target source field expression in the preset rule configuration table is "male", and the mapping field of the target source field expression in the target format data to be generated is "basic information. Sex", the "male" is determined as the value of the mapping field "basic information. Sex". Thus, the required target format data can be generated based on the requirement, and the data conversion efficiency and the adaptability are further improved.

In summary, in the data processing method provided in the foregoing embodiment, the target format data generation logic is configured and managed based on the preset rule configuration table, so that the code is greatly reduced, and the value process is standardized and simplified, so that the workload of development and maintenance is reduced, the understanding and maintenance are easy, and the data conversion efficiency is improved.

Based on the same inventive concept as the foregoing embodiments, the foregoing embodiments are described in detail below by way of a specific example, where the source field expression is used as a source field, the parameter object is used as a jsonfiledbfo object, and the target format data is used as Excel report data.

The Excel report is a data form with good readability, and is convenient for users to review data. Usually, the data is sourced from different organizations, systems or modules, and the data standards of different environments are different, so that certain conversion and combination are needed, and finally, the data is displayed in a report format suitable for reading. Considering that the data source interface has been used in advance for production and that report requirements are typically set forth by users after the system has been in use for a period of time, it is necessary for developers to understand the existing data query interface, to process logic for each field in the report, and to complete the report generation logic in a hard-coded manner (i.e., program code).

However, when the report covers a large number of data items, the report generated by hard-coding has the following problems:

1) The code is mechanically repeated: a few fields are simply obtained data and then written into a report.

2) The post-maintenance performance is poor: repeatedly stacking values and assignment logic in codes, and solving the problem of complicated positioning; when the report is regulated, the codes are required to be modified, and the system is redeployed on line.

3) The developer has difficulty in docking: because of the many data query interfaces, the response formats of the interfaces, the analysis logic and the data conversion (such as dictionary codes and Chinese mapping) modes need to be communicated by taking the current developer as the center, and the efficiency is low.

Therefore, how to alleviate the code stacking problem and improve the later maintainability under the existing business background and realize the efficient communication of developers and the rapid development of reports is a challenging problem.

In order to solve the above-mentioned problems, the present embodiment provides a data processing method, referring to fig. 2, which is a schematic diagram of an implementation architecture of the data processing method according to the present embodiment, and mainly includes the following parts: and data processing and visual management interfaces outside the system. For the off-system data part, a series of API query interfaces are used to generate reports, requiring that the transmission be in JOSN format, such as a micro-service HTTP interface. For the data processing part, firstly reading information such as a value rule, a mapping rule, an Excel layout and the like from a configuration rule table, filtering data from an API response according to the value rule, converting the data into final data according to the mapping rule, generating corresponding Excel elements, and generating an Excel report file after processing complete data. If the configuration table cannot describe the complex logic data processing, the corresponding processing can be completed through the custom processing logic. And the visual management interface is used for performing addition, deletion, examination and management on the configuration rule table data, and downloading the generated Excel report for previewing.

100 for the elements in the configuration rule table, the actions and examples of the elements and the elements are shown in table 1, and the BNF description of the value rule corresponding to the source field is shown in table 2. Wherein, the value rule is configured as follows:

1) Directly taking a single value: JSON data may be obtained from a fixed path, such as path1.Path2.Value or path1[0] path2[1] value.

2) Directly taking multiple values: JSON data may obtain values according to a fixed path, and need to take out multiple values and splice the values to obtain a result, for example, path1.Path2.Value1, value2, value3 or path1[0] path2[1], value1, value2, value3, with the result that value1, value2, value3 are directly spliced together.

3) Rule takes single value: the JSON data needs to be filtered, and only the first piece of data meeting the condition is taken. For example, path [ type=a, number >2]. Value.

4) Rule group value: the JSON data needs to be filtered, and a set of values meeting the conditions need to be taken out, for example, path [ @ type=a, number >2]. Value, "@" indicates that all values meeting the conditions are taken out, and the result is spliced by using an english comma.

5) Combinations of the above rules: for example, path1[0], path2[ @ type=a, number >2]. Value1, value2, value3.

TABLE 1

TABLE 2

The data processing method provided in this example is described in detail below with reference to the rule configuration table shown in table 3 and the JOSN data example shown in table 4. Referring to fig. 3, the data processing method provided in this embodiment mainly includes the following steps:

TABLE 3 Table 3

TABLE 4 Table 4

1. Pretreatment of

1.1. Defining a JsonFiledInfo object, which contains two attributes: type (record parameter value type, which can only be an object or an array) and field (record parameter itself).

1.2. The expression (e.g., source field in table 3) is partitioned into multiple parameters according to "," to obtain a parameter list.

1.3. Traversing the parameter list, if the parameter comprises "[", "]", the type of the parameter is an array (array), otherwise, the parameter is an object (object), a corresponding JsonFiledInfo object is created for the parameter, and the JsonFiledInfo list corresponding to the expression is added.

For example, the expression addressList [0] in table 3, providence name, results from parsing as follows, the JsonFiledInfo list contains three elements (i.e., jsonFiledInfo objects), respectively:

JsonFiledInfo(type＝object,field＝addressList)

JsonFiledInfo(type＝array,field＝0)

JsonFiledInfo(type＝object,field＝provinceName)

1.4. and returning a JsonFiledInfo list.

2. Rule parsing

2.1. A list of JSON filedbnfos and JSON strings to be processed (e.g., JSON data shown in table 4) are entered.

2.2. If the JSOnFiledInfo list is null or the JSON string is null, a null string is returned.

2.3. If the JSOnFiledInfo list has only one element, the value of the JSON object specification field is obtained from field.

2.4. If the JSOnFiledInfo list has a plurality of elements, reading a first element, acquiring a corresponding JSON object or array according to the type and field of the element, and recursively calling the JSON object or array after the subsequent JSOnFiledInfo list and the parsed JSON object or array.

For example, if the parsed JsonFiledInfo list contains three elements, first, the value of the JSON string per object address list is parsed from the first element "JsonFiledInfo (type=object, field=address list)", to obtain:

next, the above result is indexed by an array to a value of 0 according to the second element JsonFiledInfo (type=array, field=0), resulting in:

then, the result is taken as the value of providenName by object based on the third element JsonFiledInfo (type=object, field=providenName), to obtain "A market"

2.4. The acquired value is returned.

3. Rule matching

3.1. Rule matching is performed each time a JSOnFiledInfo element is processed in rule parsing, if the field of the current JSOnFiledInfo element contains "=", ">", "<", I! = "operator, then the operator is partitioned into a plurality of conditions according to the inclusion.

For example, the jsonfoldinfo element jsonfoldinfo (type=array, field= type=shareholder, shareholder ratio > 10) obtained by parsing is divided into type=shareholder and shareholder ratio >10 corresponding to the source field releate=shareholder, shareholder ratio >10, meaning that the value of type is equal to shareholder, and shareholder ratio is greater than the number 10.

3.2. Traversing the JSON array, matching the JSON object according to the condition described by the operator, and storing the matched first index into the map.

For example, for "JSON filedbfo (type=object, field=releatelist)", the resultant JSON is:

under the condition type=shareholder and shareholder ratio >10 constraint, the index stored in map is 0.

3.3. If field of the current JsonFiledInfo element contains "@", querying all indexes meeting the condition, otherwise querying the first index meeting the condition. Based on the above element "jsonfiledbfo (type=object, field=releatelist)", if field contains "@", the indexes stored in the map are 0 and 1.

3.4. If there is only one index, field of the current JsonFiledInfo is set to that index. For example, jsonfiledbfo (type=array, field= shareholder, shareholder ratio > 10) is set to jsonfiledbfo (type=array, field=0).

3.5. If there are multiple indexes, field of the current JsonFiledInfo is set to multiple indexes and true is returned. For example, jsonFiledInfo (type=array, field= @ type=shareholder, shareholder ratio > 10) may be set to JsonFiledInfo (type=array, field=0, 1).

3.6. If there is no match to any index, false is returned.

4. Result stitching

4.1. And in rule matching, the last element in the JsonFiledInfo list is subjected to result splicing processing.

4.2. If the field of the element contains ",", then it is partitioned into multiple fields according to ",". For example, for "addressList [ addresstype=id ]. Providenname, cityName, distributname, detail", the last element in the JsonFiledInfo list is JsonFiledInfo (type=object, field=providenname, cityName, distributname, detail), and four fields are included.

4.3. Traversing the field list, acquiring the value of the designated field name according to field, and splicing the acquired value into a character string to return. For example, the result obtained in the above example is concatenation of four field values, i.e., the north side of C street in B region of a city a, i.e., providence name, cityName, districtName, and detail.

4.4. If the field of the element does not contain ",", the value of the field name specified in accordance with the field is directly obtained.

Referring to table 5, an Excel report is obtained according to the JSON example shown in table 3.

TABLE 5

In summary, in the data processing method provided in this embodiment, information such as a value rule, a mapping rule, an Excel layout and the like is defined through a configuration rule table, so as to implement configuration and management of report generation logic; the method comprises the steps of preprocessing JSON data, rule analysis, rule matching, result splicing and the like, so that the value of a specified field is extracted from the JSON data; meanwhile, a language for describing the JSON data value-taking logic is provided, the value-taking process is standardized and simplified, and configuration personnel can conveniently perform configuration and maintenance. Has the following advantages: 1) Reducing the code amount: by configuring information such as a value rule, a mapping rule, an Excel layout and the like, the traditional hard coding mode is replaced, and the code quantity is greatly reduced. Therefore, the development efficiency can be improved, and the workload of code maintenance can be reduced. 2) Improving maintainability: by standardizing and simplifying the value process, the code logic is clearer and easy to understand and maintain. When the report needs to be adjusted, only the configuration rule table needs to be modified, and codes do not need to be modified and the system does not need to be deployed again, so that the maintainability of the later stage is greatly improved. 3) Efficient communication and rapid development: the online configuration maintenance platform is provided, report developers and personnel of an interface provider can modify the configuration in real time through the platform, provide respective information such as dictionary mapping relation and field value logic, complete rule configuration together, and reduce communication cost.

Based on the same inventive concept as the previous embodiments, an embodiment of the present invention provides a computer apparatus, as shown in fig. 4, including: a processor 310 and a memory 311 in which a computer program is stored; the number of the processors 310 illustrated in fig. 4 is not used to refer to one number of the processors 310, but is merely used to refer to a positional relationship of the processors 310 with respect to other devices, and in practical applications, the number of the processors 310 may be one or more; likewise, the memory 311 illustrated in fig. 4 is also used in the same sense, that is, only to refer to the positional relationship of the memory 311 with respect to other devices, and in practical applications, the number of the memories 311 may be one or more. The above described data processing method is implemented when the processor 310 runs the computer program.

The apparatus may further include: at least one network interface 312. The various components in the device are coupled together by a bus system 313. It is appreciated that the bus system 313 is used to enable connected communication between these components. The bus system 313 includes a power bus, a control bus, and a status signal bus in addition to the data bus. But for clarity of illustration the various buses are labeled as bus system 313 in fig. 4.

The memory 311 may be a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memories. Wherein the nonvolatile Memory may be Read Only Memory (ROM), programmable Read Only Memory (PROM, programmable Read-Only Memory), erasable programmable Read Only Memory (EPROM, erasable Programmable Read-Only Memory), electrically erasable programmable Read Only Memory (EEPROM, electrically Erasable Programmable Read-Only Memory), magnetic random access Memory (FRAM, ferromagnetic random access Memory), flash Memory (Flash Memory), magnetic surface Memory, optical disk, or compact disk Read Only Memory (CD-ROM, compact Disc Read-Only Memory); the magnetic surface memory may be a disk memory or a tape memory. The volatile memory may be random access memory (RAM, random Access Memory), which acts as external cache memory. By way of example, and not limitation, many forms of RAM are available, such as static random access memory (SRAM, static Random Access Memory), synchronous static random access memory (SSRAM, synchronous Static Random Access Memory), dynamic random access memory (DRAM, dynamic Random Access Memory), synchronous dynamic random access memory (SDRAM, synchronous Dynamic Random Access Memory), double data rate synchronous dynamic random access memory (ddr SDRAM, double Data Rate Synchronous Dynamic Random Access Memory), enhanced synchronous dynamic random access memory (ESDRAM, enhanced Synchronous Dynamic Random Access Memory), synchronous link dynamic random access memory (SLDRAM, syncLink Dynamic Random Access Memory), direct memory bus random access memory (DRRAM, direct Rambus Random Access Memory). The memory 311 described in embodiments of the present invention is intended to comprise, without being limited to, these and any other suitable types of memory.

The memory 311 in the embodiment of the present invention is used to store various types of data to support the operation of the device. Examples of such data include: any computer program for operating on the device, such as an operating system and application programs; contact data; telephone book data; a message; a picture; video, etc. The operating system includes various system programs, such as a framework layer, a core library layer, a driver layer, and the like, for implementing various basic services and processing hardware-based tasks. The application programs may include various application programs such as a Media Player (Media Player), a Browser (Browser), etc. for implementing various application services. Here, a program for implementing the method of the embodiment of the present invention may be included in an application program.

Based on the same inventive concept as the previous embodiments, the present embodiment further provides a computer readable storage medium having a computer program stored therein, where the computer readable storage medium may be a Memory such as a magnetic random access Memory (FRAM, ferromagnetic random access Memory), a Read Only Memory (ROM), a programmable Read Only Memory (PROM, programmable Read-Only Memory), an erasable programmable Read Only Memory (EPROM, erasable Programmable Read-Only Memory), an electrically erasable programmable Read Only Memory (EEPROM, electrically Erasable Programmable Read-Only Memory), a Flash Memory (Flash Memory), a magnetic surface Memory, a compact disc, or a compact disc Read Only Memory (CD-ROM, compact Disc Read-Only Memory); but may be a variety of devices including one or any combination of the above-described memories, such as a mobile phone, computer, tablet device, personal digital assistant, or the like. The computer program stored in the computer readable storage medium realizes the data processing method when being executed by a processor. The specific step flow implemented when the computer program is executed by the processor is described with reference to the embodiment shown in fig. 1, and will not be described herein.

The technical features of the above-described embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above-described embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.

In this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a list of elements is included, and may include other elements not expressly listed.

The foregoing is merely illustrative of the present invention, and the present invention is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present invention. The scope of the invention should therefore be protected by the following claims.

Claims

1. A method of data processing, comprising:

acquiring JSON data to be processed;

2. The method according to claim 1, wherein the performing format conversion processing on the JSON data to be processed based on the preset rule configuration table to obtain target format data includes:

3. The method according to claim 2, wherein the parsing each source field expression in the preset rule configuration table to obtain a parameter object list corresponding to each source field expression includes:

Creating a parameter object corresponding to each parameter based on the attribute of the at least one parameter; the parameter object comprises a parameter value type and a parameter identifier;

4. A method according to claim 3, wherein creating a parameter object corresponding to each parameter based on the attribute of the at least one parameter comprises:

5. The method according to any one of claims 2 to 4, wherein the performing format conversion processing on the JSON data to be processed based on the parameter object list corresponding to each source field expression and the mapping rule in the preset rule configuration table to obtain target format data includes:

6. The method according to claim 5, wherein the obtaining, based on the parameter object list corresponding to each source field expression, the value corresponding to each source field expression from JSON data to be processed includes at least one of:

7. The method of claim 6, wherein traversing each of the parameter objects according to a preset value rule to obtain a value of a JSON field that each of the parameter objects matches comprises:

8. The method of claim 5, wherein determining the value of each of the mapped fields from the value corresponding to each of the source field expressions comprises:

9. A computer device, comprising: a processor and a memory storing a computer program, which, when run by the processor, implements the steps of the data processing method of any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that a computer program is stored, which, when being executed by a processor, implements the steps of the data processing method of any of claims 1 to 8.