CN116932519A - Object conversion method, system, electronic equipment and storage medium based on XML template - Google Patents

Object conversion method, system, electronic equipment and storage medium based on XML template Download PDF

Info

Publication number
CN116932519A
CN116932519A CN202310804401.1A CN202310804401A CN116932519A CN 116932519 A CN116932519 A CN 116932519A CN 202310804401 A CN202310804401 A CN 202310804401A CN 116932519 A CN116932519 A CN 116932519A
Authority
CN
China
Prior art keywords
xml
data
cleaning
template
field
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310804401.1A
Other languages
Chinese (zh)
Inventor
李奔
彭磊
杨志文
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuhan Zhongbang Bank Co Ltd
Original Assignee
Wuhan Zhongbang Bank Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuhan Zhongbang Bank Co Ltd filed Critical Wuhan Zhongbang Bank Co Ltd
Priority to CN202310804401.1A priority Critical patent/CN116932519A/en
Publication of CN116932519A publication Critical patent/CN116932519A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21Design, administration or maintenance of databases
    • G06F16/215Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • G06F16/2282Tablespace storage structures; Management thereof
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23Updating
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/242Query formulation
    • G06F16/2433Query languages
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/80Information retrieval; Database structures therefor; File system structures therefor of semi-structured data, e.g. markup language structured data such as SGML, XML or HTML
    • G06F16/84Mapping; Conversion
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

The invention provides an object conversion method, a system, electronic equipment and a storage medium based on an XML template, wherein the main scheme comprises the steps of obtaining json messages of table operation information of other databases; filtering the data source of the JSON message through a data source filtering tag of the XML rule template, screening out data to be cleaned, and cleaning the data to be cleaned through an XML rule template data cleaning rule to obtain data information to be operated; selecting a table name to be operated and a field in the table as an XML source data service main key in data information to be operated, and defining the table name and the field in the table as XML source data service main keys in an XML rule template; and associating the XML source data service main key with the table and the field of the local database to generate an SQL sentence which can perform insertion, deletion, updating and query operations on the table corresponding to the local database. The invention aims to solve the problems that hard-coded data cleaning, processing and coding are completed and the service is required to be restarted, and the problems that the extended service scene is cleaned and the code is required to be repeatedly developed and the like.

Description

Object conversion method, system, electronic equipment and storage medium based on XML template
Technical Field
The present invention relates to the field of information technologies, and in particular, to an object transformation method, system, electronic device, and storage medium based on an XML template.
Background
Object transformation and data cleansing are frequently encountered problems in current software system architectures. Today, the network is developed abnormally, interface call of cross-system and service is frequently encountered, and data in other systems are converted and stored into the system so that the system can handle business transaction. In combination with the specificity of other system data structures, the system is cleaned to be the required data format of the system. The data cleaning and converting method is developed for different databases, different table structures and table fields, and cleaning is an ideal field attribute.
Because of a large number of interface calls in the internet and financial industries, data requests are required. Different organizations and businesses have different data structure differences. For example, the method has the advantages that the xx credit products of the A mechanism exist, the xxx credit products of the B mechanism are summarized into the statistical report of the business system, and besides the normal communication cost and the interface debugging cost, the method can be used for cleaning and converting the object data based on the XML template, and the problems that hard code data cleaning, processing and coding are required to restart services and the problems that the business scene cleaning is expanded and codes are required to be repeatedly developed are solved through configuring cleaning messages to request data. The flexibility is greatly improved, the service expansion is simple, and the maintenance and operation cost is greatly reduced.
Disclosure of Invention
The invention aims at solving the problems of hard-coded data cleaning and processing, restarting service for coding completion, expanding cleaning service scenes, repeatedly developing codes and the like.
According to a first aspect of the present invention, there is provided an object transformation method based on an XML template, comprising the steps of:
step 1, acquiring json messages containing operation information of other databases in the data bus;
step 2, filtering the data source of the json message through the data source filtering tag of the XML rule template, and screening out the data to be cleaned;
step 3, cleaning data to be cleaned through XML rule template data cleaning rules to obtain data information to be operated;
step 4, selecting a name of a table to be operated and a field in the table as an XML source data service main key in the data information to be operated, and defining the name and the field in the table in an XML rule template;
and step 5, associating the XML source data service primary key with a table and a field of the local database to generate a corresponding cleaned SQL sentence.
On the basis of the technical scheme, the invention can also make the following improvements.
Optionally, the json message including the table operation information of other databases of the acquired data bus includes: tags defining XML rule templates.
Optionally, the filtering the data source of the JSON message by the data source filtering tag of the XML rule template, and screening the data to be cleaned includes: XML source data filter tags are defined.
Optionally, in step 3, the cleaning the data to be cleaned by using the XML rule template data cleaning rule, and obtaining the data information to be operated includes: defining XML source data cleaning rules; the data cleaning rule comprises the following steps:
s3.1, simple cleaning rules: analyzing the received json message, and copying the appointed field to a corresponding value;
s3.2, complicated cleaning rules: analyzing the received json message, and performing mathematical operation, type conversion or logic processing on the designated field to obtain a corresponding value;
s3.3, custom cleaning rules: analyzing the received json message, expanding the attribute by a user-defined function method, and obtaining a corresponding value.
Optionally, the simple cleaning rule is as follows:
declaring a simple mapping relation simple tag, which contains mapping description and is used for remarking the desc attribute of the current simple mapping relation;
the attribute value relation of the result table of the field mapping in the request parameter table is declared, and the attribute is contained:
col: a statement purge target table field;
the relation value: the request source table field is declared.
Optionally, the complex cleaning rule is as follows:
declaring a simple mapping relationship complex tag;
the field mapping result table mapping relation label in the request parameter table is declared to contain attributes:
col: declaring a purge target table field;
declaring a complex mapping process rule rel tag, containing attributes:
handle: declaring a processing mode;
ref-type: declaring a value type;
data-Type: declaring a value type or a conversion mode;
rel tag value: a request parameters table field representing a flush.
Optionally, the custom cleaning rule is as follows:
< queryIndex >: the mapping table query condition is declared, and at least one < or > sub-tag is included, and the attribute includes the following: desc, mapping description, which is used for remarking the current simple mapping relation;
< or >: declaring a query condition mapping relationship, having at least one < rel > sub-tag, the attribute comprising:
rel: and declaring a corresponding attribute value field in the mapping result table.
According to a second aspect of the present invention, there is provided an XML template-based object conversion system, comprising:
the XML rule template label definition module is used for acquiring json messages containing other database table operation information in the data bus;
the XML source data filtering tag definition module is used for filtering the data source of the json message through the data source filtering tag of the XML rule template, and screening out the data to be cleaned;
the XML source data cleaning rule definition module is used for cleaning the data to be cleaned through XML rule template data cleaning rules to obtain data information to be operated;
the XML source data query condition definition module is used for selecting a table name to be operated and a field in the table as an XML source data service main key in data information to be operated and defining the table name and the field in the table as an XML rule template;
and the SQL sentence generating module is used for associating the XML source data service main key with the table and the field of the local database to generate a corresponding cleaned SQL sentence.
According to a third aspect of the present invention there is provided an electronic device comprising a memory, a processor for implementing the steps of an XML template based object conversion method when executing a computer program stored in the memory.
According to a fourth aspect of the present invention, there is provided a computer readable storage medium having stored thereon a computer program which when executed by a processor implements the steps of an XML template based object transformation method.
The invention has the technical effects and advantages that:
the invention provides an object conversion method, a system, electronic equipment and a storage medium based on an XML template, which can automatically update service system data by using a cleaning engine and only throwing the operation of a corresponding table to kafka and enabling the service system to access the kafak to call the cleaning engine. In addition, because of the expansibility of the service, in the service scene of frequent change of transaction data, the difficulty of operation is increased, the investment of manpower is increased, and if the service is required to be expanded, the service of other institutions is increased, and recoding and debugging are performed. The data cleaning method engine is used, so that great help can be obtained in the deployment of the server, the expansion of the service and the maintenance of the operation.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention may be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
Drawings
FIG. 1 is a logical block diagram of an object conversion method based on an XML template according to an embodiment of the present invention;
FIG. 2 is a diagram illustrating a source object json message input example provided by an embodiment of the present invention;
FIG. 3 is a diagram illustrating an XML template configuration example provided in an embodiment of the present invention;
fig. 4 is a diagram illustrating SQL output provided by an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Because of a large number of interface calls in the internet and financial industries, data requests are required. Different organizations and businesses have different data structure differences. (for example, there are xx credit products of a mechanism and summarized into a statistical report of a business system) besides normal communication cost and interface debugging cost, the embodiment of the invention aims to solve the problems of hard-coded data cleaning and processing, the problem that the service needs to be restarted after coding is completed, and the problem that codes need to be repeatedly developed after expanding business scene cleaning.
It can be appreciated that, based on the defects in the background technology, the embodiment of the invention provides an object conversion method based on an XML template, specifically as shown in fig. 1, which comprises the following steps:
step 1, acquiring json messages containing operation information of other databases in the data bus;
in the above technical solution, the json report that includes the table operation information of other databases in the acquired data bus specifically includes: defining tags of XML rule templates;
the tags of the XML rule templates are as follows:
< nodes >: a root node, which does not contain attributes, does not contain other types of labels, and must or at least contains a < node > sub-label;
< node >: the cleansing node must or at least contains a < mapping > sub-label, the attributes comprising:
clearresult: SQL table names representing output cleansing;
< mapping >: matching tags, must include sub-tags: < references >, < queryIndex >. There is no or only one sub-tag: < filters >. The attributes include the following:
tableName: a table name (source table name) indicating a request in the input parameter;
in the above technical solution, as shown in fig. 2, the field value rule descriptor in step S1 is as follows:
tableName: matching a tableName in a request parameter Json attribute;
clearresult: a results table showing the final cleaning;
business rules:
clearResult can only have one configuration in one configuration file (namely, a cleaning target table only needs to have one clearResult configuration at most); a tableName may have multiple configurations (meaning that it is flushed into different business tables (clearresult) according to different business scenarios (filter properties); processing logic:
when the project is started, converting xml configuration into a Java object NodeTablesVOs through an xml parsing tool Digester;
after the kafka request is acquired, the Json character strings are converted into maps, and corresponding node configuration sets List < NodeTableVO > are acquired from NodeTableVOs through the tableName acquisition in the request parameters.
Step 2, filtering the data source of the json message through the data source filtering tag of the XML rule template, and screening the data to be cleaned, which need to be cleaned;
in the above technical solution, the filtering the data source of the JSON message by the data source filtering tag of the XML rule template, and screening the data to be cleaned specifically includes: defining XML source data filtering labels; the XML source data filter tags are as follows:
< filters >: the filter configuration parent tag configures the filter attribute in the request parameter. There is no or at least one < or > sub-tag. The attributes include the following:
desc, filtering description, which is used for remarking the current filtering relation;
< or >: a filter group configuration tag having at least one < filter > sub-tag;
< filter >: filtering attribute configuration, without sub-tags. The attributes include the following:
source: declaring a table value field in the request parameter;
compare: declaring a comparison type, such as: =sum-! = (support equals and does not equal both);
filter value: filtering the matched attribute values;
the field value rule descriptor is as follows:
source: declaring a table value field (acquired in a request message Json and defaulting to an attribute after an after node is acquired) in a request parameter;
compare: declaring a comparison type, such as: =sum-! = (support equals and does not equal both); business comparison description: the attributes in request Json are of the tape type (e.g., the tape attribute value of the String type will be double quoted (") and the number type will not be quoted (where the decimal number can be matched from the value))
String comparison logic:
when the filter value configures null, the source attribute value acquired in the entry is "", and true is returned;
when the filter value is not null, =compare equalils value, ++! =inverse equal comparison;
digital comparison logic:
integer type
When the source attribute gets null, =return false, ++! Return true;
when the source attribute fetch is not null, the transition Integer comparison returns = with-! A value of =;
double type
When the source attribute gets null, =return false, ++! Return true;
when the source property get is not null, the transition to BigDecimal (precision bit 4) compares back = with-! Values of =comparison;
filter value: filtering the matched attribute values;
business rules:
the sub-labels filter in the or label, a plurality of relationships among the or labels (namely, the filter in the or label satisfies one, and the current or filtration satisfies the condition);
the or labels are in an and relationship (namely, a node configuration filters contains a plurality of peer or labels, and matching values of all the or labels need to be satisfied simultaneously);
processing logic:
judging whether List < NodeTableVO > is empty (or 0 in size), ending cleaning, and prompting that cleaning configuration does not exist; not 0, traversing List < NodeTableVO > set;
obtaining a filtering configuration List < NodeTableF filter OrsVO > in NodeTableVO, traversing the List < NodeTableF filter OrsVO > (filtering and passing when the size is null or 0, and prompting that no corresponding filtering value exists);
obtaining a filters collection List < NodeTableF files OrVO > in NodeTableF files OrsVO;
traversing List < NodeTableFotersOrVO >, acquiring corresponding xml configuration according to NodeTableFotersOrVO, acquiring a source value (acquisition type) in a request parameter, acquiring a configured xml filter value, matching the filter type in the xml, and returning true if the matching is the matching, and performing break traversal;
in step iii, the comparison result of NodeTableFolltersOrsVO returns to step ii, and if a false appears, the loop of step ii is ended, and the filtration is not passed. Then traversing List < NodeTableVO > in step i;
the whole List < NodeTableFolumtersOrsVO > passes through, and returns the traversing value NodeTableVO in the current step i, namely the currently matched xml cleaning configuration.
Step 3, cleaning data to be cleaned through XML rule template data cleaning rules to obtain data information to be operated;
in the above technical solution, the cleaning the data to be cleaned by the XML rule template data cleaning rule, and obtaining the data information to be operated specifically includes: defining XML source data cleaning rules;
the XML source data cleaning rule is as follows:
< relationships >: the mapping relationship is declared. There is no or at least one < simple >, < complex >, < customer-define > sub-tag. The attributes include the following:
desc, filtering description, which is used for remarking the current filtering relation;
further, the defining the XML source data cleansing rule specifically includes the following substeps:
s3.1, simple cleaning rules (attribute direct mapping);
that is, parse the received json message and copy the specified fields to the corresponding values
Defining a label:
< simple >: a simple mapping relationship is declared, with no or at least one < relation > sub-label. The attributes include the following:
desc, mapping description, which is used for remarking the current simple mapping relation;
< relation >: the request parameter table field maps result table attribute values are declared without word tags. The attributes include the following:
col: declaring a purge target table field;
the relation value: declaring a request source table field;
business rules:
one simple label contains a plurality of relation sub-labels, the corresponding col is the final cleaning target, and the relation sub-label value is the attribute field of the after-table node of the source table.
Processing logic:
traversing the NodeTableVO attribute after the filtering, and obtaining a corresponding simple cleaning rule attribute configuration List < NodeTableRelay SimRelVO > set;
traversing a List < NodeTableRelay SimRelVO > set, acquiring a col value and a relation tag value of xml configuration, acquiring a corresponding key value through a map in a relation tag value acquisition request parameter, and storing a corresponding value in the acquired map into a cleaned map source value (map source value is a map cleaned by using a target col as a key of the map);
s3.2, complex cleaning rules (attribute processing mapping);
defining a label:
< complex >: a complex mapping relationship is declared, with no or at least one < relation > sub-label.
The attributes include the following:
desc, mapping description, which is used for remarking the current complex mapping relation;
< relation >: the field mapping result table mapping relationship in the request parameter table is declared, and there is no or at least one < rel > sub-tag. The attributes include the following:
col: declaring a purge target table field;
< rel >: the complex mapping processing rule is declared, without sub-tags. The attributes include the following:
handle: declaring a processing mode;
ref-type: declaring a value type;
data-Type: declaring a value type or a conversion mode;
rel tag value: a request parameter table field representing a purge;
business rules:
one complex label contains a plurality of relation sub-labels, the corresponding col is the final cleaning target, and the relation sub-label value is the attribute field of the after-table of the source table after node.
Processing logic:
traversing the NodeTableVO attribute after the filtering, and obtaining a corresponding complex cleaning rule attribute configuration List < NodeTableRelay ComplexVO > set;
traversing List < NodeTableRelay ComplexVO > set, obtaining col values of xml configuration and relation tag value set List < NodeTableRelay ComRelVO >
Traversing a NodeTableRelay ComRelVO set to obtain a configured handle, refType, dataType, rel value;
if handle is +/addTarget, col representing complex mapping is added with an attribute value, the current applicable scene is String type and number type, and the column representing col is added with an attribute
If handle is-,/, the complex mapping is expressed as mathematical operation
If handle is turn, it indicates type conversion
If handle is dateFormat, indicate time formatting
If handle is a subString, it indicates that string interception is performed (the dataType configured at this time is the number of digits for interception, such as 0, 10)
If refType is local, it indicates that the value is the value of the relationship tag configuration (the value in xml configuration file)
If refType is source, the value is the key value in the request parameter
If dataType is NUM, NUM1, NUM2, NUM, represent the exact bits after NUM conversion (numbers following NUM);
if dataType is STRING: representing a string operation (handle is + represents an append operation);
if dataType is yyyy-mm-dd, yyyy/mm/dd (or other date format): representing the conversion of configuration properties from comma-to-comma-front to comma-to-rear format (to be used in conjunction with handle as dateFormat)
If dataType is =, =, x=: here, it may be defined that any attribute value (e.g., 1=s, 2=n, 1,2, 3=s, 3,4,5, 6=n) indicates that after the configuration attribute value is read, if the attribute value in front of the equal sign in the configuration is the attribute value in front of the equal sign, the attribute value is converted into the attribute value in back of the equal sign (i.e., check_flag=3, configured as 3,4, 5=n, the check_flag is converted into N);
obtaining a relation configuration value of the complex cleaning according to handle, refType, dataType, rel processing logic (when handle of the first relation is +, -,/is that a plurality of relation operations can be configured in the complex cleaning, that is, multiple operations are performed on the same attribute), and storing the complex cleaning in a mapping sourcevalue (mapping sourcevalue is a map after temporary attribute cleaning, and the mapping value is value);
example 1, as follows: setting the service type field as an EOS+original list service main key, and configuring as follows:
example 2, as follows: formatting the original time format yyyyy-MM-dd into yyymmdd is configured as follows:
example 3, as follows: the transition of transaction states 1,3 to F (failure), 0,2,4 to S (success), and the empty transition to P are configured as follows:
example 4, as follows: the transaction amount principal and the transaction amount interest are collected into the transaction amount, and the following configuration is carried out:
s3.3, custom cleaning rules (attribute expansion custom function method).
Defining a label:
< customer-definition >: custom cleansing rule (extended implementation) tags are declared. There is and only one < rel > sub-tag. The attributes include the following:
col: declaring a target implementation class of the custom cleaning rule;
business rules:
when the cleaning rules in the step 2 are not matched, an extension method of custom realization is defined, the parent class TransAnalysCurtomeSupFecade is inherited, and a handleCurtomer method of the parent class is required to be rewritten (the input parameters are current request parameters and xml configuration, the cleaned attribute Map is output parameters are temporary variables Map < String, object >);
processing logic:
traversing the NodeTableVO attribute after the filtering, and obtaining a corresponding custom rule configuration attribute NodeTableRelay DefineVO;
calling SpringUtil. GetBuan to acquire a TransAnalysis custom SupFefaade object through a value configured by col in xml;
the custom processing method of TransAnalysisConsetomerSupFacade is called to obtain the cleaned map
Traversing the map, and storing the self-defined and cleaned map, key and value into a mapingSourceValue (the mapingSourceValue is the map after temporary attribute cleaning (the key with the target col as the map, and the mapping value is the value).
Step 4, selecting a name of a table to be operated and a field in the table as an XML source data service main key in the data information to be operated, and defining the name and the field in the table in an XML rule template;
in the above technical solution, the selecting the name of the table to be operated and a field in the table as the XML source data service primary key, and defining the XML rule template specifically includes: defining XML source data query conditions; as shown in fig. 3, the tags of the XML rule template in the step S4 are as follows:
< queryIndex >: the mapping table query condition is declared, with at least one < or > sub-tag. The attributes include the following:
desc, mapping description, which is used for remarking the current simple mapping relation;
< or >: query condition mappings are declared, with at least one < rel > sub-tag. The attributes include the following:
rel: declaring a corresponding attribute value field in the mapping result table;
business rules:
acquiring a corresponding value from the cleaning temporary map (mappingSourceValue) through the configured rel configuration, assembling the post-window attribute after SQL, wherein the or label sub-label rel generates an or relationship, and the or label peer-to-peer or label generates an and relationship;
processing logic:
traversing the NodeTableVO attribute after the filtering to obtain a NodeTableQueryIndex attribute;
obtaining a set List < NodeTableQueryIndexOrs > of xml configuration from NodeTableQueryIndex
If the set List < NodeTableQueryIndexOrs > is empty, the hint configuration is empty, the configuration is checked, and the method ends
Not empty, traversing List < NodeTableQueryIndexOrs >, obtaining aggregate objects (multiple rel of xml configuration) in one or (time-of-And relationship between each NodeTableQueryIndexOrs, time-of-or relationship between or sub-tags rel)
The rel value is the key of cleaning temporary map (mappingSourceValue), the corresponding attribute value is obtained from map (mappingSourceValue), and the condition character String behind the where is generated (if the value type in the map is not String, no Shan Yinhao is added when sql is spliced).
And step 5, associating the XML source data service primary key with a table and a field of the local database to generate a corresponding cleaned SQL sentence.
In this embodiment, a corresponding cleaned SQL statement (including add, delete, modify, search) is generated; the request parameters are JSON objects of the source table, and the corresponding SQL sentences are output through types (adding, deleting, changing and checking) needing cleaning. (in the request JSON object, the operation type (I, U, D) is included).
The step S5 is as follows:
business rules:
corresponding SQL is generated through table name (table_name), operation type (op_type), washed map (mappingSourceValue) and sphere post-condition character strings in the request parameters.
Processing logic:
acquiring operation type op_type in request parameters
If D (delete), taking the conditional character string after the window, and generating an SQL sentence corresponding to delete by combining the table name;
if the character string is not D (I, U), traversing the mapingSource value, taking the corresponding key, value, combining table name and window post-condition character string, generating querySql, updateSql, insertSql character string, and packaging into the out-of-reference TransAnalysis SQLVO.
It should be noted that, in the embodiment of the present invention, by using the cleaning engine, only the operation of the corresponding table needs to be lost to kafka, and the service system accesses the kafak to call the cleaning engine, so that the service system data update can be automatically realized. In addition, because of the expansibility of the service, in the service scene of frequent change of transaction data, the difficulty of operation is increased, the investment of manpower is increased, and if the service is required to be expanded, the service of other institutions is increased, and recoding and debugging are performed. The data cleaning method engine is used, so that great help can be obtained in the deployment of the server, the expansion of the service and the maintenance of the operation.
The embodiment of the invention also provides an object conversion system based on the XML template, which comprises the following modules:
the XML rule template label definition module is used for acquiring json messages containing other database table operation information in the data bus;
the XML source data filtering tag definition module is used for filtering the data source of the json message through the data source filtering tag of the XML rule template, and screening out the data to be cleaned;
the XML source data cleaning rule definition module is used for cleaning the data to be cleaned through XML rule template data cleaning rules to obtain data information to be operated;
the XML source data query condition definition module is used for selecting a table name to be operated and a field in the table as an XML source data service main key in data information to be operated and defining the table name and the field in the table as an XML rule template;
and the SQL sentence generating module is used for associating the XML source data service main key with the table and the field of the local database to generate a corresponding cleaned SQL sentence.
It can be understood that the object conversion system based on the XML template provided by the present invention corresponds to the object conversion method based on the XML template provided in the foregoing embodiments, and the related technical features of the object conversion system based on the XML template may refer to the related technical features of the object conversion method based on the XML template, which are not described herein again.
The embodiment of the invention provides electronic equipment, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the steps of an object conversion method based on an XML template when executing the computer program.
The present embodiment provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of an XML template-based object transformation method.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
Finally, it should be noted that: the foregoing description is only illustrative of the preferred embodiments of the present invention, and although the present invention has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that modifications may be made to the embodiments described, or equivalents may be substituted for elements thereof, and any modifications, equivalents, improvements or changes may be made without departing from the spirit and principles of the present invention.

Claims (10)

1. An object conversion method based on an XML template, the method comprising the steps of:
step 1, acquiring json messages containing operation information of other databases in the data bus;
step 2, filtering the data source of the json message through the data source filtering tag of the XML rule template, and screening out the data to be cleaned;
step 3, cleaning data to be cleaned through XML rule template data cleaning rules to obtain data information to be operated;
step 4, selecting a name of a table to be operated and a field in the table as an XML source data service main key in the data information to be operated, and defining the name and the field in the table in an XML rule template;
and step 5, associating the XML source data service primary key with a table and a field of the local database to generate a corresponding cleaned SQL sentence.
2. The method for converting an object based on an XML template according to claim 1, wherein said obtaining json messages of the data bus, which include table operation information of other databases, includes: tags defining XML rule templates.
3. The method for converting an object based on an XML template according to claim 1, wherein the filtering the JSON message by using the data source filtering tag of the XML rule template to filter the data to be cleaned includes: XML source data filter tags are defined.
4. The method for converting an object based on an XML template as recited in claim 1, wherein,
in step 3, the cleaning the data to be cleaned by the XML rule template data cleaning rule, and obtaining the data information to be operated includes: defining XML source data cleaning rules; the data cleaning rule comprises the following steps:
s3.1, simple cleaning rules: analyzing the received json message, and copying the appointed field to a corresponding value;
s3.2, complicated cleaning rules: analyzing the received json message, and performing mathematical operation, type conversion or logic processing on the designated field to obtain a corresponding value;
s3.3, custom cleaning rules: analyzing the received json message, expanding the attribute by a user-defined function method, and obtaining a corresponding value.
5. The method for converting an object based on an XML template according to claim 4, wherein said simple cleaning rule is as follows:
declaring a simple mapping relation simple tag, which contains mapping description and is used for remarking the desc attribute of the current simple mapping relation;
the attribute value relation of the result table of the field mapping in the request parameter table is declared, and the attribute is contained:
col: a statement purge target table field;
the relation value: the request source table field is declared.
6. The method for converting an object based on an XML template according to claim 4, wherein said complex cleaning rule is as follows:
declaring a simple mapping relationship complex tag;
the field mapping result table mapping relation label in the request parameter table is declared to contain attributes:
col: declaring a purge target table field;
declaring a complex mapping process rule rel tag, containing attributes:
handle: declaring a processing mode;
ref-type: declaring a value type;
data-Type: declaring a value type or a conversion mode;
rel tag value: a request parameters table field representing a flush.
7. The method for converting an object based on an XML template according to claim 4, wherein said custom cleaning rule is as follows:
< queryIndex >: the mapping table query condition is declared, and at least one < or > sub-tag is included, and the attribute includes the following: desc, mapping description, which is used for remarking the current simple mapping relation;
< or >: declaring a query condition mapping relationship, having at least one < rel > sub-tag, the attribute comprising:
rel: and declaring a corresponding attribute value field in the mapping result table.
8. An XML template-based object conversion system, comprising:
the XML rule template label definition module is used for acquiring json messages containing other database table operation information in the data bus;
the XML source data filtering tag definition module is used for filtering the data source of the json message through the data source filtering tag of the XML rule template, and screening out the data to be cleaned;
the XML source data cleaning rule definition module is used for cleaning the data to be cleaned through XML rule template data cleaning rules to obtain data information to be operated;
the XML source data query condition definition module is used for selecting a table name to be operated and a field in the table as an XML source data service main key in data information to be operated and defining the table name and the field in the table as an XML rule template;
and the SQL sentence generating module is used for associating the XML source data service main key with the table and the field of the local database to generate a corresponding cleaned SQL sentence.
9. An electronic device comprising a memory, a processor for implementing the steps of the XML template-based object-transformation method of any one of claims 1-7 when executing a computer program stored in the memory.
10. A computer-readable storage medium, having stored thereon a computer management class program, which when executed by a processor, implements the steps of the XML template-based object conversion method of any one of claims 1-7.
CN202310804401.1A 2023-06-30 2023-06-30 Object conversion method, system, electronic equipment and storage medium based on XML template Pending CN116932519A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310804401.1A CN116932519A (en) 2023-06-30 2023-06-30 Object conversion method, system, electronic equipment and storage medium based on XML template

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310804401.1A CN116932519A (en) 2023-06-30 2023-06-30 Object conversion method, system, electronic equipment and storage medium based on XML template

Publications (1)

Publication Number Publication Date
CN116932519A true CN116932519A (en) 2023-10-24

Family

ID=88374783

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310804401.1A Pending CN116932519A (en) 2023-06-30 2023-06-30 Object conversion method, system, electronic equipment and storage medium based on XML template

Country Status (1)

Country Link
CN (1) CN116932519A (en)

Similar Documents

Publication Publication Date Title
US8892504B2 (en) Method and system for reconciling meta-data in a data warehouse
US6836890B1 (en) Methods and systems for message translation and parsing of data structures in a distributed component architecture
US6665662B1 (en) Query translation system for retrieving business vocabulary terms
US7721304B2 (en) Method and apparatus providing programmable network intelligence
US7210096B2 (en) Methods and apparatus for constructing semantic models for document authoring
US7490098B2 (en) Apparatus, system, and method for processing hierarchical data in disparate data repositories
US20050055365A1 (en) Scalable data extraction techniques for transforming electronic documents into queriable archives
CN112148509A (en) Data processing method, device, server and computer readable storage medium
CN109104405B (en) Binary protocol encoding and decoding method and device
CN112883022A (en) XML template-based object data cleaning and converting method
CN115543402B (en) Software knowledge graph increment updating method based on code submission
CN112256318B (en) Construction method and equipment for dependent product
CN116701325B (en) Binary file cache-based XBRL classification standard loading method
JP2008225898A (en) Conversion device, conversion program, and conversion method
CN110308907B (en) Data conversion method and device, storage medium and electronic equipment
CN116932519A (en) Object conversion method, system, electronic equipment and storage medium based on XML template
CN110806973A (en) Automatic generation method and device of interface message
CN111984684B (en) Data processing method and device
CN111737529B (en) Multi-source heterogeneous data acquisition method
CN115357286A (en) Program file comparison method and device, electronic equipment and storage medium
CN113347196A (en) Analysis method and device for analyzing network data, electronic equipment and storage medium
CN112130860A (en) JSON object analysis method and device, electronic device and storage medium
CN116644121A (en) Data format verification method, device, equipment and storage medium
CN116226066B (en) Low code platform code synchronization method and device, electronic equipment and storage medium
CN115639985A (en) Database operation method, system, device and readable storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination