CN115269228A

CN115269228A - Data adaptive transmission method, device, equipment and medium

Info

Publication number: CN115269228A
Application number: CN202210909504.XA
Authority: CN
Inventors: 余鹏; 钟海; 曹佳杰; 陈华国; 黄洁平
Original assignee: Guangzhou Huiqiao Computer Technology Co ltd
Current assignee: Guangzhou Huiqiao Computer Technology Co ltd
Priority date: 2022-07-29
Filing date: 2022-07-29
Publication date: 2022-11-01

Abstract

The invention discloses a data adaptive transmission method, a device, equipment and a medium, which are applied to a back-end server, and the method comprises the following steps: when a sending request input by a sending client is received, at least one sending task corresponding to the sending request is constructed by adopting a preset sending task template; calling a connection service to package all sending tasks into a concurrent task list and caching the concurrent task list to a preset message queue; calling an engine service to extract a concurrent task list from the message queue and remove duplication to obtain a target query task; target query tasks are executed through engine services, information to be sent is generated and sent to a receiving client side, and therefore data adaptation among different business systems is achieved more stably and efficiently, and data interaction efficiency among all the business systems is improved.

Description

Data adaptive transmission method, device, equipment and medium

Technical Field

The present invention relates to the field of data interaction technologies, and in particular, to a method, an apparatus, a device, and a medium for adaptive data transmission.

Background

With the development of society, the scale of hospitals is enlarged, the informatization technology is gradually improved, and the informatization construction of the hospitals is strengthened, so that the method is an important guarantee for improving the management level of the hospitals and promoting the scientific development and comprehensive construction of the connotation of the hospitals.

Therefore, at present, the hospital information system mainly performs data interaction by using an SDK mode, a traditional view query, a storage process query, a json return query by using an http synchronous interface, an xml message push by using a web service, a json sending mode by using an http protocol, and the like.

Because each service system usually has different independent development interfaces, an interface program used in the traditional scheme has large invasion to the service system, the updating of the interfaces easily causes instability of the service system, and meanwhile, each service system needs manual retransmission of data, so that the data interaction efficiency among the service systems is reduced.

Disclosure of Invention

The invention provides a data adaptive transmission method, a device, equipment and a medium, which solve the technical problems that interface programs used in the traditional scheme have high invasion to service systems because different independent development interfaces exist in each service system generally, the service systems are unstable because the interfaces are updated easily, and meanwhile, each service system needs to manually retransmit data, so that the data interaction efficiency among the service systems is reduced.

A data adaptation transmission method provided in a first aspect of the present invention is applied to a backend server, and the method includes:

when a sending request input by a sending client is received, adopting a preset sending task template to construct at least one sending task corresponding to the sending request;

calling a connection service to package all the sending tasks into a concurrent task list and caching the concurrent task list to a preset message queue;

calling an engine service to extract the concurrent task list from the message queue and remove the duplicate to obtain a target query task;

and executing the target query task through the engine service, generating information to be sent and sending the information to a receiving client.

Optionally, the step of constructing at least one sending task corresponding to the sending request by using a preset sending task template when receiving the sending request input by the sending client includes:

when a sending request input by a sending client is received, acquiring an input position corresponding to the sending request;

if the input position is the position of the synchronous sending link, extracting a first sending keyword carried by the sending request, loading the first sending keyword to a preset sending task template, and generating a single sending task corresponding to the sending request;

and if the input position is the position of the asynchronous sending button, extracting second sending keywords and task creation quantity carried by the sending request, and creating sending tasks with the quantity equal to the task creation quantity by adopting the sending task template.

Optionally, if the input position is the position of the asynchronous sending button, the step of extracting the second sending keyword and the number of created tasks carried by the sending request, and creating sending tasks with the number equal to the number of created tasks by using the sending task template includes:

if the input position is the position of the asynchronous sending button, extracting a task creation type, a second sending keyword and a task creation number carried by the sending request;

if the task creation type is a common query task, creating a single sending task by adopting the sending task template and the second sending keyword;

if the task creation type is a combined query task, creating sending tasks with the same number as the task creation number by adopting the sending task template and a plurality of second sending keywords;

if the task creation type is a comparison query task, retrieving existing sending data corresponding to the second sending keyword;

and comparing the existing sending data with historical returning data, determining difference data and establishing a single sending task by adopting the sending task template.

Optionally, the step of executing the target query task through the engine service, generating information to be sent, and sending the information to a receiving client includes:

extracting, by the engine service, a query primary key and a receive address from the target query task;

querying a preset message database by the engine service by adopting the query main key, and determining initial field information corresponding to the query main key;

mapping the initial field information into a text format and rendering the initial field information according to a mapping dictionary in the target query task through the engine service to generate information to be sent;

and sending the information to be sent to a receiving client corresponding to the receiving address through the engine service.

Optionally, the method further comprises:

if the data to be sent is successfully sent, recording a log of successful sending to the message database;

and if the data to be sent fails to be sent, executing the target query task again through the engine service, rendering the data to be sent, and sending the data to be sent to a receiving client again.

Optionally, the method further comprises:

when the data to be sent fails to be sent, writing the task to be executed into an abnormal task table;

and when a task click instruction aiming at the abnormal task table is received, selecting and executing a task to be executed corresponding to the task click instruction, and sending the data to be sent to a receiving client.

Optionally, the back-end server is provided with a configuration page, and the method further includes:

responding to a data source configuration instruction input in the configuration page, and creating a message database corresponding to the data source configuration instruction;

when receiving interface document data in the configuration page, analyzing the interface document data, and extracting a retransmission parameter, a message queue partition parameter, a dictionary mapping parameter, sending address information, a query parameter and a field conversion parameter;

configuring a preset standard sending template according to the sending address information to obtain a first sending template;

configuring the first sending template by adopting the retransmission parameter, the message queue partition parameter and the query parameter to obtain a second sending template;

configuring the second sending template by adopting the dictionary mapping parameters, and determining a mapping dictionary in the second sending template;

and configuring the second sending template by adopting the field conversion parameters, determining a mapping text format corresponding to the mapping dictionary, obtaining a target sending template and rendering to generate a sending task template.

The second aspect of the present invention provides a data adaptive transmission apparatus, which is applied to a backend server, and the apparatus includes:

the system comprises a sending task construction module, a sending task execution module and a sending task execution module, wherein the sending task construction module is used for constructing at least one sending task corresponding to a sending request by adopting a preset sending task template when the sending request input by a sending client is received;

the task encapsulation module is used for calling the connection service to encapsulate all the sending tasks into a concurrent task list and cache the concurrent task list to a preset message queue;

the task screening module is used for calling engine service to extract the concurrent task list from the message queue and remove duplication to obtain a target query task;

and the task execution module is used for executing the target query task through the engine service, generating information to be sent and sending the information to the receiving client.

A third aspect of the present invention provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor is caused to execute the steps of the data adaptation transmission method according to any one of the first aspect of the present invention.

A fourth aspect of the invention provides a computer readable storage medium having stored thereon a computer program which, when executed, implements a data adaptation transmission method according to any one of the first aspects of the invention.

According to the technical scheme, the invention has the following advantages:

when a back-end server receives a sending request input by a sending client, the sending request can be shunted in a gateway load balancing mode, and at least one sending task corresponding to the sending request is constructed by adopting a preset sending task template; calling a connection service to package all the sending tasks into a concurrent task list and caching the concurrent task list to a preset message queue; calling an engine service to extract the concurrent task list from the message queue and remove the duplicate to obtain a target query task; and executing the target query task through the engine service, generating information to be sent and sending the information to a receiving client. The technical problems that interface programs used in the traditional scheme are large in invasion to business systems, the business systems are easy to be unstable due to the fact that the interfaces are updated, meanwhile, the business systems need manual data retransmission, and therefore the data interaction efficiency between the business systems is reduced are solved.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without inventive exercise.

Fig. 1 is a flowchart illustrating steps of a method for adaptive data transmission according to an embodiment of the present invention;

fig. 2 is a flowchart illustrating steps of a data adaptive transmission method according to a second embodiment of the present invention;

fig. 3 is a service call graph of a data adaptation transmission method according to a second embodiment of the present invention;

fig. 4 is a block diagram of a data adaptive transmission apparatus according to a third embodiment of the present invention.

Detailed Description

The embodiment of the invention provides a data adaptive transmission method, a data adaptive transmission device, data adaptive transmission equipment and a data adaptive transmission medium, which are used for solving the technical problems that interface programs used in the traditional scheme have large invasion to service systems because different independent development interfaces usually exist in each service system, the service systems are easy to be unstable due to interface updating, and meanwhile, each service system needs to manually retransmit data, so that the data interaction efficiency among the service systems is reduced.

XML is an extensible markup language, a subset of standard generalized markup languages, XML for short. Is a markup language for marking electronic documents to be structured.

In electronic computers, a label refers to a symbol of information that can be understood by a computer, and by this label, various information such as articles and the like can be handled between computers. It can be used to mark data, define data types, and is a source language that allows a user to define his or her own markup language. It is well suited for world wide web transport, providing a unified approach to describing and exchanging structured data that is independent of the application or vendor. Is a cross-platform, content-dependent technology in the Internet environment and is also an effective tool today for processing distributed structural information. As early as 1998, the XML1.0 specification was published by W3C, which was used to simplify the transmission of Internet documents.

JSON (JavaScript Object Notification) is a lightweight data exchange format. It stores and represents data in a text format that is completely independent of the programming language, based on a subset of ECMAScript (js specification set by the european computer association). The compact and clear hierarchy makes JSON an ideal data exchange language. The network transmission method is easy to read and write by people, is easy to analyze and generate by machines, and effectively improves the network transmission efficiency.

SDK (Software Development Kit): software Development kits are generally a collection of Development tools used by some Software engineers to build application Software for a particular Software package, software framework, hardware platform, operating system, etc.

A software development tool broadly refers to a collection of related documents, paradigms, and tools that assist in developing a certain class of software.

A software development kit is a collection of development tools used by software engineers to create application software for a particular software package, software framework, hardware platform, operating system, etc., and in general, an SDK is an SDK used to develop applications on a Windows platform. It may simply be a file that provides an application program interface API for a certain programming language, but may also include complex hardware that can communicate with a certain embedded system. Typical tools include utility tools for debugging and other uses. SDKs also often include example code, supporting technical notes, or other supporting documentation to clarify suspects for basic reference material.

An Application Programming Interface (API) is an agreement for the connection of different components of a software system. The design of programming interfaces is important because of the ever increasing size of software in recent years, often requiring complex systems to be divided into small components. In the programming practice, the programming interface is designed to reasonably divide the responsibility of the software system. The good interface design can reduce the mutual dependence of all parts of the system, improve the cohesion of the composition units and reduce the coupling degree among the composition units, thereby improving the maintainability and the expansibility of the system.

HTTP (Hyper Text Transfer Protocol) the hypertext Transfer Protocol is a simple request-response Protocol. It specifies what messages the client may send to the server and what responses to get. This simple model is an instrumental minister of early Web success because it makes development and deployment very straightforward.

Web Service is a platform independent, low-coupling, self-contained, programmable Web-based application that can be described, published, discovered, coordinated and configured using the open XML (a subset of the standard generalized markup language) standard for developing distributed, interoperating applications.

The Web Service technology can ensure that different applications running on different machines can exchange data or be integrated with each other without additional and special third-party software or hardware. Applications implemented according to the Web Service specification may exchange data with each other regardless of the language, platform, or internal protocol used by them. Web Service is a self-describing, self-contained, available network module that can perform specific business functions. Web services are also easy to deploy because they are based on some conventional industry standards and some existing technologies, such as subset XML, HTTP under a standard generic markup language. Web Service reduces the cost of application interfaces. Web Service provides a generic mechanism for the integration of business processes throughout an enterprise and even across multiple organizations.

In order to make the objects, features and advantages of the present invention more obvious and understandable, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is obvious that the embodiments described below are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Referring to fig. 1, fig. 1 is a flowchart illustrating a data adaptive transmission method according to an embodiment of the present invention.

The invention provides a data adaptive transmission method, which is applied to a back-end server and comprises the following steps:

step 101, when a sending request input by a sending client is received, adopting a preset sending task template to construct at least one sending task corresponding to the sending request;

the sending client is a program corresponding to the back-end server and providing local service for the client. Except for some application programs which only run locally, the application programs are generally installed on common clients and need to be operated together with a back-end server.

In the embodiment of the application, when the back-end server receives a sending request input by a sending client, it indicates that the sending client needs to send data, and at this time, at least one sending task corresponding to the sending request may be constructed according to a preset sending task template.

It should be noted that the sending task template may be generated by configuring parameters, calling methods, and statements on a configuration page of the backend server, where the parameters, the parameter using process, the sending task execution logic, and the like of the sending task are constructed.

Step 102, calling a connection service to package all sending tasks into a concurrent task list and caching the concurrent task list to a preset message queue;

the connection service means that the connection of the request is ensured not to be disconnected, and as a bridge between the request task and the message queue, the concurrent task list can be sent to the message queue in the manner of http request through the provided interface.

The concurrent Task list refers to a list formed by the main keys corresponding to the Task objects generated after the sending Task is encapsulated.

In this embodiment, since both the data retrieval logic and the sending flow are loaded in the sending task, after the sending task is constructed, the connection service can be directly invoked to package all the sending tasks into a concurrent task list, and meanwhile, in order to ensure that data can still be sent stably under mass data push, the concurrent task list is cached in a message queue.

103, calling an engine service to extract a concurrent task list from the message queue and remove duplication to obtain a target query task;

the engine service is mainly used for acquiring tasks from the message queue, generating complete messages according to the parameters of the tasks and sending the complete messages to the corresponding back-end interface service. And meanwhile, the sending result (success or failure) is recorded, and the failed task can be retransmitted, so that the normal circulation of data is ensured.

In order to ensure normal circulation of data, an engine service can be called to extract a concurrent task list from a message queue, and after the sending tasks in the concurrent task list are deduplicated, executable target query tasks are screened.

And 104, executing the target query task through the engine service, generating information to be sent and sending the information to the receiving client.

In the embodiment of the application, after the target query task is obtained by screening of the engine service, the target query task can be executed through the engine service so as to call the task running logic of information query and information generation therein, generate the information to be sent and send the information to the receiving client.

It should be noted that the receiving client may serve as a backend interface, or may serve as other clients, such as a medical service system responsible for different functions.

In the embodiment of the application, when a back-end server receives a sending request input by a sending client, the sending request can be shunted in a gateway load balancing mode, and at least one sending task corresponding to the sending request is constructed by adopting a preset sending task template; calling a connection service to package all sending tasks into a concurrent task list and caching the concurrent task list to a preset message queue; calling an engine service to extract a concurrent task list from the message queue and remove the duplicate to obtain a target query task; and executing the target query task through the engine service, generating information to be sent and sending the information to the receiving client. The technical problems that interface programs used in the traditional scheme are large in invasion to business systems, the business systems are easy to be unstable due to the fact that the interfaces are updated, meanwhile, the business systems need manual data retransmission, and therefore the data interaction efficiency between the business systems is reduced are solved.

Referring to fig. 2, fig. 2 is a flowchart illustrating steps of a data adaptive transmission method according to a second embodiment of the present invention.

step 201, when receiving a sending request input by a sending client, acquiring an input position corresponding to the sending request.

The sending request refers to request information input by a sending client for starting a task building process of a back-end server and specifying a task type to be built.

In the embodiment of the application, when the back-end server receives a sending request input by a sending client, the back-end server detects the page corresponding to the sending client to obtain an input position corresponding to the sending request, and determines the task type required to be constructed by the sending request.

It should be noted that the sending client is provided with an input page, and responds to input instructions, such as click, touch, press and the like, input by the user through the sending client, and constructs a corresponding sending request according to a trigger position of the input instruction in the page.

Step 202, if the input position is the position of the synchronous sending link, extracting a first sending keyword carried by the sending request, loading the first sending keyword to a preset sending task template, and generating a single sending task corresponding to the sending request.

In the embodiment of the present application, if the input position is the position of the synchronous sending link, it indicates that the task type that needs to be established by the sending client at this time is the synchronous sending task, the first sending keyword carried by the sending request may be extracted, and the first sending keyword is loaded into the preset sending task template, so as to create a single sending task corresponding to the sending request.

It should be noted that the synchronous sending link refers to a link in an input page on the sending client for starting generation of a sending task.

And 203, if the input position is the position of the asynchronous sending button, extracting second sending keywords and task creation quantity carried by the sending request, and creating sending tasks with the quantity equal to the task creation quantity by adopting a sending task template.

Further, the following substeps S11-S15 may be included:

s11, if the input position is the position of the asynchronous sending button, extracting a task creation type, a second sending keyword and a task creation number carried by the sending request;

the task creation type refers to a type of sending task creation, including but not limited to a general query task, a combined query task, and a comparison query task.

In the embodiment of the application, an asynchronous sending button is further arranged on an input page of the sending client, and if the input position is the position of the asynchronous sending button, the task creation type, the second sending keyword and the task creation number carried by the sending request are extracted and used as a data basis for subsequent task creation.

S12, if the task creation type is a common query task, creating a single sending task by adopting a sending task template and a second sending keyword;

in an example of the present application, if the extracted task creation type is a common query task, which indicates that only a single query and a single transmission are required at this time, the second transmission keyword may be loaded into the transmission task template to create a single transmission task.

S13, if the task creation type is the combined query task, creating sending tasks with the same number as the task creation by adopting a sending task template and a plurality of second sending keywords;

in another example of the present application, if the task creation type is a combined query task, it indicates that multiple query statement query information is needed to jointly and simultaneously query multiple standard entry parameters, and at this time, multiple second sending keywords may be loaded into the sending task template, so as to create sending tasks equal to the number of task creations.

It should be noted that the task creation process may be as follows: selecting a standard template, configuring an execution frequency (time t2= t1+ delta t for each execution, t1 is the time of last execution, delta t is the time of frequency setting), selecting a data source, selecting a query type (two query types are provided, time rolling and general query

S14, if the task creation type is a comparison query task, retrieving existing sending data corresponding to the second sending keyword;

and S15, comparing the existing sending data with the historical returning data, determining the difference data and establishing a single sending task by adopting a sending task template.

Optionally, the data receiving party is prevented from being unable to receive data due to unstable factors such as program problems, the task creation type can be selected as a comparison query task, existing sending data corresponding to the second sending keyword can be detected, the existing sending data and the historical return data are compared, so that the difference between the existing sending data and the historical return data is determined, the difference data is obtained and loaded into a sending task template, and a single sending task is created.

It should be noted that, for the construction of the comparison and query task, the function test of the sending task may be further performed, and the implementation logic is that two statements are transmitted to the background service- > establish a dynamic data source object- > establish a data source connection- > execute the sql statement- > data queried by the two statements of the comparison and query task configured above, and the execution is considered successful if no error is reported in the flow program above when no sent data is compared.

It should be noted that after the creation of the sending task is completed, the running of the sending task may be further tested, and the testing method may be called in a post mode of http. The test method realizes the logic that whether an error exists in the process of establishing a dynamic data source object- > establishing a data source connection- > executing the sql statement in the configured sending task template, if the error exists, the failure of the 'false' test is returned, otherwise, the success of the 'true' test is returned.

In one example of the present application, the backend server is provided with a configuration page, and the method further comprises the following steps S21-S26:

s21, responding to a data source configuration instruction input in the configuration page, and creating a message database corresponding to the data source configuration instruction;

in the specific implementation, in response to a data source configuration instruction input in a configuration page, after the data source information is filled in a page form, data is sent to a background through an http protocol. The method comprises the steps that a Dynamic data source Base class (a custom-developed java class which encapsulates a series of methods for data source operation, such as data addition, deletion, check and modification) is used for converting data into java objects, new method data source information in the class is called to be inserted into a message database, and the message database corresponding to a data source configuration instruction is created.

The selected message database types comprise oracle, sqlserver, mysql and the like, and account passwords and link addresses of the message database are configured. The maximum connection number, the minimum connection number and the request timeout time of the message database can be configured, so that reasonable utilization of resources of the data source is guaranteed.

Optionally, the message database may be tested after being configured, and after the test button is clicked, a post request (a network protocol request manner) of http is sent to the server background. Calling a testDbSource method of a DbConfigController controller (java class), initializing a data source by a code, calling a default test connection statement of a message database, judging whether the test is successful or not through a return value, if the test is successful, returning to 'true', and if the test is failed, returning to 'false'.

S22, when receiving the interface document data in the configuration page, analyzing the interface document data, and extracting a retransmission parameter, a message queue partition parameter, a dictionary mapping parameter, sending address information, a query parameter and a field conversion parameter;

in this embodiment of the present application, interface document data may be uploaded in a configuration page to provide a data basis for subsequent configuration, where the type of the interface document data may be Excel, the content of the Excel is field information of a standard template, and an xml or json sample (xml, json are both standardized text formats) is sent.

When interface document data is received, it may be parsed to extract retransmission parameters, message queue partition parameters, dictionary mapping parameters, transmission address information, query parameters, and field conversion parameters. Specifically, an html form submission mode can be used, and an uploadExcel method of a template statetcontroller type can be called through a post mode of http. The logic for implementing the uploadExcel method is as follows: get file name- > get the number of all worksheets- > get a sheet, i.e. a workbook- > go from the first row-get data for each cell. Constructing an object through each cell data, calling insert statement to write data into a message database

S23, configuring a preset standard sending template according to the sending address information to obtain a first sending template;

in specific implementation, a json character string can be constructed according to sending address information, data is transmitted to a back-end server through a post method of http, and receiving system information of a sending system is written into a message database by using insert statements of the message database, so that a first sending template is constructed.

S24, configuring a first sending template by adopting the retransmission parameters, the message queue partition parameters and the query parameters to obtain a second sending template;

meanwhile, the message queue can have a plurality of partitions, each partition has independent data, the partition into which the standard data enters can be configured, and the standard data randomly enters the partition under the default condition. The advantage of selecting the partition is to ensure that the queue of the standard data is dedicated and not mixed with other partition data to ensure the timely transmission of the key service data.

For this purpose, the retransmission parameter is used to identify whether the system filters repeated information, whether retransmission is incorrect, the number of retransmissions, and the like, and the retransmission parameter is cached in the front-end object and set as an attribute of the standard object, and the standard object is a self-defined object. And simultaneously, carrying out queue configuration on partitions of various types of data according to the partition parameters of the data queue. And simultaneously configuring the query logic in the first sending template according to the query parameters so as to obtain a second sending template.

Specifically, the configuration may be performed through a query statement, i.e., an sql statement, which is a syntax of the database query, such as select from dual, and after the user writes this statement on the interface, the front-end page temporarily stores the statement in the memory object.

S25, configuring a second sending template by adopting the dictionary mapping parameters, and determining a mapping dictionary in the second sending template;

meanwhile, the second sending template is configured by using dictionary mapping parameters, and is encapsulated into a QueryResultMapping object (self-defined object) by a data mapping relation and a front-end html page, wherein the object comprises a source code, a source name, a target code and a target name. Thereby determining a mapping dictionary within the second transmission template.

S26, configuring a second sending template by adopting the field conversion parameters, determining a mapping text format corresponding to the mapping dictionary, obtaining a target sending template and rendering to generate a sending task template.

And finally, configuring a second sending template by adopting the field conversion parameters, and rendering the fields searched out by the database to xml or json texts, wherein the mapping of the template is required to be configured, so that a mapping text format corresponding to a mapping dictionary is determined, a target sending template is obtained, and the target sending template is rendered to generate a sending task template.

It should be noted that before the task template is sent for rendering, the dynamic data source object- > is established, the data source connection- > is executed, the configured sql statement- > is queried, the result is mapped to the target sending template- > whether there is an error in the execution process, if there is an error, the "false" test fails is returned, otherwise, the "true" test succeeds, and thus the feasibility of the template sending template is verified.

Alternatively, all data may be encapsulated into one object standard. All the attributes configured in the above steps are in the standard object, such as query statement, message queue partition information, deduplication information, and the like. The above information is saved to the database. And calling a saveStandardSnap method of a template dynamic controller type through a post request of http, converting the object into a json text format by the method, and automatically generating a version number, wherein the version number is increased by 0.1 from 1.0 every time. And finally, calling an insertion method of the non-relational database to write the data.

In another example of the present application, after the query parameter is filled, the template debugging interface may be entered, and a unitTest method of a template dynamic controller is called, where the method implements a design logic as follows: obtaining a standard of a test- > inquiring data by using an inquiry parameter- > rendering a template (xml or json) - > calling a send method of webservice to send data- > obtaining a return value to judge whether the sending is successful. Clicking the publish button publishes the configuration. After the publication, the publication information is written into the registry. The engine service monitors the registry to find that a new version exists and updates the local cache. A server data circulation process: calling a Release method of a template static controller, wherein the method realizes the design logic as follows: the method comprises the steps of constructing a registry node (system id + service coding) - > locking the registry to prevent concurrency- > obtaining the latest version number- > adding 0.1- > writing data into the registry on the basis of the latest version of the current standard- > monitoring the change of the data of the registry by an engine service- > pulling the latest data to update a local cache.

Step 204, calling a connection service to package all sending tasks into a concurrent task list, and caching the concurrent task list to a preset message queue;

after the sending Task is generated, the data can be packaged into a concurrent Task list, namely a Task object, and the Task object is cached to a preset message queue.

In specific implementation, after all sending tasks are packaged into a concurrent task list by calling a link service, a main key of each sending task can be extracted and pushed to a preset message queue to wait for further processing.

Step 205, invoking an engine service to extract a concurrent task list from the message queue and remove duplication to obtain a target query task;

in the embodiment of the application, the concurrent task list can be extracted from the message queue by calling the engine service, and after each sending task is deduplicated, a target query task is obtained to wait for the execution of subsequent further tasks.

And step 206, executing the target query task through the engine service, generating information to be sent and sending the information to the receiving client.

Optionally, step 206 may comprise the following sub-steps S31-S34:

s31, extracting a query primary key and a receiving address from the target query task through engine service;

s32, querying a preset message database by adopting a query main key through an engine service, and determining initial field information corresponding to the query main key;

s33, mapping the initial field information into a text format and rendering the initial field information according to the mapping dictionary in the target query task through the engine service to generate information to be sent;

and S34, sending the information to be sent to the receiving client corresponding to the receiving address through the engine service.

In the embodiment of the application, the target query task can be executed through an engine service, a query main key and a receiving address to be sent are extracted from the target query task through a configured flow, a preset message database is queried by the query main key, initial field information corresponding to the query main key is determined, the initial field information is mapped into a text format and rendered according to a mapping dictionary in the target query task, information to be sent is generated, and finally the information to be sent is sent to a receiving client corresponding to the receiving address through the engine service.

It should be noted that, if the task is a timing task, the target query task may be executed by detecting real-time, and if the task is a synchronization task, the information to be sent is directly sent according to the sending task generated after triggering.

Optionally, the method further comprises the following steps S41-S42:

s41, if the data to be sent is successfully sent, recording a log of successful sending to a message database;

and S42, if the data to be sent fails to be sent, executing the target query task again through the engine service, rendering the data to be sent, and sending the data to be sent to the receiving client again.

Further, the method comprises the following steps S43-S44:

s43, when the data to be sent fails to be sent, writing the task to be executed into an abnormal task table;

and S44, when a task click instruction aiming at the abnormal task table is received, selecting and executing a task to be executed corresponding to the task click instruction, and sending data to be sent to a receiving client.

In the embodiment of the application, a background query method can be called by clicking an abnormal task table, and abnormal data can be displayed by clicking a part to perform retransmission or clicking all the retransmission. And calling a retransmission method to acquire the information of the receiver, establishing a data source and the like.

Referring to fig. 3, the present solution may include the following services: gateway, connection management service, message queue, engine service, registry, API management service.

A gateway: the main roles are load balancing and health detection. The technique is designed to guarantee the performance and high availability of the system for a distributed microservice architecture. And the gateway automatically branches according to the requested flow, so that each service reaches the optimal state. For services that are not available, the gateway will also detect that a request is being sent to a service that is currently in a healthy state.

Connection management service: the connection of the request is ensured not to be disconnected and is used as a bridge between the request task and the message queue. The interface is provided to send the task to the message queue in the form of an http request.

Message queue: the main effect is peak clipping. When mass data pushing needs to ensure that data is stably and correctly received, the message queue can temporarily store the data, and the engine service pulls the data from the message queue according to the consumption capacity of the engine service.

Engine service: the method has the main functions of acquiring tasks from the message queue, generating complete messages according to the parameters of the tasks and sending the complete messages to corresponding backend interface services. And meanwhile, the sending result (success or failure) is recorded, and the failed task can be retransmitted, so that the normal circulation of data is ensured.

The registration center: the registry is mainly used for notification of events. The configuration center service, after modifying the attributes of the message template (e.g., the sending address), informs the registry, which pushes the tracking of the data to the engine service and the API call service at the same time.

API management services: API management provides both synchronous and asynchronous data distribution. And pushing the task generation task to a connection management service through the page configuration timing task or directly requesting to generate task synchronous calling through an interface.

When a back-end server receives a sending request input by a sending client, the sending request can be shunted in a gateway load balancing mode, and at least one sending task corresponding to the sending request is constructed by adopting a preset sending task template; calling a connection service to package all sending tasks into a concurrent task list and caching the concurrent task list to a preset message queue; calling an engine service to extract a concurrent task list from the message queue and remove duplication to obtain a target query task; and executing the target query task through the engine service, generating information to be sent and sending the information to the receiving client. The technical problems that interface programs used in a traditional scheme have high intrusiveness on business systems, the business systems are easy to be unstable due to updated interfaces, and meanwhile, the business systems need manual data retransmission, and the data interaction efficiency among the business systems is reduced are solved.

Referring to fig. 4, fig. 4 is a block diagram of a data adaptive transmission device according to a third embodiment of the present invention.

The embodiment of the invention provides a data adaptation transmission device, which is applied to a back-end server and comprises the following components:

the sending task constructing module 401 is configured to, when a sending request input by a sending client is received, construct at least one sending task corresponding to the sending request by using a preset sending task template;

a task encapsulation module 402, configured to invoke a connection service to encapsulate all sending tasks into a concurrent task list, and cache the concurrent task list to a preset message queue;

the task screening module 403 is configured to invoke an engine service to extract a concurrent task list from the message queue and perform deduplication to obtain a target query task;

and the task execution module 404 is configured to execute the target query task through an engine service, generate information to be sent, and send the information to the receiving client.

Optionally, the sending task building module 401 includes:

the input position acquisition submodule is used for acquiring an input position corresponding to a sending request when the sending request input by the sending client is received;

the synchronous task creating submodule is used for extracting a first sending keyword carried by the sending request if the input position is the position of the synchronous sending link, loading the first sending keyword to a preset sending task template and generating a single sending task corresponding to the sending request;

and the asynchronous task creating submodule is used for extracting the second sending keywords and the task creating quantity carried by the sending request if the input position is the position of the asynchronous sending button, and creating the sending tasks with the quantity equal to the task creating quantity by adopting the sending task template.

Optionally, the asynchronous task creating submodule is specifically configured to:

if the task creation type is a common query task, a single sending task is created by adopting a sending task template and a second sending keyword;

if the task creation type is a combined query task, creating sending tasks with the same number as the task creation by adopting a sending task template and a plurality of second sending keywords;

and comparing the existing sending data with the historical returning data, determining the difference data and establishing a single sending task by adopting a sending task template.

Optionally, the task execution module 404 is specifically configured to:

extracting a query primary key and a receiving address from a target query task through an engine service;

querying a preset message database by adopting a query main key through an engine service, and determining initial field information corresponding to the query main key;

mapping the initial field information into a text format and rendering the initial field information according to a mapping dictionary in a target query task through an engine service to generate information to be sent;

Optionally, the apparatus further comprises:

the successful recording module is used for recording the successful sending log to the message database if the data to be sent is successfully sent;

and the failure retransmission module is used for executing the target query task again through the engine service and rendering the data to be transmitted if the data to be transmitted fails to be transmitted, and transmitting the data to be transmitted to the receiving client again.

Optionally, the apparatus further comprises:

the abnormal task table writing module is used for writing the task to be executed into the abnormal task table when the data to be sent fails to be sent;

and the retransmission selection module is used for selecting and executing the task to be executed corresponding to the task click instruction when the task click instruction aiming at the abnormal task table is received, and sending the data to be sent to the receiving client.

Optionally, the back-end server is provided with a configuration page, and the apparatus further includes:

the message database creating module is used for responding to a data source configuration instruction input in the configuration page and creating a message database corresponding to the data source configuration instruction;

the document analysis module is used for analyzing the interface document data when the interface document data is received in the configuration page, and extracting a retransmission parameter, a message queue partition parameter, a dictionary mapping parameter, sending address information, a query parameter and a field conversion parameter;

the first configuration module is used for configuring a preset standard sending template according to the sending address information to obtain a first sending template;

the second configuration module is used for configuring the first sending template by adopting the retransmission parameters, the message queue partition parameters and the query parameters to obtain a second sending template;

the third configuration module is used for configuring the second sending template by adopting the dictionary mapping parameters and determining a mapping dictionary in the second sending template;

and the template rendering module is used for configuring a second sending template by adopting the field conversion parameters, determining a mapping text format corresponding to the mapping dictionary, obtaining a target sending template and rendering to generate a sending task template.

An embodiment of the present invention provides an electronic device, which includes a memory and a processor, where the memory stores a computer program, and when the computer program is executed by the processor, the processor is enabled to execute the steps of the data adaptation transmission method according to any embodiment of the present invention.

Embodiments of the present invention provide a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed, implements a data adaptation transmission alarm method according to any embodiment of the present invention.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one position, or may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims

1. A data adaptation transmission method is applied to a back-end server, and comprises the following steps:

2. The method according to claim 1, wherein the step of constructing at least one sending task corresponding to the sending request by using a preset sending task template when receiving the sending request input by the sending client comprises:

3. The method according to claim 2, wherein the step of extracting a second transmission keyword and a task creation number carried by the transmission request and creating transmission tasks equal to the task creation number using the transmission task template if the input position is a position where an asynchronous transmission button is located includes:

4. The method of claim 1, wherein the step of executing the target query task by the engine service, generating information to be sent, and sending the information to a receiving client comprises:

5. The method according to any one of claims 1-4, further comprising:

6. The method of claim 5, further comprising:

and when a task click command aiming at the abnormal task table is received, selecting and executing a task to be executed corresponding to the task click command, and sending the data to be sent to a receiving client.

7. The method of claim 1, wherein the back-end server is provided with a configuration page, the method further comprising:

when receiving interface document data in the configuration page, analyzing the interface document data, and extracting retransmission parameters, message queue partition parameters, dictionary mapping parameters, sending address information, query parameters and field conversion parameters;

and configuring the second sending template by adopting the field conversion parameters, determining a mapping text format corresponding to the mapping dictionary, obtaining a target sending template, and rendering to generate a sending task template.

8. A data adaptation transmission apparatus, applied to a backend server, the apparatus comprising:

9. An electronic device, comprising a memory and a processor, the memory having stored thereon a computer program that, when executed by the processor, causes the processor to perform the steps of the data adaptation transmission method according to any of claims 1-7.

10. A computer-readable storage medium, on which a computer program is stored, which, when executed, implements a data adaptation transmission method according to any of claims 1-7.