CN117041245A - Data exchange method, device, computer equipment and storage medium - Google Patents

Data exchange method, device, computer equipment and storage medium Download PDF

Info

Publication number
CN117041245A
CN117041245A CN202311085610.1A CN202311085610A CN117041245A CN 117041245 A CN117041245 A CN 117041245A CN 202311085610 A CN202311085610 A CN 202311085610A CN 117041245 A CN117041245 A CN 117041245A
Authority
CN
China
Prior art keywords
data
file
exchange
field
mark
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311085610.1A
Other languages
Chinese (zh)
Inventor
孙腾
王世泽
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Industrial Consumer Finance Co Ltd
Original Assignee
Industrial Consumer Finance Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Industrial Consumer Finance Co Ltd filed Critical Industrial Consumer Finance Co Ltd
Priority to CN202311085610.1A priority Critical patent/CN117041245A/en
Publication of CN117041245A publication Critical patent/CN117041245A/en
Pending legal-status Critical Current

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/06Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/50Network services
    • H04L67/56Provisioning of proxy services
    • H04L67/565Conversion or adaptation of application format or content

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The present application relates to a data exchange method, apparatus, computer device, storage medium and computer program product. The method is applied to data transfer terminals, and comprises the following steps: receiving a data file and a flag file from a data supply end, wherein the data file comprises first exchange data, and the flag file is used for describing the data characteristics of the first exchange data; analyzing the data file according to the mark file to obtain source exchange data; acquiring data subscription information from a data receiving end Generating second exchange data according to the data subscription information and the source exchange data; and sending the second exchange data to a data receiving end. By adopting the method, the convenience of data exchange can be improved.

Description

Data exchange method, device, computer equipment and storage medium
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a data exchange method, apparatus, computer device, and storage medium.
Background
At present, in the process of exchanging batch data among different systems, the method generally adopted is as follows: a certain file directory is shared among the systems needing data exchange, one of the systems is used as a data supply system, the generated data file is uploaded to the shared directory, and then the data consumption system directly downloads the file in the shared directory.
However, if a file uploaded by the data supply system has a structural change such as field increase or decrease, the corresponding data consumption system cannot analyze the data corresponding to the file, and at this time, all relevant data consumption systems are notified to perform a synchronous matching change on the file analysis rule, so that the complexity of data exchange is increased.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a data exchange method, apparatus, computer device, computer readable storage medium, and computer program product that can improve the convenience of data exchange.
In a first aspect, the present application provides a data exchange method. The method is applied to data transfer terminals, and comprises the following steps:
receiving a data file and a flag file from a data supply end, wherein the data file comprises first exchange data, and the flag file is used for describing the data characteristics of the first exchange data;
analyzing the data file according to the mark file to obtain source exchange data;
acquiring data subscription information from a data receiving end
Generating second exchange data according to the data subscription information and the source exchange data;
And sending the second exchange data to a data receiving end.
In one embodiment, the receiving the data file and the flag file from the data provider includes:
acquiring data release information from a data supply end, wherein the data release information comprises a file directory address;
according to the data release information, monitoring whether a mark file exists in the file directory address;
and receiving the mark file and a data file corresponding to the mark file under the condition that the mark file exists in the file directory address.
In one embodiment, the text length of the first exchange data is fixed, and the first exchange data includes a plurality of data fields, and the flag file is used to describe field characteristics of each of the data fields.
In one embodiment, the data subscription information includes a subscription field, and the generating second exchange data from the data subscription information and the source exchange data includes:
extracting the subscription field in the data subscription information;
and screening the source exchange data according to the subscription field to obtain second exchange data.
In one embodiment, the field characteristics include one or more of a field name, a field length, a field rank, and a field number.
In one embodiment, after receiving the data file and the flag file from the data provider, the method further includes:
selecting a data field in the data file;
checking the first exchange data according to the field characteristics of the mark file;
and under the condition that the verification is passed, analyzing the data file according to the mark file to obtain source exchange data.
In a second aspect, the application further provides a data exchange device. The device is applied to a data transfer terminal, and comprises:
the data receiving module is used for receiving a data file and a mark file from a data supply end, wherein the data file comprises first exchange data, and the mark file is used for describing the data characteristics of the first exchange data;
the data analysis module is used for analyzing the data file according to the mark file so as to obtain source exchange data;
a data subscription module for acquiring data subscription information from the data receiving end
The data generation module is used for generating second exchange data according to the data subscription information and the source exchange data;
and the data sending module is used for sending the second exchange data to a data receiving end.
In a third aspect, the present application also provides a computer device. The computer device comprises a memory storing a computer program and a processor which when executing the computer program performs the steps of:
receiving a data file and a flag file from a data supply end, wherein the data file comprises first exchange data, and the flag file is used for describing the data characteristics of the first exchange data;
analyzing the data file according to the mark file to obtain source exchange data;
acquiring data subscription information from a data receiving end
Generating second exchange data according to the data subscription information and the source exchange data;
and sending the second exchange data to a data receiving end.
In a fourth aspect, the present application also provides a computer-readable storage medium. The computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of:
Receiving a data file and a flag file from a data supply end, wherein the data file comprises first exchange data, and the flag file is used for describing the data characteristics of the first exchange data;
analyzing the data file according to the mark file to obtain source exchange data;
acquiring data subscription information from a data receiving end
Generating second exchange data according to the data subscription information and the source exchange data;
and sending the second exchange data to a data receiving end.
In a fifth aspect, the present application also provides a computer program product. The computer program product comprises a computer program which, when executed by a processor, implements the steps of:
receiving a data file and a flag file from a data supply end, wherein the data file comprises first exchange data, and the flag file is used for describing the data characteristics of the first exchange data;
analyzing the data file according to the mark file to obtain source exchange data;
acquiring data subscription information from a data receiving end
Generating second exchange data according to the data subscription information and the source exchange data;
And sending the second exchange data to a data receiving end.
According to the data exchange method, the data exchange device, the computer equipment, the storage medium and the computer program product, the first exchange data has the data characteristic, the data transfer terminal receives the corresponding mark file while receiving the data file, the data transfer terminal can acquire the data characteristic of the data transfer terminal, the characteristic is used for analyzing the data file, and the analyzed source exchange data can be obtained. The data file is structurally changed, the changed data characteristics can be recorded through the mark file and transmitted to the data transfer end along with the mark file, so that the first exchange data in the data file is analyzed through the mark file from the data supply end, the latest and all structural characteristics of the data file can be obtained, the data analysis logic is timely adjusted according to the structural change of the data file, synchronous adjustment of the data receiving end is not needed, namely the data receiving end is not required to be informed to change analysis rules in a matched mode, and therefore convenience of data exchange is improved. Further, after the data is analyzed by the data transfer terminal, the data receiving terminal obtains the data in a data subscription mode, so that the data receiving terminal can only obtain subscribed contents, the minimum consumption principle is met, the data safety is ensured, the data receiving terminal can also receive the analyzed second exchange data, the receiving terminal does not need to analyze, and the convenience of data exchange is further improved. Furthermore, the data receiving end does not need to follow the data supply end to carry out matched change operation, so that the error rate of the data receiving end for receiving the data can be reduced, and the data receiving end can accurately receive the required data.
Drawings
FIG. 1 is a diagram of an application environment for a data exchange method in one embodiment;
FIG. 2 is a flow chart of a data exchange method according to an embodiment;
FIG. 3 is a flowchart of step S202 in a data exchange method according to an embodiment;
FIG. 4 is a flowchart illustrating a data exchange method according to an embodiment of the present application in step S208;
FIG. 5 is a flow chart of a data exchange method according to another embodiment;
FIG. 6 is a block diagram of a data exchange device in one embodiment;
FIG. 7 is an internal block diagram of a computer device in one embodiment;
fig. 8 is an internal structural view of a computer device in another embodiment.
Detailed Description
The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
The data exchange method provided by the embodiment of the application can be applied to a data exchange system shown in fig. 1, and the data exchange system can comprise a data supply end 102, a data transfer end 104 and a data receiving end 106. Wherein the data provider 102 may communicate with the data relay 104 via a network. The data receiving end 106 may also communicate with the data relay end 104 via a network. The data storage system may store data that the data relay 104 needs to process. The data storage system may be integrated on the data transfer end 104, or may be placed on a cloud or other network server. The data storage system is used for storing data files and flag files from the data supply end so as to store first exchange data in the data files. The data storage system may also be used for parsed data, as well as data information from the data supply 102, the data receiving 106. The data supply end 102 and the data receiving end 106 may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, internet of things devices and portable wearable devices, and the internet of things devices may be smart speakers, smart televisions, smart air conditioners, smart vehicle devices, etc. The portable wearable device may be a smart watch, smart bracelet, headset, or the like. The data transfer terminal 104 may be implemented by a separate server or a server cluster formed by a plurality of servers, and similarly, the data supply terminal 102 and the data receiving terminal 106 may be implemented by a separate server or a server cluster formed by a plurality of servers.
In one embodiment, as shown in fig. 2, a data exchange method is provided, and the method is applied to the data transfer terminal in fig. 1, and is illustrated as an example, and includes the following steps S202 to S210:
step S202, a data file and a flag file are received from a data supply end, wherein the data file comprises first exchange data, and the flag file is used for describing data characteristics of the first exchange data.
The data supply end is a starting point in the data exchange system and is responsible for providing or generating data to be transmitted, and the data supply end can be a database, a file system, an application program or other data sources for generating, preparing or extracting data to ensure the accuracy and the integrity of the data.
In an exemplary embodiment, in a cluster formed by multiple devices, each device may exist as multiple roles at the same time, for example, in a data link, a first device is used as a data supply end, and a second device is used as a data receiving end, so that the first device, the second device and the data transfer end may form a data exchange system, and the construction of the data link is completed; in another data link, the second device is used as a data supply end, the first device is used as a data receiving end, and then the first device, the second device and the data transfer end can form another data exchange system, so that the construction of the data link is completed.
Further, the data file in the above step refers to a file including the first exchange data of the data supply end, where the first exchange data has a variety in terms of data format, encoding, structure, and the like. For example, the first exchange data may be stored and transferred in a table, tree structure, key-value, etc. different structures. Correspondingly, the data characteristics of the first exchange data refer to characteristics of the first exchange data in terms of data format, coding, structure and the like, the characteristics are recorded in the mark file, the mark file and the corresponding data file are acquired together, and the corresponding data characteristics of the first exchange data can be acquired at the same time when the first exchange data is acquired.
Illustratively, first, a receiving environment is prepared at the data transfer end, ensuring that there is sufficient storage space and appropriate rights to receive and process the data files and flag files. And then establishing a communication channel between the data transfer terminal and the data supply terminal, and receiving the data file and the mark file from the data supply terminal by using network connection, API call and other modes, further receiving the data file transmitted by the data supply terminal, and then storing the data file in a designated storage position, and simultaneously or immediately receiving the mark file from the data supply terminal. In addition, log information regarding the receipt of data may also be recorded for tracking and auditing.
Step S204, the data file is parsed according to the flag file to obtain source exchange data.
The flag file describes the data characteristics of the first exchange data in the data file, and the data characteristics are used for analyzing the data file, so that the first exchange data of unknown content can be analyzed into source exchange data of known content.
Illustratively, the flag file is first read from the storage location of the data relay peer. Analyzing the data characteristic related information in the mark file, extracting related information such as a data structure, a field name, a data type, a record separator and the like, then finding corresponding first exchange data content according to the file path or the name according to the related information extracted in the mark file, reading each row of the corresponding content of the first exchange data, analyzing each row according to the structure and the field information defined in the mark file, including cutting the data according to the field length, performing operations such as data type conversion according to the field type, reconstructing the analyzed data into the format of original data in the process of analyzing the data row by row, including reorganizing the field sequence, adding the separator and the like, and finally storing the reconstructed original data to a proper position, which can be a file system, a database and the like. In addition, in the process of analysis and reconstruction, the data can be checked and cleaned so as to ensure the accuracy and the integrity of the data.
Step S206, obtaining the data subscription information from the data receiving end.
The data subscription information is generated by the data receiving end in the data exchange system and is used for indicating the data transfer end to determine the content of the provided data and the way of providing the data. A instructional indication of which data to provide and how to provide. The data subscription information includes any one or more of data type, data generation time, data transmission format, data transmission frequency.
Illustratively, this information generally includes the needs and requirements of the data receiving end, and may also include how the data transfer end should process and transfer the data. The data subscription information may describe specific data content, such as a specific table, a field, a time range, etc., required by the data receiving end; the data subscription information can also specify the transmission mode of data, such as batch transmission, real-time streaming transmission and the like; the data subscription information may also contain requirements regarding data processing, such as data processing, conversion, merging, etc.; the data subscription information may also specify the frequency and timing of data transmissions, such as daily, weekly, monthly, or at a particular point in time; the data subscription information may also require that the data be transmitted in a specific format and code, or involve security and rights requirements for the data, such as encrypted transmissions, access rights controls, etc., so that the receiving end can properly parse and process the data.
Step S208, generating second exchange data according to the data subscription information and the source exchange data.
The data transfer end firstly needs to analyze the data subscription information received from the data receiving end, reads and analyzes the received data, extracts information such as data requirements, transmission modes, processing requirements and the like, extracts data needed by the data receiving end according to the analyzed data requirements, processes, converts or processes the source exchange data according to the requirements of the data receiving end, and assembles the processed and converted data into second exchange data. The second exchange data may be a file or a data stream, depending on the manner of data transmission.
Step S210, the second exchange data is sent to the data receiving end.
The data receiving end is an end point in the data exchange system and is responsible for receiving, processing and storing the data transmitted from the data transfer end, and storing the data into a target database, a file system or other storage media for subsequent use.
Illustratively, firstly, the data transfer terminal selects a proper data transmission protocol, such as a hypertext transmission protocol (Hypertext Transfer Protocol, HTTP), a file transmission protocol (File Transfer Protocol, FTP), a message queue telemetry transmission (Message Queuing Telemetry Transport, MQTT) and the like according to a preset rule of the data exchange system or according to requirements of usage scenarios and data subscription information, and establishes a connection with the data receiving terminal according to the selected transmission protocol, and the generated second exchange data is sent to the data receiving terminal through the selected transmission protocol, which may be transmission of a file, release of a message and the like.
In the data exchange method, the first exchange data has the data characteristic, the data transfer end receives the corresponding mark file while receiving the data file, the data transfer end can acquire the data characteristic of the data transfer end, and the characteristic is used for analyzing the data file, so that analyzed source exchange data can be obtained. The data file is structurally changed, and the changed data characteristics can be recorded through the mark file and transmitted to the data transfer terminal along with the mark file.
Therefore, the first exchange data in the data file is analyzed through the mark file from the data supply end, the latest and all structural characteristics of the data file can be obtained, so that the data analysis logic is adjusted timely according to the structural change of the data file, synchronous adjustment of the data receiving end is not needed, namely the data receiving end is not needed to be informed to change the analysis rule in a matching way, and the convenience of data exchange is improved. Further, after the data is analyzed by the data transfer terminal, the data receiving terminal obtains the data in a data subscription mode, so that the data receiving terminal can only obtain subscribed contents, the minimum consumption principle is met, the data safety is ensured, the data receiving terminal can also receive the analyzed second exchange data, the receiving terminal does not need to analyze, and the convenience of data exchange is further improved.
Furthermore, the data receiving end does not need to follow the data supply end to carry out matched change operation, so that the error rate of the data receiving end for receiving the data can be reduced, and the data receiving end can accurately receive the required data.
In one embodiment, as shown in fig. 3, step S202 includes the following steps S302 to S306:
step S302, obtaining data release information from the data provider, where the data release information includes a file directory address.
The data release information is generated by the data supply end and is used for indicating necessary information required by the data transfer end when the data transfer end obtains the data file and the mark file.
Illustratively, the data publishing information may be a table, a file, etc., which may include any one or more of an identity of the data provider, a data structure type, a data file name, a logo file identity.
Further, the data release information further includes a file directory address, where the file directory address is an address where the data supply end will unload the flag file, and the data file may be unloaded at the file directory address or at another address.
Further, the data distribution information may further include a schedule arrival time, wherein the schedule arrival time is time information indicating a time at which the index file is expected to arrive at the file directory address.
Step S304, according to the data release information, whether a mark file exists in the file directory address is monitored.
Illustratively, the data transfer end waits for the flag file to reach the preset time point of the file directory address according to the planned arrival time by using a timer, a timing task or other mechanisms provided by a programming language, and checks the file directory address to see whether the flag file exists at the preset time point or within a preset time range. If a flag file is found in the file directory address, the name identification of the flag file is compared to determine whether it is consistent with the data distribution information. After one monitoring operation is completed, the steps are continuously and circularly executed so as to ensure that the arrival of the mark file is captured at any time.
In step S306, in the case where the flag file exists in the file directory address, the flag file and the data file corresponding to the flag file are received.
Wherein the flag file and the data file corresponding to each other arrive at the specified file directory simultaneously or in succession in a short time, so that when the flag file arrives at the file directory address, the corresponding data file also exists in the specified file directory, and typically the data file arrives at the specified file directory earlier than the flag file. For example, when the flag file is successfully captured, the data transfer end may then match the data file corresponding to the flag file according to the specified file directory recorded by the data storage system.
In one embodiment, the text length of the first exchange data is fixed, and the first exchange data includes a plurality of data fields, and a flag file is used to describe field characteristics of each data field.
Wherein the data file containing the first exchange data is in a fixed length text format, wherein each record occupies a fixed number of bytes in the data file, each field is filled to a fixed width regardless of the actual content length of the field, and if the actual content length of the field is less than the width, a space or other filling character is used for filling, so that the field reaches the fixed length. Thus, each record has the same length and the field position is fixed.
Illustratively, a fixed length text formatted data file including a name field and an age field: wherein the width of each field is fixed, the name field occupies 15 characters wide and the age field occupies 5 characters wide, regardless of the actual content length. Spaces are used between fields to fill in to achieve a fixed length for the record.
In this embodiment, the first exchange data includes a plurality of data fields, and since each data field has a fixed width, the data fields are kept aligned in the file, so that the data features are more easily expressed and synchronized. And because the record lengths are the same, a specific position can be positioned through the data characteristics, so that the field value is resolved, and the efficiency of resolving and processing is improved. And because the field position and the width are fixed, complex separator recognition is not needed in the analysis process, so that the analysis process is relatively simple, and the convenience of data exchange is further improved. In addition, the fixed-length text format can better highlight the advantages in the data exchange process of the cross-operation platform and the cross-database, and the stable data structure can further provide convenience for later maintenance work.
In one embodiment, the field characteristics include one or more of a field name, a field length, a field rank, and a field number.
Illustratively, a data file in which the field names of the 1 st to 6 th bit data of the first exchange data are "client numbers" and the corresponding data are 000001; the field names of the 7 th bit to the 66 th bit data are 'customer names', and the corresponding data are Jack Ma; the field names of the 67 th bit to the 72 th bit data are "sex", and the corresponding data are MALE; the field names of the 73 rd bit to the 79 th bit data are "professional codes", the corresponding data are 0000001, the field names of the 80 th bit to the 87 th bit data are "industry codes", and the corresponding data are 00000001.
The field characteristics in the flag file of the above-mentioned data file include a field name, for example, "client number"; a field length, e.g., a 6-bit length; a field order, such as a first order; the number of fields, e.g., 5 fields. And the same field corresponds to different field characteristics.
In one embodiment, as shown in fig. 4, the data subscription information includes a subscription field, and step S208 includes the following steps S402 to S404:
step S402, extracting the subscription field in the data subscription information.
The subscription field is a field that the data receiving end needs to acquire from the data supplying end.
In the process of extracting the subscription field in the data subscription information, the data transfer terminal may directly extract the corresponding subscription field according to the identification information of the subscription field in the data subscription information, or may extract the field set of the corresponding type according to the type information of the subscription field in the data subscription information. Further, the identification information of the subscription field may be a unique identifier of the data field, the type information of the subscription field may be a unique identifier of the field type, and the data subscription information acquired by the data transfer end may include the unique identifier of the data field and the unique identifier of the field type.
Step S404, filtering the source exchange data according to the subscription field to obtain second exchange data.
Illustratively, the data transfer end may locate the location of the subscription field according to the identification information of the field, thereby extracting a specific field or set of fields required by the data receiving end. Further, in the process of locating a specific data field, the data transfer end can locate through a mapping relation between unique identification of a field/type and information such as a name of the specific field, and also can locate through information such as a field position, a field length, a field generation time and the like, so that the source exchange data is identified and screened, and second exchange data required by the data receiving end is obtained.
In this embodiment, the data transfer end generates the corresponding second exchange data for different data receiving ends, and each data receiving end only receives the required fields thereof, thereby satisfying the minimum consumption principle and guaranteeing the data security. Further, when the field is increased at the data supply end, the data transfer end uses the flag file to analyze the field into a known field, if the data subscription information provided by the data receiving end contains a unique identifier of the field type and the newly added field belongs to a corresponding type, the data transfer end can add the newly added field into the second exchange data through the identifier, the data receiving end does not need to change the data subscription information, and the convenience of data exchange is further improved.
In one embodiment, as shown in fig. 5, after step S202, the following steps S502 to S506 are further included:
step S502, selecting data fields in the data file.
The data transfer system may randomly select data fields in the current acquired data file, may perform full-size selection on the data fields in the current data file, may also perform screening according to the data field types in the data release information, and may perform key selection on the data fields with higher priority.
Step S504, checking the first exchange data according to the field characteristics of the mark file.
Illustratively, the flag file has recorded therein: the number of fields is 5, the single line length is 87, the number of record bars is 5, and all the characteristics can be used for carrying out integrity check on the number of record bars, the number of fields and the single line length in the data file. For example, a data file has exactly 5 records, and the value of the record number is consistent with the value of the record number in the mark file, and the data file is a complete data file.
Step S506, under the condition that the verification is passed, analyzing the data file according to the mark file to obtain the source exchange data.
When the verification is passed, the integrity of the data file meets the condition of data exchange, and the next operation can be started.
For example, after the first exchange data is checked, the source data file and the second exchange data may be checked according to the field characteristics of the flag file, where the checking step is consistent with the first exchange data, and will not be described herein.
In this embodiment, the data transfer terminal performs integrity verification on the exchanged data according to field features of the flag file, so that the data integrity can be improved while the convenience of the data exchange process is ensured, thereby reducing the possibility of data loss and data reconstruction.
In another embodiment, the data transfer terminal first obtains data release information from the data supply terminal, and the data release information includes an identity of the data supply terminal, a data structure type, a data file name, a flag file name, a file directory address, and a schedule time. The data transfer end uses a timing task to wait for a preset time point when the mark file reaches the file directory address according to the planned arrival time, and checks the file directory address within a preset time range, and when the mark file is found in the file directory address, the name identification of the mark file is compared with whether the name identification is consistent with the data release information or not, so that the arrival of the mark file is captured.
Then, the data transfer terminal receives the monitored flag file and the corresponding data file and stores the same. Wherein the data file comprises a file in a fixed length text format of a plurality of data fields. The field name of the 1 st bit to 6 th bit data of the first exchange data is 'client number', and the corresponding data is 000001; the field names of the 7 th bit to the 66 th bit data are 'customer names', and the corresponding data are Jack Ma; the field names of the 67 th bit to the 72 th bit data are "sex", and the corresponding data are MALE; the field names of the 73 rd bit to the 79 th bit data are "professional codes", the corresponding data are 0000001, the field names of the 80 th bit to the 87 th bit data are "industry codes", and the corresponding data are 00000001. The field characteristics in the markup file include field names, such as "customer number"; a field length, e.g., a 6-bit length; a field order, such as a first order; the number of fields, e.g., 5 fields. And the same field corresponds to different field characteristics.
Further, the flag file received by the data transfer terminal is recorded with: the number of fields is 5, the single line length is 87, the number of record bars is 5, and the characteristics are used for carrying out integrity check on the number of record bars, the number of fields and the single line length in the data file. For example, a data file has exactly 5 records, and the value of the record number is consistent with the value of the record number in the mark file, and the data file is a complete data file. After the verification is passed, the data file is analyzed according to the data characteristics of the mark file, and the known source data file can be obtained.
Next, the data transfer terminal obtains data subscription information from the data receiving terminal, wherein the data subscription information includes 2 subscription fields, namely a client number and a professional code. The data transfer terminal locates the 1 st bit to 6 th bit data and the 73 rd bit to 79 th bit data in the source data file according to the two subscription fields, so as to obtain the data required by the data receiving terminal, combines the data to generate second exchange data, and sends the second exchange data to the data receiving terminal to finish the data exchange.
It should be understood that, although the steps in the flowcharts related to the embodiments described above are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.
Based on the same inventive concept, the embodiment of the application also provides a data exchange device for realizing the above related data exchange method. The implementation of the solution provided by the device is similar to the implementation described in the above method, so the specific limitation in one or more embodiments of the data exchange device provided below may refer to the limitation of the data exchange method hereinabove, and will not be repeated herein.
In one embodiment, as shown in fig. 6, there is provided a data exchange apparatus comprising: a data receiving module 602, a data parsing module 604, a data subscribing module 606, a data generating module 608, and a data transmitting module 610, wherein:
a data receiving module 602, configured to receive a data file and a flag file from a data supply, where the data file includes first exchange data, and the flag file is used to describe a data feature of the first exchange data;
the data parsing module 604 is configured to parse the data file according to the flag file to obtain source exchange data;
a data subscription module 606, configured to obtain data subscription information from a data receiving end;
a data generation module 608 for generating second exchange data according to the data subscription information and the source exchange data;
The data transmitting module 610 is configured to transmit the second exchange data to the data receiving end.
In one embodiment, the data receiving module 602 includes:
the information acquisition sub-module is used for acquiring data release information from the data supply end, wherein the data release information comprises a file directory address;
the file monitoring sub-module is used for monitoring whether a mark file exists in the file directory address according to the data release information;
and the file receiving sub-module is used for receiving the mark file and the data file corresponding to the mark file under the condition that the mark file exists in the file directory address.
In one embodiment, the text length of the first exchange data is fixed, and the first exchange data includes a plurality of data fields, and a flag file is used to describe field characteristics of each data field.
In one embodiment, the data subscription information includes a subscription field and the data generation module 608 includes:
the field extraction sub-module is used for extracting subscription fields in the data subscription information;
and the field screening sub-module is used for screening the source exchange data according to the subscription field so as to obtain second exchange data.
In one embodiment, the field characteristics include one or more of a field name, a field length, a field rank, and a field number.
In one embodiment, the apparatus further comprises:
the field selection module is used for selecting data fields in the data file;
the field verification module is used for verifying the first exchange data according to the field characteristics of the mark file;
and the data exchange module is used for analyzing the data file according to the mark file under the condition that the verification is passed so as to obtain source exchange data.
The respective modules in the above-described data exchange apparatus may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In one embodiment, a computer device is provided, which may be a server, the internal structure of which may be as shown in fig. 7. The computer device includes a processor, a memory, an Input/Output interface (I/O) and a communication interface. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface is connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is used for storing data such as first exchange data, source exchange data and the like. The input/output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a data exchange method.
In one embodiment, a computer device is provided, which may be a terminal, and the internal structure thereof may be as shown in fig. 8. The computer device includes a processor, a memory, an input/output interface, a communication interface, a display unit, and an input means. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input/output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless mode can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement a data exchange method. The display unit of the computer device is used for forming a visual picture, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, can also be a key, a track ball or a touch pad arranged on the shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.
It will be appreciated by those skilled in the art that the structure shown in FIG. 8 is merely a block diagram of some of the structures associated with the present inventive arrangements and is not limiting of the computer device to which the present inventive arrangements may be applied, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.
In one embodiment, a computer device is provided comprising a memory and a processor, the memory having stored therein a computer program, the processor when executing the computer program performing the steps of: receiving a data file and a flag file from a data supply end, wherein the data file comprises first exchange data, and the flag file is used for describing the data characteristics of the first exchange data; analyzing the data file according to the mark file to obtain source exchange data; acquiring data subscription information from a data receiving end Generating second exchange data according to the data subscription information and the source exchange data; and sending the second exchange data to a data receiving end.
In one embodiment, the processor when executing the computer program further performs the steps of: acquiring data release information from a data supply end, wherein the data release information comprises a file directory address; according to the data release information, whether a mark file exists in the file directory address or not is monitored; in the case where the flag file exists in the file directory address, the flag file and the data file corresponding to the flag file are received.
In one embodiment, the processor when executing the computer program further performs the steps of: extracting subscription fields in the data subscription information; and screening the source exchange data according to the subscription field to obtain second exchange data.
In one embodiment, the processor when executing the computer program further performs the steps of: selecting data fields in a data file; checking the first exchange data according to the field characteristics of the mark file; and under the condition that verification is passed, analyzing the data file according to the mark file to obtain the source exchange data.
In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored which, when executed by a processor, carries out the steps of the method embodiments described above.
In an embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the steps of the method embodiments described above.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magnetic random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (Phase Change Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like. The databases referred to in the embodiments provided herein may include at least one of a relational database and a non-relational database. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processor referred to in the embodiments provided in the present application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, or the like, but is not limited thereto.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The foregoing examples illustrate only a few embodiments of the application and are described in detail herein without thereby limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of the application should be assessed as that of the appended claims.

Claims (10)

1. A method for data exchange, wherein the method is applied to a data transfer terminal, the method comprising:
receiving a data file and a flag file from a data supply end, wherein the data file comprises first exchange data, and the flag file is used for describing the data characteristics of the first exchange data;
analyzing the data file according to the mark file to obtain source exchange data;
Acquiring data subscription information from a data receiving end
Generating second exchange data according to the data subscription information and the source exchange data;
and sending the second exchange data to a data receiving end.
2. The method of claim 1, wherein receiving the data file and the flag file from the data provider comprises:
acquiring data release information from a data supply end, wherein the data release information comprises a file directory address;
according to the data release information, monitoring whether a mark file exists in the file directory address;
and receiving the mark file and a data file corresponding to the mark file under the condition that the mark file exists in the file directory address.
3. The method of claim 1, wherein the first exchange data has a fixed text length and comprises a plurality of data fields, and wherein the flag file is used to describe field characteristics of each of the data fields.
4. The method of claim 3, wherein the data subscription information includes a subscription field, and wherein generating second exchange data from the data subscription information and the source exchange data comprises:
Extracting the subscription field in the data subscription information;
and screening the source exchange data according to the subscription field to obtain second exchange data.
5. The method of claim 3, wherein the field characteristics include one or more of a field name, a field length, a field rank, and a field number.
6. The method according to any one of claims 3 to 5, further comprising, after receiving the data file and the flag file from the data provider:
selecting a data field in the data file;
checking the first exchange data according to the field characteristics of the mark file;
and under the condition that the verification is passed, analyzing the data file according to the mark file to obtain source exchange data.
7. A data exchange device, the device being applied to a data transfer terminal, the device comprising:
the data receiving module is used for receiving a data file and a mark file from a data supply end, wherein the data file comprises first exchange data, and the mark file is used for describing the data characteristics of the first exchange data;
The data analysis module is used for analyzing the data file according to the mark file so as to obtain source exchange data;
a data subscription module for acquiring data subscription information from the data receiving end
The data generation module is used for generating second exchange data according to the data subscription information and the source exchange data;
and the data sending module is used for sending the second exchange data to a data receiving end.
8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any of claims 1 to 6 when the computer program is executed.
9. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 6.
10. A computer program product comprising a computer program, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 6.
CN202311085610.1A 2023-08-28 2023-08-28 Data exchange method, device, computer equipment and storage medium Pending CN117041245A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311085610.1A CN117041245A (en) 2023-08-28 2023-08-28 Data exchange method, device, computer equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311085610.1A CN117041245A (en) 2023-08-28 2023-08-28 Data exchange method, device, computer equipment and storage medium

Publications (1)

Publication Number Publication Date
CN117041245A true CN117041245A (en) 2023-11-10

Family

ID=88628108

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311085610.1A Pending CN117041245A (en) 2023-08-28 2023-08-28 Data exchange method, device, computer equipment and storage medium

Country Status (1)

Country Link
CN (1) CN117041245A (en)

Similar Documents

Publication Publication Date Title
CN110781230B (en) Data access method, device and equipment
CN110297944B (en) Distributed XML data processing method and system
CN112328552A (en) Bottom layer data management method, system and computer readable storage medium
CN112559475A (en) Data real-time capturing and transmitting method and system
CN104202387A (en) Metadata recovery method and related device
CN108932640B (en) Method and device for processing orders
CN103778258B (en) A kind of sending, receiving method of database data, client, server
CN107480182A (en) A kind of spacecraft telemetry source tracing method and system
CN102387121A (en) Management server, image dissemination control system and image dissemination control method
CN110728118B (en) Cross-data-platform data processing method, device, equipment and storage medium
CN116910820A (en) Data report processing method, device, computer equipment and storage medium
CN117041245A (en) Data exchange method, device, computer equipment and storage medium
CN115185841A (en) System reconfiguration test method, device, equipment and storage medium
CN113111078A (en) Resource data processing method and device, computer equipment and storage medium
CN116263659A (en) Data processing method, apparatus, computer program product, device and storage medium
CN113037848A (en) File uploading method and system
CN117061582B (en) Vehicle management system interaction method based on data center and computer equipment
CN109547924B (en) Position information sharing method, device, equipment and storage medium
CN116881247A (en) Service identifier generation method, device, computer equipment and storage medium
US20150234908A1 (en) Techniques for data synchronization using compressive sensing
CN116974984A (en) Private cloud product document processing method and device and computer equipment
CN117834613A (en) Data transmission method, device, computer equipment and storage medium
CN115186020A (en) Data access storage processing method, system and storage medium
CN117493142A (en) Buried point processing method, buried point processing device, computer equipment and storage medium
CN115237768A (en) Financial certification testing method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination