CN117290312A - Information merging processing method and device - Google Patents

Information merging processing method and device Download PDF

Info

Publication number
CN117290312A
CN117290312A CN202311245376.4A CN202311245376A CN117290312A CN 117290312 A CN117290312 A CN 117290312A CN 202311245376 A CN202311245376 A CN 202311245376A CN 117290312 A CN117290312 A CN 117290312A
Authority
CN
China
Prior art keywords
client
information record
application partition
customer
service
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311245376.4A
Other languages
Chinese (zh)
Inventor
李小佳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Construction Bank Corp
CCB Finetech Co Ltd
Original Assignee
China Construction Bank Corp
CCB Finetech Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Construction Bank Corp, CCB Finetech Co Ltd filed Critical China Construction Bank Corp
Priority to CN202311245376.4A priority Critical patent/CN117290312A/en
Publication of CN117290312A publication Critical patent/CN117290312A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/16File or folder operations, e.g. details of user interfaces specifically adapted to file systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • G06F16/2228Indexing structures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23Updating
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2458Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/95Retrieval from the web
    • G06F16/955Retrieval from the web using information identifiers, e.g. uniform resource locators [URL]

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Software Systems (AREA)
  • Human Computer Interaction (AREA)
  • Computing Systems (AREA)
  • Fuzzy Systems (AREA)
  • Mathematical Physics (AREA)
  • Probability & Statistics with Applications (AREA)
  • Computational Linguistics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses an information merging processing method and device, and relates to the technical field of data processing. One embodiment of the method comprises the following steps: receiving a client information file to be merged, and acquiring a first client number and a second client number of each client; calculating a first application partition number according to the first client number, and inquiring a first client information record from a first application partition corresponding to the first application partition number; calculating a second application partition number according to the second client number, replacing the first client number in the first client information record with the second client number, and replacing the first application partition number with the second application partition number to obtain a second client information record; determining a second application partition, invoking the wholesale transaction service, inserting the second customer information record into the second application partition, and deleting the first customer information record from the first application partition. The embodiment ensures that the update of the new record and the old record is completed in one transaction, and ensures the consistency and the integrity of data.

Description

Information merging processing method and device
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a method and an apparatus for information merging processing.
Background
Information fusion is the process of combining multiple sets of information or data into one whole. Taking a bank as an example, the core banking systems all adopt distributed frame structures and database sub-databases and sub-tables in domestic technology stacks. In the distributed core banking system, the charging evidence, arrearage evidence, and charge related information such as relief evidence of a banking customer are not stored in a single database instance, but are stored in different database partitions in a uniform manner.
In the process of implementing the present invention, the inventor finds that at least the following problems exist in the prior art: when the customer information merging occurs in the conventional banking system, as the new and old customer numbers may be stored in different database partitions, the data integrity is difficult to ensure, and the new customer number records may be inserted, but the old customer number records fail to be deleted, and a spam mechanism is required to ensure the consistency and the integrity of the data, so that the requirements of the customer information merging service cannot be met in the prior art.
Disclosure of Invention
In view of this, embodiments of the present invention provide an information merging processing method and apparatus, which at least can solve the problem that in the prior art, a client information merging service cannot be implemented.
To achieve the above object, according to an aspect of an embodiment of the present invention, there is provided an information merging processing method, including:
receiving a client information file to be merged, and acquiring a first client number and a second client number of each client in the client information file; wherein the first customer number is marked and deleted;
calculating a first application partition number according to the first client number, and inquiring a first client information record corresponding to the first client number from a first application partition corresponding to the first application partition number;
calculating a second application partition number according to the second client number, replacing the first client number in the first client information record with the second client number, and replacing the first application partition number with the second application partition number to obtain a second client information record;
determining a second application partition corresponding to the second application partition number, invoking the wholesale transfer transaction service, inserting the second customer information record into the second application partition, and deleting the first customer information record from the first application partition.
Optionally, after the obtaining the first client number and the second client number of each client in the client information file, the method further includes:
Acquiring a route index type configured for a second client number in advance, determining a route index fragment corresponding to the route index type, and synchronizing the second client number into the route index fragment in response to the fact that the second client number is not found in the route index fragment;
the invoking the wholesale transaction service, inserting the second customer information record into the second application partition, and deleting the first customer information record from the first application partition, comprising:
calling a batch transfer transaction service, and determining a newly added micro service resource positioning address URL and a deleted micro service URL corresponding to the route index fragment;
calling the new micro-service according to the URL of the new micro-service, and inserting a second client information record into a second application partition; and
and calling the deletion micro-service according to the deletion micro-service URL, and deleting the first client information record from the first application partition.
Optionally, the calculating the first application partition number according to the first client number includes:
acquiring a first numerical value of a first preset number of digits positioned in front from a first customer number, and determining an application fragment corresponding to the first numerical value;
and obtaining a second numerical value of a second preset number of digits from the first client number, and arranging the second numerical value in a reverse order to obtain a third numerical value, and determining a first application partition number corresponding to the third numerical value in the application partition.
Optionally, after the obtaining the second client information record, the method further includes:
generating new and old customer information record files based on the first customer information record and the second customer information record;
and according to the file format of the batch transfer transaction service, merging the customer information into a transaction name to be used as a header record, storing the new and old customer information record files into a data area, and generating a batch transfer file.
Optionally, the method further comprises:
monitoring the processing result of each client information record to screen the processing result as a failed client information record set;
triggering a re-processing mechanism, receiving a modification operation for each client information record in the client information record set, and then executing re-merging processing for the modified client information records.
To achieve the above object, according to another aspect of the embodiments of the present invention, there is provided an information merge processing apparatus including:
the receiving module is used for receiving the client information file to be merged and obtaining a first client number and a second client number of each client in the client information file; wherein the first customer number is marked and deleted;
the processing module is used for calculating a first application partition number according to the first client number, and inquiring a first client information record corresponding to the first client number from a first application partition corresponding to the first application partition number;
Calculating a second application partition number according to the second client number, replacing the first client number in the first client information record with the second client number, and replacing the first application partition number with the second application partition number to obtain a second client information record;
determining a second application partition corresponding to the second application partition number, invoking the wholesale transfer transaction service, inserting the second customer information record into the second application partition, and deleting the first customer information record from the first application partition.
Optionally, the apparatus further includes a synchronization module configured to:
acquiring a route index type configured for a second client number in advance, determining a route index fragment corresponding to the route index type, and synchronizing the second client number into the route index fragment in response to the fact that the second client number is not found in the route index fragment;
the processing module is used for:
calling a batch transfer transaction service, and determining a newly added micro service resource positioning address URL and a deleted micro service URL corresponding to the route index fragment;
calling the new micro-service according to the URL of the new micro-service, and inserting a second client information record into a second application partition; and
and calling the deletion micro-service according to the deletion micro-service URL, and deleting the first client information record from the first application partition.
Optionally, the processing module is configured to:
acquiring a first numerical value of a first preset number of digits positioned in front from a first customer number, and determining an application fragment corresponding to the first numerical value;
and obtaining a second numerical value of a second preset number of digits from the first client number, and arranging the second numerical value in a reverse order to obtain a third numerical value, and determining a first application partition number corresponding to the third numerical value in the application partition.
Optionally, the processing module is further configured to:
generating new and old customer information record files based on the first customer information record and the second customer information record;
and according to the file format of the batch transfer transaction service, merging the customer information into a transaction name to be used as a header record, storing the new and old customer information record files into a data area, and generating a batch transfer file.
Optionally, the processing module is further configured to:
monitoring the processing result of each client information record to screen the processing result as a failed client information record set;
triggering a re-processing mechanism, receiving a modification operation for each client information record in the client information record set, and then executing re-merging processing for the modified client information records.
To achieve the above object, according to still another aspect of the embodiments of the present invention, there is provided an information merging processing electronic device.
The electronic equipment of the embodiment of the invention comprises: one or more processors; and the storage device is used for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors are enabled to realize the information merging processing method.
To achieve the above object, according to still another aspect of the embodiments of the present invention, there is provided a computer-readable medium having stored thereon a computer program which, when executed by a processor, implements any one of the above-described information merging processing methods.
To achieve the above object, according to still another aspect of an embodiment of the present invention, there is provided a computer program product. The computer program product of the embodiment of the invention comprises a computer program, and the program is executed by a processor to realize the information merging processing method provided by the embodiment of the invention.
According to the solution provided by the present invention, one embodiment of the above invention has the following advantages or beneficial effects: based on the client number, calculating the application partition number, possibly storing the old client information record and the new client information record in different application partitions, and executing different processing logics respectively, so as to realize the purpose of merging the new client information and the old client information across the partitions based on the client number, and the whole processing process can ensure that the record update of the new client number and the old client number is processed and completed in one transaction, and ensure the consistency and the integrity of data.
Further effects of the above-described non-conventional alternatives are described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a schematic flow diagram of an information merging processing method according to an embodiment of the present invention;
FIG. 2 is a flow chart of an alternative information merging processing method according to an embodiment of the present invention;
FIG. 3 is a flow chart of another alternative information merging processing method according to an embodiment of the present invention;
FIG. 4 is a schematic overall flow diagram of a specific information merging processing method according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of main modules of an information merging processing apparatus according to an embodiment of the present invention;
FIG. 6 is an exemplary system architecture diagram in which embodiments of the present invention may be applied;
fig. 7 is a schematic diagram of a computer system suitable for use in implementing a mobile device or server of an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present invention will now be described with reference to the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding, and are to be considered merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
It is noted that embodiments of the invention and features of the embodiments may be combined with each other without conflict. In the technical scheme of the invention, the related aspects of acquisition, analysis, use, transmission, storage and the like of the personal information of the user accord with the regulations of related laws and regulations, are used for legal and reasonable purposes, are not shared, leaked or sold outside the legal use aspects and the like, and are subjected to supervision and management of a supervision department. Necessary measures should be taken for the personal information of the user to prevent illegal access to such personal information data, ensure that personnel having access to the personal information data comply with the regulations of the relevant laws and regulations, and ensure the personal information of the user.
Once these user personal information data are no longer needed, the risk should be minimized by limiting or even prohibiting the data collection and/or deletion. User privacy is protected, when applicable, by de-identifying the data, including in some related applications, such as by removing a particular identifier (e.g., date of birth, etc.), controlling the amount or specificity of stored data (e.g., collecting location data at a city level rather than at a specific address level), controlling how the data is stored, and/or other methods.
In the original bank core system, the charge evidence, arrearage evidence, evidence-free and other charge related information of a bank client are stored in a single database example in a partition mode by taking the client as a dimension and taking the client number as a main key. When the merging of the client information occurs, the update client number information processing batch processing module can complete the update of the new and old client numbers only by accessing one data inventory instance, and specifically: based on the input files (including new and old customer numbers, etc.) of the merging of the customer information, the file records are read one by one through a pure batch processing component, the records of the customer expense evidence table are queried by the old customer numbers, the new records are inserted into the warehouse table after the old customer numbers are replaced by the new customer numbers, and the old records are deleted after success.
The distributed core system of bank is an improvement to original core system. Distributed is a distributed application solution built based on a distributed framework, which is physically a loosely coupled system of multiple processors interconnected by communication lines. The rest of the physical machines and corresponding resources are remote from the perspective of a single physical machine in the system, with only its own resources being local.
In the existing distributed bank core system design, a pure batch processing mode is adopted: and splitting the input files of the merging of the client information according to the fragments to which the three digits belong before the client number, generating the files to be processed corresponding to the new client number, and executing the input files in one-to-one correspondence with each fragment operation. Because the new file and the old file are executed separately, the processing of the new file and the old file cannot be guaranteed to be completed in one transaction.
The scheme uses the CENTER CUT mode to replace the existing pure batch processing mode, the existing pure batch processing mode can not realize the atomicity of data processing, and the processing of new and old files can not be guaranteed to be completed in one transaction. The central CUT mode is a batch online adjustment mode, online transaction is natural and provided with transaction processing, and processing of different fragments can be ensured to be successful or failed simultaneously, so that the integrity of data to be processed is ensured.
Referring to fig. 1, a main flowchart of an information merging processing method provided by an embodiment of the present invention is shown, including the following steps:
s101: receiving a client information file to be merged, and acquiring a first client number and a second client number of each client in the client information file; wherein the first customer number is marked and deleted;
S102: calculating a first application partition number according to the first client number, and inquiring a first client information record corresponding to the first client number from a first application partition corresponding to the first application partition number;
s103: calculating a second application partition number according to the second client number, replacing the first client number in the first client information record with the second client number, and replacing the first application partition number with the second application partition number to obtain a second client information record;
s104: determining a second application partition corresponding to the second application partition number, invoking the wholesale transfer transaction service, inserting the second customer information record into the second application partition, and deleting the first customer information record from the first application partition.
In the above embodiments, in the fields of computer science and information management, information fusion is commonly used to integrate and process information from different sources in order to better analyze, manage and utilize such information. Merging may involve aspects such as merging data in different databases, integrating multiple files or documents, or aggregating the content of multiple sources into a unified interface. In summary, information merging aims to integrate scattered information, providing more comprehensive, consistent and useful results.
For step S101, in addition to the client information, a copy of the information is stored in the distributed system of the bank, due to the information performance requirement. After the host computer finishes the merging of the client information, the copy information of the core banking system needs to be synchronously updated. The platform receives a copy of the client information, namely a client information file, wherein the file comprises the new and old client numbers, client properties (public and private) and the like of each client, and the copy is shown in table 1:
TABLE 1
Zhang San A B
Li Si C D
Wang Wu E F
It should be noted that, here, the old client number is a number that has been used before, and the new client number may be a client number that has been used before. For example, a customer sheet three has two customer numbers A and B, the present requirement is to merge customer information under number B under number A, which is not newly generated. Therefore, in order to identify which client information under a client number in a file needs to be merged, the client number may be marked and deleted to perform number distinction, for example, the number B may be marked and deleted, before modification, etc., and here, it is preferable to delete the number a, or the number a may not be marked, or may be marked and merged, after modification, etc.
For step S102, the client number in this embodiment is exemplified by 18 characters, and the number B is taken as the first client number to be deleted, the number a is taken as the second client number to be merged, and it is assumed that the number B is 000000000000000001, and the number a is 00000000000000002.
The application fragment number is calculated from the customer number, and for number B, the application fragment where 000 is located is determined from the first three digits (i.e., the first preset number of digits, which is just an example, actually adjustable) taken in the order from front to back. In actual operation, the responsible range of each application slice needs to be preset, such as slice 1: [000-010], partition 2: (010-020 ], segment 3 (020-030) … …. Thus, it is determined that number B is in segment 1.
After determining the application tile number, the partition number in the tile may be determined. For number B, the last three bits (i.e., the second predetermined number of bits, for example only, actually adjustable) are truncated according to a back-to-front order, here 001. The reverse processing is performed on 001 to obtain 100, the application partition number is usually set to 7 bits, and the first four bits are set to 0000, so that the application partition number is 0000100, and thus the number B is stored in the 0000100 application partition of the partition 1, and then the old client information record corresponding to the number B is obtained from the 0000100 application partition of the partition 1, where the number of records is one or more, such as the client expense record.
For step S103, the application fragment number is calculated from the client number, and for number a, the application fragment where 000 is located is determined from the first three digits, here 000, cut in the order from front to back. In actual operation, the responsible range of each application slice needs to be preset, such as slice 1: [000-010], partition 2: (010-020 ], segment 3 (020-030) … …. Thus, it is determined that number A is in segment 1.
After determining the application tile number, the partition number in the tile may be determined. For number a, the last three bits, here 002, are truncated in a back-to-front order. The reverse processing of 002 gives 200, the application partition number is typically set to 7 bits, the first four bits are 0000, so here the application partition number 0000200 is obtained, and thus the number a is stored in 0000200 application partition of partition 1.
Since the client information record of the number B is stored under the number a, the number B in the old client information record corresponding to the number B needs to be replaced with the number a, and the application partition number 0000100 needs to be replaced with 0000200, so that a new client information record is obtained. New and old customer information record files are then generated based on the old customer information record and the new customer information record, such as shown in table 2:
TABLE 2
Zhang San A New customer information record B Old customer information record
Li Si C New customer information record D Old customer information record
Wang Wu E New customer information record F Old customer information record
For step S104, the present solution uses the CENTER CUT to BATCH and connect transaction service, and the CENTER CUT provides the user or operator with on-line input in the BATCH (BATCH) environment, performs BATCH (BATCH) processing, and uses on-line program for execution.
After the new and old customer information record files are generated, the scheme can also use a fixed format of the CENTER CUT, a header record and a data area, wherein the header record comprises legal information, a customer information merging transaction name and a transaction mechanism, and also can only comprise the customer information merging transaction name, and the data area comprises the new and old customer information record files to generate a batch transfer file. The transaction name is herein a logical interface to the business function, such as a customer information merge transaction name. Here, the legal person information may be the name of a banking institution, and the transaction institution may be an institution for the banking to perform the merging of the customer information, such as a headquarter, a branch, and a website. See table 3 for batch transfer file format:
TABLE 3 Table 3
Invoking the CENTER CUT wholesale transaction service, inserting the new customer information record into the 0000200 application partition of partition 1, and deleting the old customer information record from the 0000100 application partition of partition 1.
The method provided by the embodiment can finish the merging of the client information of the distributed bank core system in an online transaction mode, can ensure that the record updating of the new client number and the old client number is finished in one transaction, can not cause the condition that the new client number and the old client number exist simultaneously, ensures the consistency and the integrity of data, does not need to use complex split input file logic as a whole, and has the advantage of simple and convenient operation.
Referring to fig. 2, a flowchart of an optional information merging processing method according to an embodiment of the present invention is shown, including the following steps:
s201: receiving a client information file to be merged, and acquiring a first client number and a second client number of each client in the client information file; wherein the first customer number is marked and deleted;
s202: acquiring a route index type configured for a second client number in advance, determining a route index fragment corresponding to the route index type, and synchronizing the second client number into the route index fragment in response to the fact that the second client number is not found in the route index fragment;
s203: calculating a first application partition number according to the first client number, and inquiring a first client information record corresponding to the first client number from a first application partition corresponding to the first application partition number;
s204: calculating a second application partition number according to the second client number, replacing the first client number in the first client information record with the second client number, and replacing the first application partition number with the second application partition number to obtain a second client information record;
s205: calling a batch transfer transaction service, and determining a newly added micro service resource positioning address URL and a deleted micro service URL corresponding to the route index fragment;
S206: determining a second application partition corresponding to the second application partition number, calling the newly added micro service according to the newly added micro service URL, and inserting a second client information record into the second application partition;
s207: and calling the deletion micro-service according to the deletion micro-service URL, and deleting the first client information record from the first application partition.
In the above embodiment, the steps S201, S203, S204 may be described with reference to fig. 1, and will not be described again.
For step S202, when the second client number, i.e., the new client number, is newly generated, the platform needs to synchronize the newly added client number in the route index library. The client numbers that have been previously generated and used are stored in the route index library, and since it is not determined whether a new client number has been previously generated, it is necessary to retrieve whether the new client number exists in the route index library.
It should be noted that, in the banking system, different identifiers may be generated for the client in different services, for example, the client number is used for merging the client information, and the client account (multiple accounts can be associated under one account) is used for the client transaction, so that when the client number is generated, the routing index type such as APD03 can be marked to indicate that the client number is only used for merging the client information.
Different route index fragments can be set in the route index library according to different route index types, such as setting a route index fragment 1 according to APD03, setting a route index fragment 2 according to APD04, and client numbers are stored in the route index fragment 1 so as to realize fragment management of the route index, and the route indexes of different fragments are mutually independent, so that whether the second client number exists or not can be directly searched from the route index fragment 1, and searching efficiency is improved.
For steps S205 to S207, the CENTER CUT service may directly determine a newly added micro service resource location address URL (Uniform Resource Locator ) and a deleted micro service URL corresponding to the route index fragment 1, and if the query result is not present, the error reporting process may be performed.
The CENTER CUT service calls the CENTER CUT transaction through the batch job processing module, reads the new and old customer information record files to process the customer merging information: 1) Calling a customer information merging new micro-addition service, and inserting a new customer information record of a new customer number into a corresponding fragment, such as a 0000200 application partition of the fragment 1; 2) And calling the customer information merging and deleting micro service, and recording the old customer information of the old customer number in the corresponding fragment for deleting, such as deleting from the 0000100 application partition of the fragment 1.
The method provided by the embodiment uses a distributed route index mechanism to set route index fragments to independently manage client numbers, and sets newly added micro services and deleted micro services corresponding to the route index fragments to realize independent micro services aiming at merging of client information and realize independent management.
Referring to fig. 3, a flowchart of another optional information merging processing method according to an embodiment of the present invention is shown, including the following steps:
s301: receiving a client information file to be merged, and acquiring a first client number and a second client number of each client in the client information file; wherein the first customer number is marked and deleted;
s302: calculating a first application partition number according to the first client number, and inquiring a first client information record corresponding to the first client number from a first application partition corresponding to the first application partition number;
s303: calculating a second application partition number according to the second client number, replacing the first client number in the first client information record with the second client number, and replacing the first application partition number with the second application partition number to obtain a second client information record;
s304: determining a second application partition corresponding to the second application partition number, invoking a wholesale transfer transaction service, inserting a second customer information record into the second application partition, and deleting the first customer information record from the first application partition;
S305: monitoring the processing result of each client information record to screen the processing result as a failed client information record set;
s306: triggering a re-processing mechanism, receiving a modification operation for each client information record in the client information record set, and then executing re-merging processing for the modified client information records.
In the above embodiment, the steps S301 to S304 are described with reference to fig. 1, and are not described herein.
For steps S305 and S306, CENTER CUT transaction post-processing: and after the error log information is written after the error record file is screened, the error log can be checked by subscribing the alarm information through the alarm platform. Therefore, whether deleting or inserting is carried out, the scheme needs to monitor the processing result of each client information record, screen out the record of which the processing result is failure, generate error log information based on the failure record and carry out alarm processing on the error log information. For example, 2023-04-03-customer three has 100 record processing failures.
For the processing of the failed customer information record, a reprocessing mechanism can be triggered, assuming 200 records in the new and old customer information record files, if a failure condition occurs when the 51 th record is processed, recording, continuing to process the 52 th data, simultaneously transmitting the 51 th record and an error log to a worker, and after the worker modifies the record, clicking a reprocessing option by the worker to trigger a re-merging processing mechanism for the 51 th customer information record.
According to the method provided by the embodiment, when the online transaction is failed, the operation can be restarted, and the failed transaction can be modified and re-merged, so that the information merging success rate is improved as much as possible.
Referring to fig. 4, an overall flow chart of another specific information merging processing method according to an embodiment of the present invention is shown, including the following steps:
1. the client information component receives a copy (file) of the client information, from which a new client number and an old client number are obtained;
2. acquiring a route index type configured for a new client number in advance, determining a route index fragment pka corresponding to the route index type, and synchronizing the new client number into the route index fragment in response to no new client number being inquired in the route index fragment; the route index fragments are stored in a distributed private service library, and interact with other components by an integration service layer;
3. determining a first fragment number according to the first three digits of the old client number, and determining a first application partition number according to the reverse order of the last three digits, and inquiring old client fee information records such as charging evidence, arrearage evidence and exemption evidence;
4. determining a second segment number according to the first three digits of the new client number, and determining a second application segment number in reverse order of the last three digits;
5. Replacing an old client number in the old client cost information record with a new client number, replacing a first application partition number with a second application partition number to obtain a new client cost information record, and generating a new and old client information record file according to the new and old client cost information record;
6. CENTER CUT input file preparation: and reading the new and old customer information record files through a batch processing operation processing module, and assigning the information of the infusion person, the trade name, the trade mechanism, the customer merging record information and the like according to the CENTER CUT format.
7. CENTER CUT transaction processing: calling a CENTER CUT transaction through a batch job processing module, and determining a newly added micro service URL and a deleted micro service URL corresponding to the route index fragment;
8. the new micro-service (i.e. execution unit) is invoked according to the new micro-service URL, the new customer information record is inserted into the second application partition, and the deletion micro-service (i.e. execution unit) is invoked according to the deletion micro-service URL, the old customer information record is deleted from the first application partition.
9. CENTER CUT transaction post-processing: and screening the error record file and writing error log information. The error log may be viewed by subscribing to the alert information through the alert platform.
The method provided by the embodiment of the invention uses a distributed routing index mechanism and a CENTER CUT mechanism, and has at least the following advantages compared with the prior art:
1. The client expense information merging is completed in an online transaction mode, so that record updating of new and old client numbers can be guaranteed to be processed and completed in one transaction, the condition that new and old records exist simultaneously can not occur, and the consistency of data is guaranteed;
2. when the client information data volume is increased and the expansion fragmentation is needed, such as route index fragmentation and application fragmentation, the client merging processing logic does not need to be changed, and the client information data volume can be dynamically processed through a route index mechanism completely without maintaining complex splitting logic;
3. and when the online transaction fails to be sent, and when the operation is restarted, the RESTART mechanism is supported, and the failed transaction can be modified and re-merged, so that the information merging success rate is improved as much as possible.
Referring to fig. 5, a schematic diagram of main modules of an information merging processing apparatus 500 according to an embodiment of the present invention is shown, including:
a receiving module 501, configured to receive a client information file to be merged, and obtain a first client number and a second client number of each client in the client information file; wherein the first customer number is marked and deleted;
the processing module 502 is configured to calculate a first application partition number according to the first client number, and query a first client information record corresponding to the first client number from a first application partition corresponding to the first application partition number;
Calculating a second application partition number according to the second client number, replacing the first client number in the first client information record with the second client number, and replacing the first application partition number with the second application partition number to obtain a second client information record;
determining a second application partition corresponding to the second application partition number, invoking the wholesale transfer transaction service, inserting the second customer information record into the second application partition, and deleting the first customer information record from the first application partition.
The implementation device of the invention also comprises a synchronization module for:
acquiring a route index type configured for a second client number in advance, determining a route index fragment corresponding to the route index type, and synchronizing the second client number into the route index fragment in response to the fact that the second client number is not found in the route index fragment;
the processing module 502 is configured to:
calling a batch transfer transaction service, and determining a newly added micro service resource positioning address URL and a deleted micro service URL corresponding to the route index fragment;
calling the new micro-service according to the URL of the new micro-service, and inserting a second client information record into a second application partition; and
and calling the deletion micro-service according to the deletion micro-service URL, and deleting the first client information record from the first application partition.
In the embodiment of the present invention, the processing module 502 is configured to:
acquiring a first numerical value of a first preset number of digits positioned in front from a first customer number, and determining an application fragment corresponding to the first numerical value;
and obtaining a second numerical value of a second preset number of digits from the first client number, and arranging the second numerical value in a reverse order to obtain a third numerical value, and determining a first application partition number corresponding to the third numerical value in the application partition.
In the embodiment of the present invention, the processing module 502 is further configured to:
generating new and old customer information record files based on the first customer information record and the second customer information record;
and according to the file format of the batch transfer transaction service, merging the customer information into a transaction name to be used as a header record, storing the new and old customer information record files into a data area, and generating a batch transfer file.
In the embodiment of the present invention, the processing module 502 is further configured to:
monitoring the processing result of each client information record to screen the processing result as a failed client information record set;
triggering a re-processing mechanism, receiving a modification operation for each client information record in the client information record set, and then executing re-merging processing for the modified client information records.
In addition, the implementation of the apparatus in the embodiments of the present invention has been described in detail in the above method, so that the description is not repeated here.
Fig. 6 shows an exemplary system architecture 600 in which embodiments of the invention may be applied, including terminal devices 601, 602, 603, a network 604, and a server 605 (by way of example only).
The terminal devices 601, 602, 603 may be various electronic devices having a display screen and supporting web browsing, are installed with various communication client applications, and a user may interact with the server 605 through the network 604 using the terminal devices 601, 602, 603 to receive or transmit messages, etc.
The network 604 is used as a medium to provide communication links between the terminal devices 601, 602, 603 and the server 605. The network 604 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
The server 605 may be a server providing various services, and it should be noted that, the method provided by the embodiment of the present invention is generally executed by the server 605, and accordingly, the apparatus is generally disposed in the server 605.
It should be understood that the number of terminal devices, networks and servers in fig. 6 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 7, there is illustrated a schematic diagram of a computer system 700 suitable for use in implementing an embodiment of the present invention. The terminal device shown in fig. 7 is only an example, and should not impose any limitation on the functions and the scope of use of the embodiment of the present invention.
As shown in fig. 7, the computer system 700 includes a Central Processing Unit (CPU) 701, which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 702 or a program loaded from a storage section 708 into a Random Access Memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the system 700 are also stored. The CPU 701, ROM 702, and RAM 703 are connected to each other through a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.
The following components are connected to the I/O interface 705: an input section 706 including a keyboard, a mouse, and the like; an output portion 707 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like; a storage section 708 including a hard disk or the like; and a communication section 709 including a network interface card such as a LAN card, a modem, or the like. The communication section 709 performs communication processing via a network such as the internet. The drive 710 is also connected to the I/O interface 705 as needed. A removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 710 as necessary, so that a computer program read therefrom is mounted into the storage section 708 as necessary.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication portion 709, and/or installed from the removable medium 711. The above-described functions defined in the system of the present invention are performed when the computer program is executed by a Central Processing Unit (CPU) 701.
The computer readable medium shown in the present invention may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules involved in the embodiments of the present invention may be implemented in software or in hardware. The described modules may also be provided in a processor, for example, as: a processor comprises a receiving module and a processing module. The names of these modules do not in any way constitute a limitation of the module itself, for example, a processing module may also be described as a "merging module".
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be present alone without being fitted into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to perform any of the information fusion processing methods described above.
The computer program product of the present invention comprises a computer program which, when executed by a processor, implements the information merging processing method in the embodiment of the present invention.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives can occur depending upon design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.

Claims (13)

1. An information merging processing method is characterized by comprising the following steps:
receiving a client information file to be merged, and acquiring a first client number and a second client number of each client in the client information file; wherein the first customer number is marked and deleted;
Calculating a first application partition number according to the first client number, and inquiring a first client information record corresponding to the first client number from a first application partition corresponding to the first application partition number;
calculating a second application partition number according to the second client number, replacing the first client number in the first client information record with the second client number, and replacing the first application partition number with the second application partition number to obtain a second client information record;
determining a second application partition corresponding to the second application partition number, invoking the wholesale transfer transaction service, inserting the second customer information record into the second application partition, and deleting the first customer information record from the first application partition.
2. The method of claim 1, wherein after said obtaining the first customer number and the second customer number for each customer in the customer information file, the method further comprises:
acquiring a route index type configured for a second client number in advance, determining a route index fragment corresponding to the route index type, and synchronizing the second client number into the route index fragment in response to the fact that the second client number is not found in the route index fragment;
The invoking the wholesale transaction service, inserting the second customer information record into the second application partition, and deleting the first customer information record from the first application partition, comprising:
calling a batch transfer transaction service, and determining a newly added micro service resource positioning address URL and a deleted micro service URL corresponding to the route index fragment;
calling the new micro-service according to the URL of the new micro-service, and inserting a second client information record into a second application partition; and
and calling the deletion micro-service according to the deletion micro-service URL, and deleting the first client information record from the first application partition.
3. The method of claim 1, wherein calculating a first application partition number from a first client number comprises:
acquiring a first numerical value of a first preset number of digits positioned in front from a first customer number, and determining an application fragment corresponding to the first numerical value;
and obtaining a second numerical value of a second preset number of digits from the first client number, and arranging the second numerical value in a reverse order to obtain a third numerical value, and determining a first application partition number corresponding to the third numerical value in the application partition.
4. The method of claim 1, wherein after said obtaining a second customer information record, the method further comprises:
Generating new and old customer information record files based on the first customer information record and the second customer information record;
and according to the file format of the batch transfer transaction service, merging the customer information into a transaction name to be used as a header record, storing the new and old customer information record files into a data area, and generating a batch transfer file.
5. The method according to any one of claims 1-4, further comprising:
monitoring the processing result of each client information record to screen the processing result as a failed client information record set;
triggering a re-processing mechanism, receiving a modification operation for each client information record in the client information record set, and then executing re-merging processing for the modified client information records.
6. An information merging processing apparatus, comprising:
the receiving module is used for receiving the client information file to be merged and obtaining a first client number and a second client number of each client in the client information file; wherein the first customer number is marked and deleted;
the processing module is used for calculating a first application partition number according to the first client number, and inquiring a first client information record corresponding to the first client number from a first application partition corresponding to the first application partition number;
Calculating a second application partition number according to the second client number, replacing the first client number in the first client information record with the second client number, and replacing the first application partition number with the second application partition number to obtain a second client information record;
determining a second application partition corresponding to the second application partition number, invoking the wholesale transfer transaction service, inserting the second customer information record into the second application partition, and deleting the first customer information record from the first application partition.
7. The apparatus of claim 6, further comprising a synchronization module to:
acquiring a route index type configured for a second client number in advance, determining a route index fragment corresponding to the route index type, and synchronizing the second client number into the route index fragment in response to the fact that the second client number is not found in the route index fragment;
the processing module is used for:
calling a batch transfer transaction service, and determining a newly added micro service resource positioning address URL and a deleted micro service URL corresponding to the route index fragment;
calling the new micro-service according to the URL of the new micro-service, and inserting a second client information record into a second application partition; and
And calling the deletion micro-service according to the deletion micro-service URL, and deleting the first client information record from the first application partition.
8. The apparatus of claim 6, wherein the processing module is configured to:
acquiring a first numerical value of a first preset number of digits positioned in front from a first customer number, and determining an application fragment corresponding to the first numerical value;
and obtaining a second numerical value of a second preset number of digits from the first client number, and arranging the second numerical value in a reverse order to obtain a third numerical value, and determining a first application partition number corresponding to the third numerical value in the application partition.
9. The apparatus of claim 6, wherein the processing module is further configured to:
generating new and old customer information record files based on the first customer information record and the second customer information record;
and according to the file format of the batch transfer transaction service, merging the customer information into a transaction name to be used as a header record, storing the new and old customer information record files into a data area, and generating a batch transfer file.
10. The apparatus of any one of claims 6-9, wherein the processing module is further configured to:
monitoring the processing result of each client information record to screen the processing result as a failed client information record set;
Triggering a re-processing mechanism, receiving a modification operation for each client information record in the client information record set, and then executing re-merging processing for the modified client information records.
11. An electronic device, comprising:
one or more processors;
storage means for storing one or more programs,
when executed by the one or more processors, causes the one or more processors to implement the method of any of claims 1-5.
12. A computer readable medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any of claims 1-5.
13. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-5.
CN202311245376.4A 2023-09-25 2023-09-25 Information merging processing method and device Pending CN117290312A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311245376.4A CN117290312A (en) 2023-09-25 2023-09-25 Information merging processing method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311245376.4A CN117290312A (en) 2023-09-25 2023-09-25 Information merging processing method and device

Publications (1)

Publication Number Publication Date
CN117290312A true CN117290312A (en) 2023-12-26

Family

ID=89258206

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311245376.4A Pending CN117290312A (en) 2023-09-25 2023-09-25 Information merging processing method and device

Country Status (1)

Country Link
CN (1) CN117290312A (en)

Similar Documents

Publication Publication Date Title
CN106886371B (en) Caching data processing method and device
CN111324606B (en) Data slicing method and device
CN110188103A (en) Data account checking method, device, equipment and storage medium
CN110895603B (en) Multi-system account information integration method and device
CN113204598B (en) Data synchronization method, system and storage medium
CN112579695A (en) Data synchronization method and device
CN113760722A (en) Test system and test method
CN115858488A (en) Parallel migration method and device based on data governance and readable medium
CN112765169A (en) Data processing method, device, equipment and storage medium
CN111753141A (en) Data management method and related equipment
CN112100168A (en) Method and device for determining data association relationship
CN112835863A (en) Processing method and processing device of operation log
CN117290312A (en) Information merging processing method and device
US11768741B2 (en) Replicating changes written by a transactional virtual storage access method
CN112948733B (en) Interface maintenance method, device, computing equipment and medium
CN114911575A (en) ClickHouse cluster management method and related equipment
CN111061719B (en) Data collection method, device, equipment and storage medium
CN113468218A (en) Method and device for monitoring and managing database slow SQL
CN113704222A (en) Method and device for processing service request
CN113111066A (en) Automatic online method, device and system for database operation work order and computer equipment
CN111352985A (en) Data service platform, method and storage medium based on computer system
CN111008202A (en) Distributed transaction processing method and framework
CN112463887A (en) Data processing method, device, equipment and storage medium
CN113761075A (en) Method, device, equipment and computer readable medium for switching databases
CN113779017A (en) Method and apparatus for data asset management

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination