JP4603401B2 - Data conversion system and data conversion program - Google Patents

Description

  The present invention relates to a data conversion system and a data conversion program, and more particularly, to a technique for converting a configuration of target data into a necessary format as a pre-process for essential information processing.

In the field of information processing, data processing such as data item rearrangement, replenishment, deletion, and combination is performed on the target data before the original processing such as data processing, collation, and registration, and the subsequent Processing efficiency is improved.
For example, in Patent Document 1 below, data collation processing is automated by executing, for example, unification of customer names and currency or supplementation of missing items as preprocessing for collation processing between payment schedule information and payment information. An automatic verification device that realizes the above is disclosed.
JP 2000-285188 A

FIG. 10 is a block diagram showing a conventional general system configuration for realizing such data conversion processing.
In this case, when the system 62 receives the input data 60 having a predetermined data item via a communication network or the like, the data conversion processing unit 64 reads the conversion rule defined for each data item from the conversion rule storage unit 66, In accordance with this, necessary data is acquired from the database server 68, or data items are combined and deleted, and processed as output data 70 to the subsequent system.

FIG. 11 is a schematic diagram showing the points when performing conversion processing such as replenishment of data items using the conventional data conversion system 62. The input having data items such as brand code, fund code, and securities company code. The data is supplemented with data items such as brand name, fund name, and securities company name, and output data is generated.
In this case, the data conversion processing unit 64 transmits the stock code or securities company code of the input data to the database server 68 as reference source data, obtains the name associated with each code, and then obtains the name corresponding to the output data. Embedded in the item.
As shown in this figure, in generating output data from input data, the data conversion system 62 repeatedly refers to the database server 68 for each data item to be supplemented, and processes one piece of input data. The database has been inquired five times.
In general, information processing takes the most time to wait for a query to the database. Therefore, as the number of data items to be replenished and the number of records of target input data increases, the conversion process takes a huge amount of time. Is required.

Of course, if a dedicated system focused on a specific data format and hardware configuration is used, it is possible to speed up the overall processing by taking various efficiency methods at the source code level. Become.
For example, by sending a predetermined amount of reference source data to the database server and acquiring the data that needs to be replenished at a time, the number of accesses to the database server can be reduced and the overall processing time can be reduced. Is also possible.

However, in the case of a dedicated system, the above-described advantage of speeding up the processing can be obtained, but there is a problem that the operation is not flexible.
In other words, if the system operation is specified at the program level as described above, the source code description should be modified if the data item configuration changes or the reference database changes. It is necessary to regenerate the executable file, and a situation occurs in which the operation cannot be immediately modified according to the request of the site.

  The present invention has been devised in order to solve the above-mentioned conventional problems relating to data conversion processing, and enables high-speed data conversion processing while assuming general-purpose processing with excellent flexibility. It aims to realize technology.

In order to achieve the above object, a data conversion system according to claim 1 is a basic configuration in which an input data storage unit storing a plurality of input data and a record number of input data to be collectively processed are set in advance. a setting storage unit, a conversion rule, the conversion rule storing unit for storing the data item unit of output data defining the content of the processing required to generate the output data in a predetermined format based on the input data, converted After obtaining the number of records to be imported at one time with reference to the basic setting storage unit and the output data storage unit for storing the processed data, the number of records from the input data storage unit is obtained. extracted, and the target data acquisition unit to be stored in the first memory file, reads out the conversion rule from the transformation rule storage unit in the data item basis, the first memory Executes the process defined in the conversion rule based on data relating to all the records of the corresponding data items stored in Airu, a conversion processing unit for storing collectively the results data in the appropriate place in the second memory file A data conversion system including a conversion data registration unit that moves each data to the output data storage unit when the data related to the necessary data items is collected in the second memory file. The import unit, the conversion processing unit, and the conversion data registration unit repeat the above processing for all the records stored in the input data storage unit. Further, as one of the conversion rules , specific data of the input data When a request SQL statement for sequentially acquiring related data from the database based on the value of the item is stored , the conversion processing unit: (1) the request S Editing the QL statement and automatically generating a batch SQL statement with a content requesting batch transmission of result data corresponding to multiple reference source data set in the batch SQL issue table; (2) First extracting the value of for all records relating to the data item of which is stored in a memory file, a process of setting the batch SQL issuance table as reference source data, (3) transmits the above bulk SQL statements to the appropriate database , Processing to request transmission of the corresponding result data, and (4) when each result data sent from the database is compared with each reference source data, and when the reference source data and the result data correspond one-to-one The result data is stored in the corresponding location of the second memory file, and when there are a plurality of result data for the reference source data, only one result data selected according to a predetermined rule is stored in the second memory file. Stored in the corresponding portion of the Le, if the result data for the reference original data does not exist is characterized by executing the a process described in the appropriate place in the second memory file a code indicating the missing data.

According to a second aspect of the present invention, there is provided a data conversion program comprising: an input data storage unit storing a plurality of input data; a basic setting storage unit for presetting the number of records of input data to be collectively processed; the conversion rule that defines the contents of the processing required to generate the output data in a predetermined format based on the input data, conversion rule storage unit for storing the data item unit of output data, the completed data conversion process After obtaining the number of records to be fetched at one time with reference to the output data storage unit for storing and the basic setting storage unit, this number of records are extracted from the input data storage unit , and the first memory target data acquisition unit for storing a file, it reads out the conversion rule from the transformation rule storage unit in the data item basis, rank in the first memory file Has been based on the data relating to all the records of the corresponding data item to execute a process defined in the conversion rule, the result conversion unit which stores the data simultaneously to the corresponding portion of the second memory file, the second A data conversion program for functioning as a conversion data registration unit that moves each data to the output data storage unit when data relating to necessary data items is stored in the memory file, the target data capturing unit, the conversion process The conversion data registration unit repeats the above processing for all records stored in the input data storage unit, and further, based on the value of a specific data item of the input data as one of the conversion rules. If the request SQL statement to sequentially retrieve related data from the database Te is stored, the conversion processing unit, the (1) the request SQL statements Condensed, and the process of automatically generating a batch SQL statement content requesting collective transmission result data corresponding to the reference source data of a plurality number of items that are set collectively SQL issuance table, (2) the first memory file extracting the value of for all records of the corresponding stored data items, the process of setting the batch SQL issuance table as reference source data, and transmits the (3) above bulk SQL statements to the appropriate database, the corresponding (4) Compare each result data sent from the database with each reference source data, and if the reference source data and the result data correspond one-to-one, the result data Is stored in the corresponding location of the second memory file, and when there are a plurality of result data for the reference source data, only one result data selected according to a predetermined rule is stored in the second memory file. Stored to those locations, if the result data for the reference original data does not exist is characterized by executing the a process described in the appropriate place in the second memory file a code indicating the missing data.

In the data conversion system according to claim 1 and the data conversion program according to claim 2, the input data is not subjected to conversion processing in units of cases, but is processed in units of a certain number of cases, and the database is inquired. Since it is executed in units of the same number, the number of inquiries can be greatly reduced, and the overall conversion process can be speeded up.
In addition, the result data sent from the database is not directly adopted as output data, but has a function to match and match the reference source data, so that an incorrect data string is formed. It can be prevented in advance.
Furthermore, since conversion rules relating to each data item are supplied in the form of external data, it is possible to flexibly change the object and operation of conversion processing by changing these descriptions.
Moreover, when querying the database, SQL statements for acquiring multiple data at once are automatically generated, so operators in the field can define conversion rules with the same ease as in conventional sequential processing. It becomes possible to do.

  FIG. 1 is a block diagram showing the overall configuration of a contract management system in which a data conversion system according to the present invention is incorporated. A contract management system 10 arranged in a financial institution such as a bank is composed of a received data conversion system 12 and A data collation system 14 and a transmission data conversion system 16 are provided.

  This contract management system 10 is connected to bank asset management departments, overseas securities firms, overseas institutional investors, domestic securities firms, domestic institutions via communication networks such as the bank network 18, SWIFT network 19, and various data exchange service networks 20. The transaction report data sent from the transaction server 22 such as the investor and the operation instruction data are collated, and the data (confirmed agreement data) that has been confirmed to be fulfilled is confirmed in the bank via the bank network 23. It plays a role of passing to the system 24.

The verification process itself between the transaction report data and the operation instruction data is executed in the data verification system 14, but as a pre-processing, the data sent from each transaction server 22 in the received data conversion system 12 is a standard format. The process of converting to the data format is executed.
That is, the transaction report data and operation instruction data transmitted from each transaction server 22 have their own formats, and cannot be compared with each other as they are, or may not be connected to subsequent processing. Therefore, the received data conversion system 12 is once unified into a general-purpose standard format in advance.
Specifically, the following conversion process is executed.
(1) Convert the order of data items of each received data into a standard sequence.
(2) The data code of each received data is converted into a standard data code (for example, the stock code is converted from an external code to an in-house code).
(3) When a data item that does not exist in each received data is requested in the standard format, the missing data item is supplemented from an external database or the like.

In this way, since each received data is unified in advance in the standard format in the received data conversion system 12, the matching process in the data matching system 14 can be executed very efficiently.
Transaction data (combination of operation instruction data and transaction report data) that has been verified in the data verification system 14 is transmitted to the bank account system 24 as described above, and is used for various processes there. Before that, the transmission data conversion system 16 converts the data from the standard format back to the original format required by the bank account system 24.

The data conversion system according to the present invention can be suitably used as the reception data conversion system 12 and the transmission data conversion system 16 in such a contract management system 10.
As shown in FIG. 2, the data conversion system 30 includes a target data capture unit 32, a conversion processing unit 34, a conversion data registration unit 36, an input data storage unit 38, a basic setting storage unit 40, a conversion A rule storage unit 42, an output data storage unit 44, a first memory file 46, a second memory file 48, and a batch SQL issue table 50 are provided.

The target data capturing unit 32, the conversion processing unit 34, and the conversion data registration unit 36 are realized by the CPU of the computer constituting the system 30 executing necessary processes according to the OS and a dedicated application program.
The input data storage unit 38, the basic setting storage unit 40, the conversion rule storage unit 42, and the output data storage unit 44 are provided in the hard disk of the computer, and the first memory file 46, the second memory file 48 and the batch SQL issue table 50 are set in the internal memory of the computer.

In a predetermined directory in the input data storage unit 38, transaction data and the like transmitted from the external transaction server 22 are stored as input data.
In the basic setting storage unit 40, the number of records of input data to be collectively processed is set in advance.
Further, the conversion rule storage unit 42 defines the contents of conversion processing for input data in units of data items of output data.

Next, a processing procedure in the data conversion system 30 will be described with reference to the flowchart of FIG.
First, the target data capturing unit 32 is activated, and acquires the number of records to be captured at one time with reference to the basic setting storage unit 40 (S10).
For example, when “300 cases” is set as a unit of batch processing in the basic setting storage unit 40, the target data fetching unit 32 extracts 300 records collectively from a specific directory in the input data storage unit 38, The data is stored in the first memory file 46 (S12).
FIG. 4 shows an image in this case, and shows a state in which records of REC 1 to REC 300 having items 1 to n are set in the first memory file 46.

Next, the conversion processing unit 34 is activated, reads the conversion rules stored in the conversion rule storage unit 42 in units of data items, analyzes the contents (S14), and executes necessary processes according to the contents.
Here, for example, when the rule that the corresponding brand name should be acquired from an external database is defined in the conversion rule storage unit 42 using the brand code described in the item 1 of the input data as a key (S16), the conversion The processing unit 34 first edits the conversion rule itself into one that supports batch processing.
In other words, the conversion rule (request SQL statement) stored in the conversion rule storage unit 42 is a content that requires the records to be converted sequentially, so the conversion processing unit 34 edits this and issues a batch SQL. An SQL statement (JOIN-type SQL statement with a batch SQL issue table) for acquiring the issue names corresponding to the issue codes stored in the table 50 at a time from the database server is automatically generated (S18).

Next, the conversion processing unit 34 extracts 300 brand codes from the first memory file 46 and stores them in the batch SQL issue table 50 (S20).
Next, the conversion processing unit 34 transmits the edited SQL sentence to the database server 52 and requests batch transmission of corresponding data (S22).
Receiving this, the database server 52 extracts the brand name corresponding to each brand code stored in the batch SQL issue table 50 in accordance with the edited SQL statement from the brand database, and transmits it to the conversion processing unit 34.
The batch SQL issue method will be described in detail later.

Next, the conversion processing unit 34 executes a matching process on each data transmitted from the database server 52 (S24).
That is, each data transmitted from the database server 52 does not necessarily correspond one-to-one with the reference source data (stock code) stored in the collective SQL issuing table 50, and data loss or duplication occurs. There is a possibility.
For example, even if 300 brand names are sent from the database server 52 to 300 brand codes, multiple brand names are actually returned for the same brand code, In some cases, the corresponding brand name may not be transmitted, and it is not a problem that it is sufficient if the numbers match.
These inconsistencies are caused by deficiencies on the database server 52 side. However, if they are formally linked to the reference source code in this state, an incorrect data string is formed. A mechanism is provided for determining whether or not the acquired data is appropriate, making necessary corrections, and storing it in the second memory file 48 (S26).

FIG. 5 shows an example of this matching processing. As shown in FIG. 5A, when data of result-1 is returned for the reference source record number 1 from the database server 52, the reference source data and Since a one-to-one correspondence with the reference result data is recognized, the conversion processing unit 34 stores the result −1 as it is as the new item 2 of the second memory file 48 as shown in FIG.
On the other hand, for reference source record number 5, since result-2 and result-3 are returned in duplicate, only one result data is adopted according to a predetermined rule. Here, the conversion processing unit 34 stores the result-2 as the new item 2 on the second memory file 48 in accordance with the rule of adopting the top result data.
Further, since the result data is not returned for the reference source records 2 to 4 and it is clear that the corresponding data is missing, the conversion processing unit 34 means that the reference source data 2 to 4 are missing data. “null” is stored in the second memory file 48.

When the processing for 300 items is completed for one item as described above, the conversion processing is repeated again in units of 300 for the next data item (S28, S14 to S26).
Then, at the time when 300 pieces of data related to the necessary data items are collected on the second memory file 48, the conversion data registration unit 36 stores these in the output data storage unit 44 (S30).
Subsequently, the target data fetching unit 32 fetches 300 next target records from the input data storage unit 38, and the necessary conversion process is repeated (S32, S12 to S30).

In the above, an example in which the corresponding data is replenished from the database server 52 based on the data of a certain data item has been described. However, depending on the data item, it can be applied to a predetermined item in the second memory file 48 as it is (simple transfer) In addition, there is a case where it is not necessary to make an inquiry to the database server 52 because the conversion rule is to combine data of different data items (character string combination).
In such a case, the conversion processing unit 34 performs the necessary processing without storing a large number of reference source data in the batch SQL issue table 50, editing the SQL statement, or querying the database server 52. After that, it is sufficient to store the corresponding data at a necessary position on the second memory file 48 (S34).
For example, as shown in FIG. 4, since the “brand code” corresponding to the item 1 of the input data is maintained as the item 1 as it is after the conversion, the conversion processing unit 34 stores these data in the first memory file 46. The item 1 is copied as it is to the new item 1 of the second memory file 48.
Even in this case, since batch processing can be performed in a unit of the number of cases, the processing speed can be increased as compared with the case of processing in units of one record.

Here, a specific example of the batch SQL issue processing method described above will be described.
FIG. 6 shows an example of editing a conversion rule, and shows a state in which a relatively complicated batch SQL statement is generated based on an extremely simple request SQL statement.
As shown in the flowchart of FIG. 7, first, the conversion processing unit 34 checks whether or not ORDER and UNION exist in the request SQL statement (S40). Since the editing contents of the batch SQL statement differ depending on the presence or absence of ORDER and UNION as described later, the existence check is performed in advance.

Next, the conversion processing unit 34 searches for a predetermined keyword from the request SQL sentence (S42), edits the batch SQL sentence according to each keyword, and executes host array editing of the batch SQL issue table (S46).
Below, the edit content concerning each keyword is demonstrated.

First, when the keyword is “SELECT”, “W_PACK_SQL.SQL_RECORD_NO,” is edited immediately ((1) in FIG. 6). Here, “W_PACK_SQL” is a batch SQL issue table, and is the content of adding the record number of the first memory file 46 to the acquisition column at the time of batch SQL issue. However, in the case of a SELECT statement (subquery) within the WHERE condition, the above editing is not performed.
Also, when the keyword is “FROM”, “W_PACK_SQL,” is edited immediately after that to make a JOIN type SQL statement with the batch SQL issue table (FIG. 6 (2)). However, in the case of a SELECT statement (subquery) within the WHERE condition, the above editing is not performed.
If the keyword is “WHERE”, immediately edit “W_PACK_SQL. CUST_NO =: V1 AND W_PACK_SQL.WORK_ID =: V2 AND W_PACK_SQL.JOBLIST_ID =: V3 AND W_PACK_SQL.RULE_NO =: V4 AND (” ((3 Here, V1 to V4 specify the extraction condition key of the batch SQL issue table, but the above editing is not performed in the case of a SELECT statement (subquery) within the WHERE condition.
When the keyword is “; [n];”, the item value corresponding to n is acquired from the first memory file 46 and edited in the host array area of the batch SQL issue table. At this time, “; [n];” (item number) is replaced with the column name of the corresponding batch SQL issue table and edited into a batch SQL statement ((4) in FIG. 6).

Next, the following editing is performed at the end of the edited SQL sentence according to the combination of the presence / absence of “ORDER” and “UNION” checked in advance.
(1) When neither “ORDER” nor “UNION” exists in the request SQL statement Edit “) ORDR BY W_PACK_SQL.SQL_RECORD_NO” at the end of the edited SQL statement (FIG. 6 (5)).
(2) When there is no "ORDER" in the request SQL statement and only "UNION" exists No processing (If there is a UNION, the order of the acquisition columns is guaranteed, so ORDER editing is not necessary).
(3) When only "ORDER" exists in the request SQL statement and "UNION" does not exist Edit ") ORDR BY W_PACK_SQL. RECORD_NO, TBL-A. ITEM-1, ..." at the end of the edited SQL statement. "TBL-A. ITEM-1, ..." means that the specified ORDER condition is expanded as it is (ASC, DESC expansion is also expanded as it is).
(4) When both "ORDER" and "UNION" exist in the request SQL statement Edit ") ORDR BY W_PACK_SQL. RECORD_NO, ITEM-1, ..." at the end of the edited SQL statement. “ITEM-1,...” Means that the specified ORDER condition is expanded as it is (ASC and DESC expansion are also expanded as they are).

When the above editing process is completed for all keywords (S44), the host array of the batch SQL issue table is batch inserted into the batch SQL issue table (S48).
FIG. 8 shows an image in which the variable value of item 2 is set from the first memory file 46 to ITEM01 in the host array area of the batch SQL issue table, and is collectively inserted into the batch SQL issue table 50.
At this time, a comparison table 58 between the item numbers and the column names of the batch SQL issue table 50 is created on the memory.

Finally, a batch SQL statement is issued to the database server 52 by batch FETCH, and the result is stored in the second memory file 48 (S50).
FIG. 9 is a flowchart showing a processing procedure from issuance of the batch SQL statement to acquisition of the result.
When issuing the "OPEN CURSOR statement" in S64, the extraction key for the batch SQL issue table is specified.
A specific example of an OPEN CURSOR statement is shown below.
"EXEC SQL
OPEN CUR01 USING: CUST-NO,: WORK-ID,: JOBLIST-ID,: RULE-NO
END-EXEC "
This means that the extraction condition key of the batch SQL issue table is set and executed (the actual value of V1 to V4 is specified).

In this data conversion system 30, the input data is not subjected to conversion processing on a case-by-case basis, but queries to the database server 52 are executed in a certain number of cases, so the number of inquiries is greatly increased. It is possible to reduce the speed of the conversion process.
For example, in the case of the data conversion example shown in FIG. 11, according to the conventional sequential conversion method, a total of 1,500 times (300 times × 5 times) is required for the database server when converting 300 records. In the case of the batch conversion method according to the present invention, the calculation is only required 5 times.
The number of records to be subjected to the batch conversion process is determined in consideration of the memory capacity installed in the computer and the data amount of the target input data, and is set in the basic setting storage unit 40 in advance.

In the case of this data conversion system 30, since the number of records to be converted at a time as described above and the conversion rules related to each data item are given to the system in the form of external data, by changing these descriptions It is possible to easily change the processing target and operation.
In addition, when querying the database, SQL statements for acquiring multiple data at once are automatically generated, so operators in the field can use conversion rules (requests as easily as in conventional sequential processing). SQL statement) can be defined.
Furthermore, since a function for automatically aligning a plurality of pieces of data transmitted from the database server 52 is provided, it is possible to prevent an erroneous data string from being formed.

In the above, the example in which the data conversion system 30 is applied to the reception data conversion system 12 of the contract management system 10 has been described. However, the data conversion system 30 can also be applied to the transmission data conversion system 16.
In this case, the input data to be converted is the collated standard data output from the data collation system 14, and the converted output data is data in a unique format required by the bank account system 24.
In order to realize this, a rule for converting standard data may be registered in the conversion rule storage unit 42 for each data item of output data.
The data conversion system 30 can also be applied to systems other than the contract management system 10 by rewriting the conversion rules in the conversion rule storage unit 42 and the setting values in the basic setting storage unit 40.

In the above, the example in which the target data capturing unit 32 captures input data to be converted from the input data storage unit 38 provided in the local disk of the system 30 has been described, but the present invention is limited to this. It is not a thing.
For example, the target data fetching unit 32 uses a function of monitoring and detecting a file generated in a predetermined directory of the remote machine by FTP, extracts a predetermined number of records from the disk of another computer, and extracts the first record It can also be stored in the memory file 46.
Alternatively, the target data capturing unit 32 uses a function for monitoring and detecting in-queue messages transmitted by the MQ (message queuing system), extracts a predetermined number of records from the queue area, and uses the first memory. It can also be stored in the file 46.

It is a mimetic diagram showing the whole contract management system composition in which the data conversion system concerning this invention was built. It is a block diagram which shows the function structure of this data conversion system. It is a flowchart which shows the process sequence in this data conversion system. It is a schematic diagram which shows the image of the data storage in a 1st memory file and a 2nd memory file. It is a schematic diagram which shows the specific example of a matching process. It is explanatory drawing which shows the example of automatic editing of a conversion rule. It is a flowchart explaining the detail of batch SQL issue processing. It is a schematic diagram which shows the image at the time of batch-inserting reference source data into a batch SQL issue table. It is a flowchart which shows the process sequence from issue of a batch SQL sentence to acquisition of a result. It is a block diagram which shows the structure of the conventional data conversion processing system. It is a conceptual diagram which shows the point of the conversion process in the conventional data conversion system.

Explanation of symbols

10 Contract management system
12 Received data conversion system
14 Data verification system
16 Transmission data conversion system
18 Bank network
19 SWIFT Net
20 Various data exchange service nets
22 Trading server
23 Bank network
24 Bank account system
30 Data conversion system
32 Target data import section
34 Conversion processor
36 Conversion data register
38 Input data storage
40 Basic setting memory
42 Conversion rule storage
44 Output data storage
46 First memory file
48 Second memory file
50 Bulk SQL issue table
52 Database server
58 Comparison table between item number and column name of batch SQL issue table

Claims (2)

An input data storage unit storing a plurality of input data ;
A basic setting storage unit for presetting the number of records of input data to be collectively processed;
The conversion rule that defines the contents of the processing required to generate the output data in a predetermined format based on the input data, and the conversion rule storing unit for storing the data item unit of output data,
An output data storage unit for storing the converted data,
The target data fetching unit that acquires the number of records to be fetched at one time with reference to the basic setting storage unit, extracts the records for the number of records from the input data storage unit, and stores it in the first memory file When,
The conversion rule from the transformation rule storage unit reads the data item unit executes a process defined in the conversion rule based on data relating to all the records of the corresponding data items stored in the first memory file, the result A conversion processing unit that collectively stores data in corresponding portions of the second memory file;
A data conversion system comprising a conversion data registration unit that moves each data to the output data storage unit when the data related to the necessary data items is prepared in the second memory file,
The target data capturing unit, the conversion processing unit, and the conversion data registration unit repeat the above processing for all records stored in the input data storage unit,
Furthermore, as one of the conversion rules , when a request SQL statement for sequentially acquiring related data from the database based on the value of a specific data item of the input data is stored , the conversion processing unit,
(1) Edit the request SQL statement and automatically generate a batch SQL statement with a content requesting batch transmission of result data corresponding to multiple reference source data set in the batch SQL issue table;
(2) a process of extracting a value of for all records relating to the corresponding data items stored in the first memory file, is set to batch SQL issuance table as reference source data,
(3) Sending the above batch SQL statement to the corresponding database and requesting transmission of the corresponding result data ;
(4) Each result data transmitted from the database is compared with each reference source data. When the reference source data and the result data correspond one-to-one, the result data is stored in the corresponding location of the second memory file. When there are a plurality of result data for the reference source data, only one result data selected according to a predetermined rule is stored in the corresponding portion of the second memory file, and there is no result data for the reference source data In some cases, a process for describing a code indicating data loss in a corresponding portion of the second memory file ;
The data conversion system characterized by performing . Computer
An input data storage unit storing a plurality of input data ;
A basic setting storage unit that presets the number of records of input data to be batch processed,
Conversion rule storing unit for storing the data item unit of conversion rules, output data defining the content of the processing required to generate the output data in a predetermined format based on the input data,
An output data storage unit for storing the converted data,
The target data fetching unit that acquires the number of records to be fetched at one time with reference to the basic setting storage unit, extracts the records for the number of records from the input data storage unit, and stores it in the first memory file ,
The conversion rule from the transformation rule storage unit reads the data item unit executes a process defined in the conversion rule based on data relating to all the records of the corresponding data items stored in the first memory file, the result A conversion processing unit that collectively stores data in corresponding portions of the second memory file;
A data conversion program that functions as a conversion data registration unit that moves each data to the output data storage unit when the data related to the necessary data items is prepared in the second memory file,
The target data capturing unit, the conversion processing unit, and the conversion data registration unit repeat the above processing for all records stored in the input data storage unit,
Furthermore, as one of the conversion rules , when a request SQL statement for sequentially acquiring related data from the database based on the value of a specific data item of the input data is stored , the conversion processing unit,
(1) Edit the request SQL statement and automatically generate a batch SQL statement with a content requesting batch transmission of result data corresponding to multiple reference source data set in the batch SQL issue table;
(2) a process of extracting a value of for all records relating to the corresponding data items stored in the first memory file, is set to batch SQL issuance table as reference source data,
(3) Sending the above batch SQL statement to the corresponding database and requesting transmission of the corresponding result data ;
(4) Each result data transmitted from the database is compared with each reference source data. When the reference source data and the result data correspond one-to-one, the result data is stored in the corresponding location of the second memory file. When there are a plurality of result data for the reference source data, only one result data selected according to a predetermined rule is stored in the corresponding portion of the second memory file, and there is no result data for the reference source data In some cases, a process for describing a code indicating data loss in a corresponding portion of the second memory file ;
The data conversion program characterized by performing this .

Images