JP2021184166A - Cleansing rule generation support program, cleansing rule generation support method, and cleansing rule generation support system - Google Patents

Abstract

To provide a cleansing rule generation support program, a cleansing rule generation support method, and a cleansing rule generation support system which prevent the occurrence of errors due to the order of procedures of cleansing processing during data cleansing for improving data quality by clarifying, deleting and correcting errors, duplication, description fluctuations, and so on in a database.SOLUTION: In the cleansing rule generation support system, a cleansing rule generation support unit of a cleansing rule generation support device includes a rule editing unit which accepts designation of a plurality of libraries to be subjected to data conversion and designation of the execution order of the plurality of libraries as cleansing rules and refers to condition information registered in a storage unit and, if the execution order does not meet a condition, outputs a warning.SELECTED DRAWING: Figure 11

Description

The present invention relates to a cleansing rule creation support program, a cleansing rule creation support method, and a cleansing rule creation support system.

Conventionally, data cleansing has been known as one of the processes for improving the quality of data by identifying errors, duplications, fluctuations in markings, etc. from a database and deleting or correcting them. Further, it has been conventionally known that in data cleansing, it is necessary to create a cleansing rule according to the data characteristics of the target system.

As a method of creating a cleansing rule, for example, a method of preparing a set of libraries for performing simple data conversion in advance and creating a cleansing rule by combining the libraries for each system to be targeted for data cleansing is known.

JP-A-2010-244539

However, since the library performs simple and mechanical processing, the conventional method requires a high level of knowledge of data manipulation. Further, in the conventional method, when an error occurs due to the execution order of the libraries, the combination of libraries and the execution order are determined by trial and error, which takes a lot of time.

In one aspect, the present invention aims to suppress the occurrence of errors due to the order of processing in data cleansing.

In one embodiment, the designation of a plurality of libraries and the designation of the execution order of the plurality of libraries are accepted as cleansing rules, the condition information registered in the storage unit is referred to, and the order does not satisfy the conditions. To output a warning, let the computer execute the process.
Is.

The purpose is to suppress the occurrence of errors due to the order of processing in data cleansing.

It is a figure which shows an example of the system configuration of the cleansing rule creation support system. It is a figure which shows an example of the hardware composition of the cleansing rule creation support device. It is a figure which shows an example of a library information storage part. It is a figure which shows an example of the form information storage part. It is a figure which shows an example of the information storage part which cannot be arranged in front. It is a figure which shows an example of the information storage part which cannot be rearranged. It is a figure which shows an example of the rule information storage part. It is a figure which shows an example of the functional structure of a cleansing rule creation support part. It is a flowchart which shows the processing of a pre-processing part. It is the first figure which shows the display example of a terminal apparatus. It is a flowchart which shows the process of a rule editing part. It is a second figure which shows the display example of a terminal apparatus. It is a flowchart which shows the process of execution determination part. It is a third figure which shows the display example of a terminal apparatus.

Hereinafter, the present embodiment will be described with reference to the drawings. FIG. 1 is a diagram showing an example of a system configuration of a cleansing rule creation support system.

The cleansing rule creation support system 100 of the present embodiment includes a cleansing rule creation support device 200 and a terminal device 300, each of which is connected via a network N or the like.

The cleansing rule creation support device 200 of the present embodiment supports the creation of cleansing rules according to the data characteristics of the system that is the target of data cleansing. In the following description, data cleansing may be referred to as cleansing processing. The cleansing process indicates that the data conversion process is executed in the order indicated by the cleansing rule according to the cleansing rule.

The user of the terminal device 300 is, for example, a creator who creates a cleansing rule, and a user of a service provided by the cleansing rule creation support system 100 of the present embodiment. The system to be targeted for data cleansing may be specified by, for example, the user of the terminal device 300.

Data cleansing refers to the process of identifying errors, duplications, fluctuations in markings, etc. from the database, deleting or correcting them, and improving the quality of the data.

A cleansing rule is a combination of multiple libraries. In other words, a cleansing rule specifies the execution order of multiple libraries.

The library of this embodiment is a program component that describes a function of converting data. Therefore, the cleansing rule indicates the execution order of the function (process) for converting a plurality of data.

The cleansing rule creation support device 200 of the present embodiment has a library information storage unit 210, a format information storage unit 220, a front-arrangeable information storage unit 230, a rear-arrangement-unable information storage unit 240, a rule information storage unit 250, and a cleansing rule creation support. It has a part 260.

The library information storage unit 210 stores library information related to the library. The format information storage unit 220 stores format information regarding the file format for each library.

Condition information is stored in the front-arrangeable information storage unit 230 and the rear-arrangement-incapable information storage unit 240. The conditional information is information referred to when determining whether or not the library can be executed in the specified execution order. The rule information storage unit 250 stores rule information indicating a cleansing rule.

When creating a cleansing rule, the cleansing rule creation support unit 260 outputs a warning (alert) when it detects an error caused by the execution order of a plurality of specified libraries.

Specifically, the cleansing rule creation support unit 260 has a preprocessing unit 270, a rule editing unit 280, and an execution determination unit 290.

The pre-processing unit 270 receives input of library information from the terminal device 300 and stores it in the library information storage unit 210. Further, the pre-processing unit 270 receives input of format information from the terminal device 300 and stores it in the format information storage unit 220. Further, the pre-processing unit 270 receives input of condition information from the terminal device 300 and stores it in the pre-arrangement impossible information storage unit 230 and the rear arrangement impossible information storage unit 240.

The rule editing unit 280 accepts a plurality of libraries to be added to the cleansing rule and a specification of the execution order of the plurality of libraries. Next, the rule editing unit 280 refers to the condition information stored in the pre-arrangement impossible information storage unit 230 and the rear arrangement impossible information storage unit 240, and whether the execution order of the plurality of libraries is the permitted execution order. Judge whether or not. Then, the rule editing unit 280 causes the terminal device 300 to display a warning when the execution order of the plurality of libraries is permitted. When the execution order of the library is permitted, it means that the execution order of the library is logically established.

In the following description, the fact that the execution order of a plurality of libraries is permitted may be expressed as the condition that the execution order of a plurality of libraries satisfies the condition.

That is, the condition of the execution order of the plurality of libraries is that the plurality of libraries are executed in the order other than the order prohibited by the condition information.

In other words, when the execution order of a plurality of libraries satisfies the condition, it means that the libraries are allowed to be executed according to the execution order.

As described above, in the present embodiment, when the cleansing rule is created, if the execution order of the plurality of libraries to be added to the cleansing rule does not satisfy the condition, a warning to that effect is output. Therefore, in the present embodiment, when data cleansing is executed according to the cleansing rule once created, the error caused by the execution order is not found for the first time, and the error caused by the order of the library (processing) occurs. Can be suppressed.

Further, in the present embodiment, since an error caused by the execution order of the library (process) is detected during the creation of the cleansing rule, the library is executed without repeating the creation of the cleansing rule and the execution of the cleansing process. You can decide the order. Therefore, according to the present embodiment, it is possible to reduce the man-hours (time) related to the creation of the cleansing rule.

Further, when the cleansing process is executed, the execution determination unit 290 determines whether or not the file format of the target data is the file format corresponding to the first library in the cleansing rule. In other words, the execution determination unit 290 determines whether or not the data to be cleansed is input in a format in which the first library can be executed.

Then, the execution determination unit 290 causes the terminal device 300 to display a warning when the file format does not correspond to the first library.

Therefore, according to the present embodiment, it is possible to notify an error that occurs when the cleansing process based on the created cleansing rule is executed before the execution of the cleansing process. Therefore, in the present embodiment, it is possible to notify the user of an error without repeating the creation of the cleansing rule and the execution of the cleansing process, and the man-hours (time) related to the creation of the cleansing rule can be reduced.

In the example of FIG. 1, the library information storage unit 210, the format information storage unit 220, the front-arrangeable information storage unit 230, the rear-arrangeable information storage unit 240, and the rule information storage unit 250 have a cleansing rule creation support device 200. It is supposed to have, but it is not limited to this.

A part of each of these storage units may be provided in the cleansing rule creation support device 200, and a part thereof may be provided in an external device of the cleansing rule creation support device 200. Further, all of these storage units may be provided in an external device of the cleansing rule creation support device 200. The external device may be any device that can communicate with the cleansing rule creation support device 200 and can refer to information according to the processing of the cleansing rule creation support unit 260.

Further, in the example of FIG. 1, the number of terminal devices 300 included in the cleansing rule creation support system 100 is one, but the number of terminal devices 300 included in the cleansing rule creation support system 100 may be arbitrary.

The cleansing rule creation support device 200 of the present embodiment will be described below. FIG. 2 is a diagram showing an example of the hardware configuration of the cleansing rule creation support device.

The cleansing rule creation support device 200 of the present embodiment is an input device 21, an output device 22, a drive device 23, an auxiliary storage device 24, a memory device 25, an arithmetic processing unit 26, and an interface device, which are interconnected by a bus B, respectively. It is a computer including 27.

The input device 21 is a device for inputting various types of information, and is realized by, for example, a keyboard, a pointing device, or the like. The output device 22 is for outputting various kinds of information, and is realized by, for example, a display or the like. The interface device 27 includes a LAN card and the like, and is used for connecting to a network.

The cleansing rule creation support program that realizes the cleansing rule creation support unit 260 is at least a part of various programs that control the cleansing rule creation support device 200. The cleansing rule creation support program is provided, for example, by distributing the storage medium 28, downloading from the network, or the like. The storage medium 28 on which the cleansing rule creation support program is recorded is such as a storage medium such as a CD-ROM, a flexible disk, a magneto-optical disk, etc., which records information optically, electrically, or magnetically, a ROM, a flash memory, or the like. Various types of storage media such as semiconductor memories that electrically record information can be used.

Further, when the storage medium 28 recording the cleansing rule creation support program is set in the drive device 23, the cleansing rule creation support program is installed in the auxiliary storage device 24 from the storage medium 28 via the drive device 23. The cleansing rule creation support program downloaded from the network is installed in the auxiliary storage device 24 via the interface device 27.

The auxiliary storage device 24 realizes a library information storage unit 210, a format information storage unit 220, a front-arrangeable non-arrangeable information storage unit 230, a rear-arrangeable non-arrangeable information storage unit 240, a rule information storage unit 250, and the like included in the cleansing rule creation support device 200. It stores the cleansing rule creation support program installed in the cleansing rule creation support device 200, and also stores various necessary files, data, and the like by the cleansing rule creation support device 200. The memory device 25 reads and stores the cleansing rule creation support program from the auxiliary storage device 24 when the cleansing rule creation support device 200 is started. Then, the arithmetic processing unit 26 realizes various processes as described later according to the cleansing rule creation support program stored in the memory device 25.

Next, each storage unit included in the cleansing rule creation support device 200 will be described with reference to FIGS. 3 to 7. Each storage unit of the cleansing rule creation support device 200 is realized by, for example, an auxiliary storage device 24, a memory device 25, or the like.

FIG. 3 is a diagram showing an example of a library information storage unit. The library information storage unit 210 of the present embodiment stores library information related to the library. The library information is stored in the library information storage unit 210 by the processing of the preprocessing unit 270.

The library in this embodiment is a program component that describes a process for performing data conversion. In other words, the library of the present embodiment is one unit including a plurality of instructions describing a process for performing data conversion.

Specifically, the library includes, for example, a program component that describes a process of "replacing A in the data with B", a program component that describes a process of "deleting C in the data", and the like. Further, the library includes, for example, a program component that describes a process of complementing missing data, a program component that describes a process of converting a value calculated as an average value into an instantaneous value, and the like.

The library information of the present embodiment has a library ID, a library name, a conversion method, an input, and an output as information items, and the item "library ID" is associated with other items. In the following description, the information including the value of the item "library ID" and the value of other items stored in the library information storage unit 210 is referred to as library information.

The value of the item "library" is identification information for identifying the library. The value of the item "library name" indicates the name of the library. The value of the item "conversion method" indicates the method of data conversion performed by the library specified by the library ID.

The value of the item "input" indicates how to convert the data when the data is input to the library, and the value of the item "output" indicates how to convert the data when the data is output from the library. ..

In the example of FIG. 3, for example, the library specified by the library ID "L04" has the name "○○○○ conversion library", the conversion method is "□□□ conversion", and the data to be converted is The conversion method performed when input is "△△△△", and the conversion method when outputting the data as the converted data is "◇◇◇".

It is assumed that the library information storage unit 210 stores library information related to the library, but the library information storage unit 210 stores the library (which describes the function of converting data) in association with the library information. It may have been done.

FIG. 4 is a diagram showing an example of a format information storage unit. The format information storage unit 220 stores format information regarding the file format for each library. The format information is stored in the format information storage unit 220 by the processing of the preprocessing unit 270.

The format information of the present embodiment includes a format check ID, a library ID, a file format, a check location, and a check value as information items, and the item "library ID" is associated with other items. In the following description, the information including the value of the item "library ID" and the value of other items stored in the format information storage unit 220 is referred to as format information.

The value of the item "format check ID" indicates the identification information that identifies the format information. The value of the item "file format" indicates the format of the data file that can be input to the library specified by the library ID. In other words, the value of the item "file ID" indicates the file format corresponding to the library specified by the library ID.

The value of the item "check location" indicates the position to be checked when the cleansing process is executed in the data input as the data to be converted. The value of the item "check value" indicates the value of the data to be stored at the position indicated by the item "check location".

In FIG. 4, for example, the format information specified by the format check ID “K6” is stored in the check position “AAA” where the library specified by the library ID “L04” has a file format of csv format. Indicates that the value corresponds to the data whose value is "○○".

In other words, the format information specified by the format check ID "K6" is in csv format, and the data input to the library specified by the library ID "L04" is stored in the checked position "AAA". Indicates that the value should be "○○".

FIG. 5 is a diagram showing an example of a front-arrangeable information storage unit. A part of the condition information is stored in the pre-arrangeable information storage unit 230 of the present embodiment. The condition information of the present embodiment is information indicating the execution order of the library that does not logically hold. In other words, the conditional information indicates the execution order in which execution is prohibited.

The pre-arrangeable information storage unit 230 stores information indicating a combination of a certain library and a library prohibited to be arranged in front of this library (pre-arrangeable library). Placing the library in front of a certain library means specifying the library to be executed immediately before the execution of the certain library.

In the following description, the information stored in the pre-arrangement impossible information storage unit 230 may be referred to as pre-arrangement impossible information.

The pre-arrangeable information storage unit 230 of the present embodiment has a reference library ID and a pre-arrangeable library ID as information items, and these items are associated with each other.

The value of the item "reference library ID" indicates the library ID that identifies the reference library. The value of the item "previous impossible library ID" indicates the library ID of the library arranged immediately before the reference library.

In FIG. 5, for example, it can be seen that it is prohibited to place the library specified by the library ID “L02” immediately before the library specified by the library ID “L01”.

In other words, it is not logical to execute the library specified by the library "L02" immediately before the execution of the library specified by the library ID "L01". That is, the library specified by the library "L02" cannot be placed immediately before the library specified by the library ID "L01".

FIG. 6 is a diagram showing an example of an information storage unit that cannot be rearranged. A part of the condition information is stored in the information storage unit 240 that cannot be rearranged after the present embodiment.

The post-displaceable information storage unit 240 stores information indicating a combination of a certain library and a library (post-displaceable library) for which placement is prohibited after this library. By placing it in a library after a certain library, it means specifying a library to be executed subsequently after executing a certain library.

In the following description, the information stored in the post-arrangement impossible information storage unit 240 may be referred to as post-arrangement impossible information.

The post-arrangeable information storage unit 240 of the present embodiment has a reference library ID and a post-unabletable library ID as information items, and these items are associated with each other.

The value of the item "post-impossible library ID" indicates the library ID of the library placed immediately after the reference library.

In FIG. 6, for example, it can be seen that it is prohibited to place the library specified by the library ID “L03” immediately after the library specified by the library ID “L01”.

In other words, it is not logical to execute the library specified by the library "L03" immediately after the execution of the library specified by the library ID "L01". That is, the library specified by the library "L03" cannot be placed immediately after the library specified by the library ID "L01".

As described above, the condition information of the present embodiment includes the information that cannot be arranged before and the information that cannot be arranged after.

FIG. 7 is a diagram showing an example of the rule information storage unit. The rule information storage unit 250 of the present embodiment stores rule information indicating a cleansing rule.

The rule information storage unit 250 includes a cleansing rule ID, a cleansing rule name, a library order, and a library ID as information items, and the item "cleansing rule ID" is associated with other items.

In the following description, in the rule information storage unit 250, information including the value of the item "cleansing rule ID" and the value of other items may be referred to as rule information.

The value of the item "cleansing rule ID" indicates identification information for identifying the cleansing rule. The value of the item "cleansing rule name" indicates the name of the cleansing rule. The value of the item "library order" indicates the execution order of the library indicated by the corresponding library ID in the cleansing rule specified by the cleansing rule ID.

In FIG. 7, for example, in a cleansing rule in which the cleansing rule ID is “R01” and the cleansing rule name is “◇◇◇ rule”, the execution order of the library specified by the library ID “L01” is No. 1. be. Further, in the cleansing rule in which the cleansing rule ID is "R01" and the cleansing rule name is "◇◇◇ rule", the execution order of the library specified by the library ID "L06" is No. 2.

As described above, in the example of FIG. 7, in the cleansing rule in which the cleansing rule ID is "R01" and the cleansing rule name is "◇◇◇ rule", the library and the library ID "L01" specified by the library ID "L01" are used. It can be seen that the library specified by "L06" and the library specified by the library ID "L03" are executed in this order.

Next, with reference to FIG. 8, the function of the cleansing rule creation support unit 260 of the cleansing rule creation support device 200 of the present embodiment will be described. Each part shown in FIG. 8 is realized by reading and executing the cleansing rule creation support program stored in the auxiliary storage device 25 or the like by the arithmetic processing unit 26.

FIG. 8 is a diagram showing an example of the functional configuration of the cleansing rule creation support unit. The cleansing rule creation support unit 260 of the present embodiment has a preprocessing unit 270, a rule editing unit 280, and an execution determination unit 290.

When the pre-processing unit 270 receives the input of the library information, the format information, the pre-arrangement impossible information, the post-arrangement impossible information, and the rule information, the pre-processing unit 270 stores the input in each corresponding storage unit.

The rule editing unit 280 has an input receiving unit 281, a library extraction unit 282, a condition reference unit 283, an arrangement possibility determination unit 284, an alert output unit 285, and a library addition unit 286.

The input receiving unit 281 receives various inputs to the cleansing rule creation support device 200. Specifically, the input receiving unit 281 receives from the terminal device 300 the input of the library to be added to the cleansing rule, the execution order of the library, the execution instruction of the cleansing process, and the like.

The library extraction unit 282 extracts the selected library from the library information storage unit 210.

The pre-arrangement impossible information and the rear arrangement impossible information stored in the condition reference unit 283, the front arrangement impossible information storage unit 230, and the rear arrangement impossible information storage unit 240 are referred to. The arrangement possibility determination unit 284 determines whether or not the added libraries can be arranged in the designated execution order.

The alert output unit 285 outputs an alert (warning) when it is determined by the arrangement availability determination unit 284 that it is impossible to arrange the added libraries in the specified execution order. That is, the alert output unit 285 is an example of a warning output unit that outputs a warning.

The library addition unit 286 adds the library accepted by the input reception unit 281 to the cleansing rule.

The execution determination unit 290 of the present embodiment includes a selection reception unit 291, a library identification unit 292, a format reference unit 293, a format determination unit 294, an alert output unit 295, and an execution unit 296.

The selection reception unit 291 accepts the selection of the cleansing rule corresponding to the cleansing process to be executed. The library specifying unit 292 identifies the first library included in the selected cleansing rule.

The format reference unit 293 refers to the format information of the library specified by the library identification unit 292 in the format information storage unit 220. The format determination unit 294 determines whether or not the file format of the data to be cleansed corresponds to the file format indicated by the referenced format information.

The alert output unit 295 outputs an alert when the format determination unit 294 determines that the file format of the data to be cleansed does not correspond to the file format indicated by the referenced format information.

When the format determination unit 294 determines that the file format of the data to be cleansed corresponds to the file format indicated by the referenced format information, the execution unit 296 determines that the target data is in accordance with the selected cleansing rule. And execute the cleansing process.

Next, with reference to FIG. 9, the processing of the preprocessing unit 270 of the present embodiment will be described. FIG. 9 is a flowchart showing the processing of the preprocessing unit.

In the cleansing rule creation support unit 260, the preprocessing unit 270 determines whether or not the input of the library information has been accepted (step S901). If the input of the library information is not accepted in step S901, the preprocessing unit 270 waits until the input is accepted.

Upon receiving the input of the library information in step S901, the preprocessing unit 270 stores the input library information in the library information storage unit 210 (step S902).

Subsequently, the preprocessing unit 270 determines whether or not the input of the format information has been accepted (step S903). If the input of the format information is not accepted in step S903, the preprocessing unit 270 waits until the input is accepted.

Upon receiving the input of the format information in step S903, the preprocessing unit 270 stores the input format information in the format information storage unit 220 (step S904).

Subsequently, the preprocessing unit 270 determines whether or not the input of the condition information has been accepted (step S905). Specifically, the pre-processing unit 270 determines whether or not the input of the pre-arrangement impossible information and the post-arrangement impossible information is accepted.

When the input of the pre-arrangement impossible information and the post-arrangement impossible information is not accepted in step S905, the preprocessing unit 270 waits until the input is accepted.

When the input of the pre-arrangement impossible information and the post-arrangement impossible information is received in step S905, the pre-arrangement impossible information and the post-arrangement impossible information are stored in the pre-arrangement impossible information and the post-arrangement impossible information by the pre-arrangement impossible information storage unit 230 and the rear arrangement unit 270. It is stored in each of the impossible information storage units 240 (step S906).
Next, with reference to FIG. 10, the input screen of each information stored by the preprocessing unit 270 will be described. FIG. 10 is a first diagram showing a display example of the terminal device. The screen 101 shown in FIG. 10A is a diagram showing an example of a library information input screen. The screen 102 shown in FIG. 10B is a diagram showing an example of a format information input screen. The screen 103 shown in FIG. 10C is a diagram showing an example of a condition information input screen.

In the present embodiment, each of the screens 101, 102, and 103 may be displayed on the terminal device 300, a display device included in the cleansing rule creation support device 200, a display device connected to the cleansing rule creation support device 200, or the like. It may be displayed in.

The screen 101 has a display field 101a. In the display field 101a, the input field 101b and the operation buttons 101c, 101d, 101e are displayed.

The input field 101b is for inputting the values of the items "library name", "conversion method", "input", and "output" included in the library information. The value of the item "library ID" included in the library information may be automatically assigned when the library information is registered.

The operation button 101c is an operation button for transitioning the screen 101 to the screen 102, which is an input screen for format information. The operation button 101d is an operation button for transitioning the screen 101 to the screen 103, which is a condition input screen.

The operation button 101e is an operation button for storing the information input in the input field 101b in the library information storage unit 210 as library information.

For example, on the screen 101, when the operation button 101e is operated after the input to the input field 101b is completed, the preprocessing unit 270 generates a library ID, and the library ID is added to the information input to the input field 101b. Is added and used as library information. Then, the preprocessing unit 270 stores the library information in the library information storage unit 210.

On the screen 101, "○○○○ conversion library" is input as the library name, and "□□□ conversion" is input as the conversion method. Further, on the screen 101, "△△△△" is input as a conversion method to be performed when the data to be converted is input, and "◇◇◇" is input as a conversion method to output the data.

When the operation button 101e is operated in this state, the preprocessing unit 270 assigns the library ID "L04" to the input information, and stores the library information about the library specified by the library ID "L04" in the library information. Store in unit 210.

Further, when the operation button 101c is operated, the preprocessing unit 270 shifts the screen 101 displayed on the terminal device 300 to the screen 102 shown in FIG. 10 (B).

The screen 102 is an example of an input screen for inputting the format information corresponding to the library in which the library information is input on the screen 101.

The screen 102 has a display field 102a. In the display field 102a, the input fields 102b and 102c and the operation button 102d are displayed.

In the input field 102b, the value of the item "file format" included in the format information is input. In the input field 102c, the values of the items "check place" and "check value" included in the format information are input.

The operation button 102d is an operation button for storing the information input in the input fields 102b and 102c in the format information storage unit 220 as format information.

Specifically, when the operation button 102d is operated, the preprocessing unit 270 assigns a format check ID to the information input in the input fields 102b and 102c, and format information as format information associated with the library ID. It is stored in the storage unit 220.

Further, when the operation button 101d is operated on the screen 101, the preprocessing unit 270 shifts the display of the terminal device 300 from the screen 101 to the screen 103.

The screen 103 is an example of an input screen for inputting the condition information corresponding to the library in which the library information is input on the screen 101.

The screen 103 has a display field 103a. The input field 103b and the operation button 103c are displayed in the display field 103a.

In the input field 103b, the name of the library that cannot be placed immediately before the library in which the library information is input on the screen 101 and the name of the library that cannot be placed immediately after the library information are input. A library ID may be input in the input field 103b instead of the library name.

The operation button 103c is an operation button for storing condition information in each of the front-arrangeable information storage unit 230 and the rear-arrangement-incapable information storage unit 240.

On the screen 103, in the input field 103b, the existing conversion X library is input as a library in which the ○○○○ conversion library cannot be placed immediately before, and the existing conversion Z library is entered as a library that cannot be placed immediately after. Has been entered.

Further, in the input field 103b, the existing conversion Y library is input as a library in which the ○○○○ conversion library cannot be placed immediately before or after.

In this state, when the operation button 103c is operated, the preprocessing unit 270 sets the library ID “L04” of the ○○○○ conversion library as the pre-impossible library ID, and uses the library ID “L01” of the existing conversion X library as a reference. It is used as a library ID (see FIG. 3).

Then, the pre-processing unit 270 stores the pre-arrangement impossible information in which the reference library ID and the pre-arrangement impossible library ID are associated with each other in the pre-arrangement impossible information storage unit 230.

Further, the preprocessing unit 270 uses the library ID "L04" of the XX conversion library as the post-impossible library ID and the library ID "L03" of the existing conversion Z library as the reference library ID (see FIG. 3).

Then, the pre-processing unit 270 stores the post-displaceable information in which the reference library ID and the post-displaceable library ID are associated with each other in the post-placeable information storage unit 240.

Further, the pre-processing unit 270 pre-arranges the pre-arrangeable information using the library ID “L04” of the ○○○○ conversion library as the pre-arrangeable library ID and the library ID “L02” of the existing conversion Y library as the reference library ID. Impossible Stored in the information storage unit 230. Further, the pre-processing unit 270 post-arranges the post-displaceable information using the library ID "L04" of the conversion library as the reference library ID and the library ID "L02" of the existing conversion Y library as the reference library ID. Impossible Stored in the information storage unit 240.

In the present embodiment, information is stored in each storage unit of the cleansing rule creation support device 200 by such processing of the preprocessing unit 270.

In the example of FIG. 10, the library information, the format information, and the condition information are input on separate screens, but the present invention is not limited to this. The preprocessing unit 270 may display an input screen including each input field of library information, format information, and condition information on one screen on the terminal device 300.

In the present embodiment, by preparing in advance the library information, the format information, the condition information including the pre-arrangement impossible information and the post-arrangement impossible information, this execution is performed when the execution order of the library is specified. It is possible to detect whether or not the order is a prohibited order.

Next, the process of the rule editing unit 280 will be described with reference to FIG. FIG. 11 is a flowchart showing the processing of the rule editing unit.

The rule editing unit 280 of the present embodiment determines whether or not the input reception unit 281 has accepted the selection of the library to be added to the cleansing rule (step S1101).

At this time, when the rule editing unit 280 causes the terminal device 300 to display the library list screen and accepts the operation of selecting the library from the list in the terminal device 300, it may be assumed that the library selection is accepted.

If the selection of the library is not accepted in step S1101, the rule editing unit 280 waits until the selection of the library is accepted.

When the selection of the library is accepted in step S1101, the rule editing unit 280 refers to the library information storage unit 210 by the library extraction unit 282 and extracts the selected library as a "post-library" (step S1102). The "post-library" of the present embodiment means a library arranged after a certain library. In other words, "post-library" refers to a library that runs immediately after running a library.

In the present embodiment, after accepting the selection of the library, it is determined whether or not another library already added to the cleansing rule exists, and if the other library exists, the selected library is referred to as ". It may be extracted as a "post-library".

Then, if there is no other library already added to the cleansing rule, the rule editing unit 280 may wait until the selection of the library to be added next is accepted.

That is, the rule editing unit 280 may proceed to the processing after step S1102 when a plurality of libraries included in the cleansing rule are selected.

In step S1102, the library extraction unit 282 extracts library information including the library ID of the selected library from the library information storage unit 210, for example.

Subsequently, the rule editing unit 280 extracts the last library being edited as the "previous library" by the library extraction unit 282 (step S1103). The "pre-library" of the present embodiment means a library placed (executed) in front of a certain library. In other words, a "pre-library" refers to a library that runs before a meal that runs a library.

In step S1103, the library extraction unit 282 identifies the last selected library among the libraries already selected as the library included in the cleansing rule, and extracts the library ID of the specified library from the library information storage unit 210. do.

Subsequently, the rule editing unit 280 refers to the pre-arrangement impossible information storage unit 230 by the condition reference unit 283 (step S1104).

Subsequently, in the rule editing unit 280, the pre-arrangement impossible information in which the reference library is the "rear library" and the pre-arrangement library is the "pre-arrangement library" in the pre-arrangement impossible information storage unit 230 by the arrangement possibility determination unit 284 is obtained. It is determined whether or not it exists (step S1105).

Specifically, the placement possibility determination unit 284 uses the reference library ID as the library ID of the "rear library" and the front impossible library ID as the library ID of the "previous library". It is determined whether or not it exists in the unit 230.

In step S1105, when the pre-arrangement impossible information corresponding to the pre-arrangement impossible information storage unit 230 exists, the rule editing unit 280 pushes the alert by the alert output unit 285 (step S1106) and ends the process.

In step S1105, when the corresponding unplaceable information exists, the library whose execution order is the last in the cleansing rule being edited is a library that cannot be placed before the library selected in step S1101. Is shown. In other words, the presence of the corresponding pre-arrangeable information indicates that the library selected in step S1101 cannot be added to the cleansing rule being edited.

In step S1105, when the corresponding pre-arrangement impossible information does not exist in the pre-arrangement impossible information storage unit 230, the rule editing unit 280 refers to the rear-arrangement impossible information storage unit 240 by the condition reference unit 283 (step S1107).

Subsequently, in the rule editing unit 280, the rear-arrangement impossible information is stored in the rear-arrangement impossible information storage unit 240 by the arrangement possibility determination unit 284. It is determined whether or not it exists (step S1108).

Specifically, the placement enablement / disapproval determination unit 284 stores the post-placement impossible information in which the reference library ID is the library ID of the “pre-library” and the rear-impossible library ID is the library ID of the “rear library”. It is determined whether or not it exists in the unit 240.

In step S1108, when the post-arrangement impossible information corresponding to the post-arrangement impossible information storage unit 240 exists, the rule editing unit 280 pushes the alert by the alert output unit 285 (step S1109) and ends the process.

In step S1108, when the corresponding post-displaceable information exists, it means that the library selected in step S1101 is a library whose execution order cannot be placed after the last library in the cleansing rule being edited. show. In other words, the case where the corresponding post-displaceable information exists indicates that the library selected in step S1101 cannot be added to the cleansing rule being edited.

In step S1108, when there is no post-arrangement impossible information corresponding to the post-arrangement impossible information storage unit 240, the rule editing unit 280 sets the selected library as the last library to be executed in the cleansing rule by the library addition unit 286. Add (step S1110) and end the process.

The cleansing rule creation support unit 260 of the present embodiment executes the process of FIG. 11 every time a library is added after the editing of the cleansing rule is started.

Further, in the present embodiment, when an operation indicating the completion of editing of the cleansing rule is received, the rule information indicating the plurality of libraries included in the cleansing rule and the execution order of the plurality of libraries is stored in the rule information storage unit 250. do.

In the present embodiment, by retaining the rule information indicating the created cleansing rule in this way, it is not necessary to create the cleansing rule every time the cleansing process is executed, and the cleansing rule created in advance can be selected. , You can select the cleansing rule to apply.

In this embodiment, the content of the alert displayed in step S1106 may be different from the content of the alert displayed in step S1107.

Specifically, the alert displayed in step S1106 may include a message indicating that the last library being edited is an unpredictable library that cannot be placed before the selected library. good.

Further, the alert displayed in step S1109 may include a message indicating that the selected library is a post-impossible library that cannot be placed after the last library being edited.

Hereinafter, the processing of the rule editing unit 280 will be further described with reference to FIG. FIG. 12 is a second diagram showing a display example of the terminal device.

The screen 121 shown in FIG. 12 is an example of a screen for editing a cleansing rule displayed on the terminal device 300.

The screen 121 has a display field 122. The display field 122 has an input field 123, display fields 124 and 125, and an operation button 126. In the input field 123, the name of the cleansing rule to be edited is input.

The display field 124 displays the library added to the cleansing rule and the execution order of the library. In the display field 125, a list of libraries in which library information is stored is displayed in the library information storage unit 210.

In the example of the screen 121 of FIG. 12, the cleansing rule named "○○○○○○○○ rule" is edited, and in the display field 124, the E conversion library, the ◇◇ conversion library, and the D conversion library are edited. , A conversion library, four libraries are specified so that they are executed in this order.

Further, on the screen 121, an operation of selecting a B conversion library from the list of libraries displayed in the display field 125 and adding the cleansing rule and the like is performed for the cleansing rule being edited.

In this embodiment, it is assumed that a list of library names is displayed in the display field 125, and a library name is selected from the list and dragged and dropped to the display field 124 to be selected as a cleansing rule being edited. The library that was created was supposed to be added.

Further, in the present embodiment, by this drag-and-drop operation, the A conversion library and the B conversion library are specified, and further, the execution order in which the library executed next to the A conversion library is the B conversion library is specified. It will be accepted. In other words, the cleansing rule creation support device 200 has accepted a plurality of libraries and designation of the execution order of the plurality of libraries.

In the example of FIG. 12, the library name “B conversion library” is selected by the mouse cursor 127, and the drag-and-drop operation is performed to the display field 124.

When this drag-and-drop operation is performed, the rule editing unit 280 assumes that the selection and addition of the library is accepted, and starts the process of FIG. 11.

Specifically, when the input reception unit 281 receives an additional operation of the B conversion library, the rule editing unit 280 extracts the library ID "L06" of the B conversion library as the library ID of the rear library. Subsequently, the rule editing unit 280 extracts the library ID "L05" of the A conversion library, which is the library whose execution order is the last, as the library ID of the previous library in the cleansing rule being edited.

Next, the rule editing unit 280 sets the library ID "L06" of the rear library as the reference library ID and the library ID "L05" of the front library as the front-impossible library ID. Then, the rule editing unit 280 determines whether or not the pre-arrangement impossible information storage unit 230 has the pre-arrangement impossible information that matches this combination.

In the present embodiment, the corresponding pre-arrangeable information is present in the pre-arrangeable non-arrangeable information storage unit 230 (see FIG. 5).

Therefore, the rule editing unit 280 displays the alert 128 on the screen 121. In the example of the screen 121, only the alert 128 indicating that the addition of the B conversion library causes an error is displayed, but the present invention is not limited to this.

For example, on the screen 121, a message indicating that the A conversion library cannot be placed in front of the B conversion library may be displayed as an alert. Further, the rule editing unit 280 may display, for example, a message indicating that the A conversion library cannot be placed in front of the B conversion library by pop-up or the like when the alert 128 is selected with the cursor 127.

Further, in the present embodiment, when the operation button 126 is operated on the screen 121, the input field 123 and the display field 124 become blank, and a new cleansing rule can be created.

As described above, according to the present embodiment, it is possible to detect an error caused by the execution order of the library and notify the creator of the cleansing rule while the cleansing rule is being created. Therefore, according to the present embodiment, it is possible to suppress the occurrence of an error due to the order of processing in data cleansing, and it is possible to reduce the man-hours (time) related to the creation of the cleansing rule.

Next, the process of the execution determination unit 290 will be described with reference to FIG. FIG. 13 is a flowchart showing the processing of the execution determination unit.

The execution determination unit 290 of the present embodiment determines whether or not the selection reception unit 291 has received the selection of the cleansing rule and the execution instruction of the file format check (step S1301).

If the execution instruction of the cleansing rule selection and the file format check is not accepted in step S1301, the execution determination unit 290 waits.

Upon receiving the execution instruction of the cleansing rule selection and the file format check in step S1301, the execution determination unit 290 specifies the first library of the selected cleansing rule by the library specifying unit 292 (step S1302). Specifically, the library specifying unit 292 refers to the rule information storage unit 250 and specifies a library ID that corresponds to the selected cleansing rule ID and whose library order is “1”.

Subsequently, the execution determination unit 290 specifies the file format of the specified library by the format reference unit 293 (step S1303). Specifically, the format reference unit 293 refers to the format information storage unit 220 and specifies the file format corresponding to the specified library ID.

Subsequently, the execution determination unit 290 determines whether or not the file format of the data to be cleansed matches the file format specified in step S1303 by the format determination unit 294 (step S1304).

The data to be cleansed is the data given as input data to the library at the head of the cleansing rule.

If the file formats do not match in step S1304, the execution determination unit 290 outputs an alert indicating that the file formats do not match by the alert output unit 295 (step S1305), and ends the process. Specifically, the alert output unit 295 may display a message indicating that the file format of the data to be cleansed does not match the library on the terminal device 300 as an alert.

In step S1304, when the file formats match, the execution determination unit 290 outputs (displays) information indicating that the file formats match to the terminal device 300 (step S1306), and whether or not the execution instruction of the cleansing process is accepted. (Step S1307).

If the execution instruction is not received in step S1307, the execution determination unit 290 waits until the execution instruction is received.

Upon receiving the execution instruction in step S1307, the execution determination unit 290 starts the execution of the cleansing process by the execution unit 296 (step S1308), and ends the process.

Next, with reference to FIG. 14, the processing of the execution determination unit 290 will be further described. FIG. 14 is a third diagram showing a display example of the terminal device.

The screen 141 shown in FIG. 14 is an example of the cleansing process execution screen displayed on the terminal device 300.

The screen 141 has a display field 142. The display field 142 has a selection field 143, 144, an operation button 145, a display field 146, and an operation button 147.

The selection field 143 is for selecting a cleansing rule. In the present embodiment, in the selection field 143, the list of cleansing rules stored in the rule information storage unit 250 may be displayed in a pull-down format, and the cleansing rules selected from the list may be displayed in the selection field 143. On screen 141, "○○○○○○○ rule" is selected as a cleansing rule.

The selection field 144 is for selecting the data to be cleansed. On the screen 141, the "ABCDEF.csv" file is selected as the data to be cleansed.

The operation button 145 is for instructing the execution of the file format check. In the display column 146, the determination result by the format determination unit 294 is displayed. The operation button 147 is for instructing the execution of the cleansing process.

The execution determination unit 290 executes the process shown in FIG. 13 when the cleansing rule and the data to be the cleansing process are selected in the selection fields 143 and 144 on the screen 141 and the operation button 145 is operated.

Specifically, the execution determination unit 290 refers to the rule information storage unit 250, and specifies the library ID of the library at the beginning of the "○○○○○○○ rule". In the present embodiment, the library ID of the library whose library order is "1", which is the rule information corresponding to the cleansing rule ID "R02" of the "○○○○○○○ rule", is "L04" ( See FIG. 7).

Subsequently, the execution determination unit 290 specifies the file format corresponding to the specified library ID. In this embodiment, the file format corresponding to the library ID "L04" is "csv" (see FIG. 4).

Therefore, the execution determination unit 290 determines whether or not the file format of the data selected in the selection field 144 is "csv".

In the example of FIG. 14, the data selected in the selection field 144 is “ABCDEF.csv”, and the file format matches the file format corresponding to the library ID “L04”. Therefore, the execution determination unit 290 displays "OK" as the determination result in the display field 146.

When the operation button 147 is operated in this state, the execution determination unit 290 sets the cleansing rule "," ○○○○○○○ rule "for the data" ABCDEF.csv "that is the target of the cleansing process. Therefore, the cleansing process is executed.

If the file format of "ABCDEF.csv" and the file format corresponding to the library ID "L04" do not match, NG is displayed as the determination result in the display field 146. At this time, the display of the operation button 147 may be grayed out, for example, so that it cannot be selected.

In the present embodiment, as described above, before starting the execution of the cleansing process, it is possible to confirm whether or not the cleansing rule and the data to be the target of the cleansing process are consistent. Therefore, according to the present embodiment, it is possible to notify the user of an error before executing the cleansing process.

Therefore, according to the present embodiment, it is possible to suppress the occurrence of work such as reviewing the cleansing rule to identify the cause of the error during or after the execution of the cleansing process, and to reduce the time required for creating the cleansing rule. Can be done.

The disclosed technology may have the form described in the appendix below.
(Appendix 1)
Specifying multiple libraries and specifying the order of execution of the multiple libraries are accepted as cleansing rules.
A cleansing rule creation support program that refers to the condition information registered in the storage unit, outputs a warning when the above order does not meet the conditions, and causes the computer to execute the process.
(Appendix 2)
The library is a program component that describes a function for converting data.
The cleansing rule creation support program according to Appendix 1, wherein information about the plurality of libraries and condition information are stored in a storage unit in advance, and processing is executed by the computer.
(Appendix 3)
The condition information is
Pre-arrangement information indicating a library, a pre-unplaceable library that is prohibited from being executed immediately before executing the library, and
Includes a library, post-displaceable information indicating post-displaceable libraries that are prohibited from being executed immediately after executing the library, and post-displaceable information.
The cleansing rule creation support program according to Appendix 2, which outputs the warning and causes the computer to execute the process when the order includes the order prohibited in the pre-arrangement impossible information or the post-arrangement impossible information.
(Appendix 4)
The above conditions are
The cleansing rule creation support program according to any one of Supplementary note 1 to 3, wherein the plurality of libraries are executed in an order other than the order prohibited by the condition information.
(Appendix 5)
The cleansing rule creation support according to any one of Supplementary note 1 to 4, which stores rule information indicating a cleansing rule in which the order satisfies each of the plurality of libraries and the order. program.
(Appendix 6)
Accepts the selection of data to be cleansed and cleansing rules,
Whether or not the file format corresponding to the library to be executed first matches the file format of the target data in the selected cleansing rule by referring to the format information indicating the file format corresponding to the library. Judging,
The cleansing rule creation support program according to any one of Supplementary note 1 to 5, which outputs a determination result and causes the computer to execute a process.
(Appendix 7)
It is a method of supporting the creation of cleansing rules by a computer, and the computer is
Specifying multiple libraries and specifying the order of execution of the multiple libraries are accepted as cleansing rules.
A cleansing rule creation support method that refers to the condition information registered in the storage unit and outputs a warning when the above order does not satisfy the condition.
(Appendix 8)
A cleansing rule creation support system having a cleansing rule creation support device and a terminal device.
The cleansing rule creation support device is
An input receiving unit that accepts the designation of multiple libraries and the order of execution of the plurality of libraries as cleansing rules.
A cleansing rule creation support system having a warning output unit that refers to condition information registered in the storage unit and outputs a warning when the order does not satisfy the conditions.
The present invention is not limited to the specifically disclosed embodiments, and various modifications and modifications can be made without departing from the scope of claims.

100 Cleansing rule creation support system 200 Cleansing rule creation support device 210 Library information storage unit 220 Format information storage unit 230 Pre-arrangement impossible information storage unit 240 Post-arrangement impossible information storage unit 250 Rule information storage unit 260 Rule information storage unit 260 Pre-processing Part 280 Rule editing part 281 Input reception part 281 Library extraction part 283 Condition reference part 284 Placement availability judgment part 285 Alert output part 286 Library addition part 290 Execution judgment part 291 Selection reception part 292 Library specification part 293 Format reference part 294 Format judgment part 295 Alert output unit 296 Execution unit 300 Terminal device

Claims (7)

Specifying multiple libraries and specifying the order of execution of the multiple libraries are accepted as cleansing rules.
A cleansing rule creation support program that refers to the condition information registered in the storage unit, outputs a warning when the above order does not meet the conditions, and causes the computer to execute the process. The library is a program component that describes a function for converting data.
The cleansing rule creation support program according to claim 1, wherein the information about the plurality of libraries and the condition information are stored in a storage unit in advance, and the computer is made to execute a process. The condition information is
Pre-arrangement information indicating a library, a pre-unplaceable library that is prohibited from being executed immediately before executing the library, and
Includes a library, post-displaceable information indicating post-displaceable libraries that are prohibited from being executed immediately after executing the library, and post-displaceable information.
The cleansing rule creation support program according to claim 2, wherein when the order includes the order prohibited in the pre-arrangement impossible information or the post-arrangement impossible information, the warning is output and the processing is executed by the computer. The cleansing rule creation according to any one of claims 1 to 3, wherein the storage unit stores the rule information indicating the cleansing rule in which the order satisfies each of the plurality of libraries and the order. Support program. Accepts the selection of data to be cleansed and cleansing rules,
Whether or not the file format corresponding to the library to be executed first matches the file format of the target data in the selected cleansing rule by referring to the format information indicating the file format corresponding to the library. Judging,
The cleansing rule creation support program according to any one of claims 1 to 4, which outputs a determination result and causes the computer to execute a process. It is a method of supporting the creation of cleansing rules by a computer, and the computer is
Specifying multiple libraries and specifying the order of execution of the multiple libraries are accepted as cleansing rules.
A cleansing rule creation support method that refers to the condition information registered in the storage unit and outputs a warning when the above order does not satisfy the condition. A cleansing rule creation support system having a cleansing rule creation support device and a terminal device.
The cleansing rule creation support device is
An input receiving unit that accepts the designation of multiple libraries and the order of execution of the plurality of libraries as cleansing rules.
A cleansing rule creation support system having a warning output unit that refers to condition information registered in the storage unit and outputs a warning when the order does not satisfy the conditions.

Images