JP7492451B2 - Data control system and data control method - Google Patents

Description

The present invention relates to a data control system and a data control method, and is suitable for use in a data control system and a data control method that monitors changes in data in a flow-based application.

In recent years, the implementation of applications (flow-based applications) developed by combining componentized software parts (software components) has progressed, and efficient operation methods for such systems have been proposed.

For example, Patent Document 1 discloses "an application operation management system and method that enables flexible control of service levels, such as functionality, performance, or reliability, in response to changes in the business environment and the fulfillment of SLAs."

JP 2006-252189 A

In serverless applications such as flow-based applications, functions are loosely coupled, so data changes can occur as data input to the application is processed by each software component. For example, unexpected data changes can occur due to design errors in software components. If such data changes are overlooked, data integrity and confidentiality can be reduced, and the flow-based application can output erroneous results. The above-mentioned Patent Document 1 describes the operation management of flow-based applications, but does not disclose data changes that occur as data input to the flow-based application passes through each software component, or control associated with data changes.

The present invention has been made in consideration of the above points, and aims to propose a data control system and a data control method that are capable of detecting data changes during data processing in a flow-based application.

In order to solve such problems, the present invention provides a data control system that detects changes in data processed in a flow-based application that is composed of a combination of loosely coupled software components, in whole or in part, and the plurality of loosely coupled software components that compose the flow-based application each include a software component that functions as a data processing unit that performs arbitrary data processing on input data and outputs processed data, and the data control system includes an attribute value calculation unit that calculates attribute values that quantify changes in data due to the data processing using predetermined calculation data, and a control function request unit that requests the attribute value calculation unit to calculate the attribute values for the processed data of the data processing, attaching the calculation data related to the data processing, each time the data processing by each of the data processing units is completed, and the control function request unit receives updated attribute values that are the calculation results of the attribute values for the processed data from the attribute value calculation unit, and then transmits the processed data and the updated attribute values to the next software component of the data processing unit.

In order to solve such problems, the present invention provides a data control method using a data control system that detects changes in data processed in a flow-based application that is composed of a combination of loosely coupled software components, in whole or in part, and the loosely coupled software components that compose the flow-based application each include a software component that functions as a data processing unit that performs arbitrary data processing on input data and outputs processed data, and the data control method includes the following steps: a request step for requesting calculation of an attribute value for the processed data of the data processing by attaching calculation data related to the data processing each time the data processing by each software component that functions as the data processing unit is completed; an attribute value calculation step for calculating, for the processed data, the attribute value that quantifies the data change due to the data processing using the calculation data attached to the request in the request step; and an output step for transmitting, after the calculation in the attribute value calculation step, the processed data and the updated attribute value that is the attribute value for the processed data to the next software component of the data processing unit.

The present invention makes it possible to detect data changes during data processing in a flow-based application.

1 is a diagram illustrating an example of a configuration of a data control system 1 according to a first embodiment of the present invention. 2 is a block diagram showing an example of a hardware configuration of a computer 10 on which the data control system 1 shown in FIG. 1 operates. 1 is a diagram illustrating an example of the configuration of data processed by a data processing unit 112. FIG. FIG. 11 is a diagram illustrating an example of the configuration of attribute value information. 13 is a flowchart illustrating an example of a processing procedure for attribute value recalculation processing. 11A and 11B are diagrams for visually explaining an example of a calculation process of attribute values in the attribute value recalculation process. 1 is a sequence diagram showing an example of an overall processing procedure in the data control system 1. FIG. 1 is a block diagram showing an example of the configuration of a data processing verification system 2 using a data control system 1. 11 is a sequence diagram showing an example of an overall processing procedure in the data processing verification system 2. FIG. FIG. 11 is a diagram illustrating an example of a configuration of a data control system 3 according to a second embodiment. 10 is a sequence diagram showing an example of an overall processing procedure in the data control system 3. FIG. FIG. 4 is a diagram showing an attribute value threshold list 410. FIG. 4 is a diagram showing an evaluation formula list 420. FIG. 4 is a diagram showing a storage destination list 430.

The following describes an embodiment of the present invention in detail with reference to the drawings.

1 is a diagram showing an example of the configuration of a data control system 1 according to a first embodiment of the present invention. The data control system 1 includes a flow-based application 100 that processes input data, and a data control function 150 that is connected to the flow-based application 100 and detects changes in data associated with data processing that is sequentially executed by the flow-based application 100.

The flow-based application 100 is an application that is composed entirely or partially of N software components loosely coupled, and has input and output based on the JSON format, which is a structured data format, for example. The JSON format is an example of a data format for structured data used in explaining this embodiment, and structured data in other data formats may be used. Each of the above software components processes input data, such as processing, based on the purpose of the flow-based application. In the flow-based application 100, these multiple software components are linked together, and data input to the flow-based application 100 is processed in sequence in each processing block 101, 102, 103, as shown in processing 1, processing 2, ... processing N in Figure 1, and the final data processing result is output.

As shown in FIG. 1, the flow-based application 100 includes an information input unit 111, a data processing unit 112, a data storage unit 113, a control function request unit 114, a first information output unit 115, and a second information output unit 116. Note that the control function request unit 114 is a functional unit for connecting a data control function to the flow-based application 100, and is not a functional unit required for data processing, which is an original function of the flow-based application, and therefore can also be said to be a functional unit that constitutes the data control function 150.

The information input unit 111 accepts input to the flow-based application 100 from outside the data control system 1. Specifically, data 121 requested to be processed by the flow-based application 100 and attribute values 122 of the data are input to the information input unit 111. The information input unit 111 inputs the input data 121 and attribute values 122 to the data processing unit 112 of the processing block 101 of processing 1. Note that in processing blocks 102 and 103 from processing 2 onwards, instead of inputting the data 121 and attribute values 122 from the information input unit 111, processed data 125 and updated attribute values 126 are input to the data processing unit 112 from the control function request unit 114 in the immediately preceding processing block.

In this embodiment, the index value that quantifies the change in data that may occur in the process of data processing in the flow-based application 100 is called an "attribute value." As shown in FIG. 4, which will be described later, in this embodiment, an "attribute value" is assigned to each item (data item) that constitutes the data. Therefore, the attribute value 122 corresponding to the data 121 and the updated attribute value 126 corresponding to the processed data 125 have the format of attribute value information that lists the attribute values assigned to each data item.

The data processing unit 112 performs some processing (e.g., data processing) on the input data based on the purpose of the flow-based application 100, and outputs the execution result. As shown in FIG. 1, the flow-based application 100 has N data processing units 112, and each data processing unit 112 is realized by a separate software component. The software component of this data processing unit 112 has a JSON-format input/output interface similar to the entire flow-based application 100, and has a loosely coupled configuration in which each software component has strong independence. In a loosely coupled configuration, there is little room for mutual dependency on other software components other than the interface, so that when a software component is improved or replaced, the impact on other software components is very small.

In addition, in this embodiment, since the data control function 150 is connected to the flow-based application 100, the data output from the data processing unit 112 (processed data 125) is input to the data processing unit 112 or the first information output unit 115 of the next processing block via the control function request unit 114 in the same processing block.

The data storage unit 113 stores data and the like used in processing by the data processing unit 112 in the same processing block. Specifically, for example, the data storage unit 113 holds data (internal data 123) that the data processing unit 112 adds to input data when processing data, and attribute values 124 of data items of the internal data 123. The internal data 123 and the attribute values 124 of the data items will be described later in the explanation of Figures 5 and 6.

When the processing by the data processing unit 112 in the same processing block is completed, the control function request unit 114 executes a request process for recalculating the attribute value to the data control function 150. When executing the request process, the control function request unit 114 transmits calculation data related to the data processing by the data processing unit 112 to the attribute value calculation unit 151 of the data control function 150. After that, when the control function request unit 114 receives the recalculated updated attribute value 126 from the attribute value calculation unit 151, it transmits the processed data 125 and the updated attribute value 126 to the data processing unit 112 or the first information output unit 115 of the next processing block. Details of the calculation data will be described later in the explanation of FIG. 5. The control function request unit 114 is realized by a software component independent of the data processing unit 112.

The first information output unit 115 and the second information output unit 116 output information indicating the final processing results by the flow-based application 100 and the data control function 150, i.e., the processed data 125 and the updated attribute value 126 transmitted from the control function request unit 114 of the processing block 103 (processing N), by a predetermined output means. In FIG. 1, the first information output unit 115 that outputs the processed data 125 and the second information output unit 116 that outputs the updated attribute value 126 are shown separately, but they may be configured as a single information output unit.

The data control function 150 includes an attribute value calculation unit 151 that calculates (recalculates) the attribute value of data in response to changes in the data during data processing by the flow-based application 100. The attribute value calculation unit 151 is realized by a software component.

The attribute value calculation unit 151 calculates (recalculates) the attribute value of the processed data in response to a request for attribute value recalculation from the control function request unit 114 each time the software component of each data processing unit 112 in the flow-based application 100 finishes processing, and transmits the calculated attribute value (updated attribute value 126) to the control function request unit 114 (attribute value recalculation process). The attribute value calculation unit 151 is realized by an independent software component separate from the data processing unit 112 and the control function request unit 114. Details of the attribute value recalculation process will be described later with reference to Figures 5 and 6, etc.

Figure 2 is a block diagram showing an example of the hardware configuration of a computer 10 on which the data control system 1 shown in Figure 1 operates. As shown in Figure 2, the computer 10 is a server configured by a CPU (Central Processing Unit) 11, a memory 12, a storage device 13, a communication unit 14, an input unit 15, and an output unit 16, all connected together via a bus 17.

The CPU 11 is a central processing unit that implements necessary functions by executing programs stored in the memory 12 (or storage device 13). The memory 12 is a main storage device that the CPU 11 uses when executing processes, and is composed of volatile storage elements such as RAM (Random Access Memory). The storage device 13 is an auxiliary storage device for storing input data provided to the CPU 11 and output data output from the CPU 11, and is composed of non-volatile storage elements such as an HDD (Hard Disk Drive) or SSD (Solid State Drive).

The communication unit 14 is an interface used by the computer 10 to communicate with external devices, and is composed of a NIC (Network Interface Card) or the like. The communication unit 14 is connected to a network (e.g., the Internet) and communicates with external devices via the network. The input unit 15 is an interface that accepts input from a user (operator), and is composed of a keyboard, touch panel, card reader, or voice input device, or the like. The output unit 16 is an interface that outputs data to the operator, and is composed of a display, speaker, printer, or the like. The bus 17 is an internal communication path for the computer 10.

In this embodiment, the flow-based application 100 and data control function 150 shown in FIG. 1 can be executed on one or more computers 10 having a hardware configuration as shown in FIG. 2 to realize the processes described below.

Figure 3 is a diagram for explaining an example of the structure of data processed by the data processing unit 112. The data 210 shown in Figure 3 is shown in a tabular format to visually explain the structure of data 121 input to the flow-based application 100, which is processed by the software components of the data processing unit 112 and the results are output. Therefore, the actual data does not necessarily need to be held in a tabular format like that of Figure 3.

In this embodiment, the input/output format is the JSON format, so the data 121 can be described by a combination of a key and a value. Specifically, in the case of FIG. 3, the data 210 has four data items 211 to 214, indicated by the item names "Item 1" to "Item 4", as keys, and a value associated with each of the data items 211 to 214 is set, so that the value of the data composed of a combination of the multiple data items 211 to 214 can be held in each row.

Figure 4 is a diagram illustrating an example of the configuration of attribute value information. The attribute value information 220 shown in Figure 4 is shown in a table format to visually explain the configuration of the attribute value 122 and the updated attribute value 126. Therefore, the actual attribute value information does not necessarily need to be held in a table format like that of Figure 4.

Specifically, in the case of FIG. 4, the attribute value information 220 is composed of an attribute value name 221 and one or more data items 211-214. Here, the item names "Item 1" to "Item 4" of the data items 211-214 in the attribute value information 220 correspond to the item names of the data items 211-214 in the data 210 shown in FIG. 3. In other words, in the attribute value information 220, an attribute value having a certain attribute value name 221 is assigned to each of the data items 211-214 constituting the data 210.

Figure 5 is a flowchart showing an example of the processing procedure for the attribute value recalculation process. Also, Figure 6 is a diagram visually explaining an example of the attribute value calculation process in the attribute value recalculation process.

As described above, the attribute value recalculation process is executed by the attribute value calculation unit 151 each time a software component of the data processing unit 112 finishes processing in each processing block of the flow-based application 100.

The processes executed by each data processing unit 112 in the flow-based application 100 can be broadly categorized into the following three types of processes. However, the data processing unit 112 is not necessarily defined as executing any one of the three types of processes, and may execute other types of processes (e.g., validation, etc.).

The first type of processing is processing in which the data processing unit 112 retrieves new data items from the accessible data storage unit 113 for the input data, and inserts them in a format linked to the values in each data item. The second type of processing is processing in which unnecessary data items are deleted from the input data. The third type of processing is processing in which specific processing is performed on data items that already exist in the data. In the third type of processing, the data itself of the target data item is changed, and therefore the attribute value may also change. Therefore, when the third type of processing is executed by the data processing unit 112, the attribute value calculation unit 151 changes the attribute value corresponding to the data item that is the target of processing.

Below, the procedure for the attribute value recalculation process will be explained according to the flowchart in Figure 5, while also referring to Figure 6 as necessary.

First, after processing by the data processing unit 112 (hereinafter, described as data processing, which is a representative process), the attribute value calculation unit 151 starts the attribute value recalculation process shown in FIG. 5 when it receives an attribute value recalculation request from the control function request unit 114, which includes calculation data for recalculating the attribute value.

Specifically, the calculation data is data in which six types of data are described in JSON format for the input data and output data of the data processing: an item name list of data items constituting the input data (input data item name list), an item name list of data items constituting the output data (output data item name list), an item name list of data items constituting the internal data 123 added to the output data from other than the input data (internal data item name list), attribute value information linked to each data item constituting the input data (input attribute value information), information indicating the attribute value 124 linked to each data item constituting the internal data 123 (changed attribute value information), and information related to the processing content of the data processing (processing content information). The internal data item name list may be included in the calculation data only when the internal data 123 is added in the data processing. Note that the data configuration of the input data, output data, and internal data is based on the configuration of the data 210 in FIG. 3, and the data configuration of each attribute value information is based on the configuration of the attribute value information 220 in FIG. 4.

When the attribute value recalculation process starts, the attribute value calculation unit 151 first checks whether the calculation data received from the control function request unit 114 is properly prepared (step S101). If the data is not properly prepared, the attribute value calculation unit 151 returns an error to the control function request unit 114 that sent the error, notifying the user that the calculation data is not properly prepared, and ends the attribute value recalculation process. On the other hand, if the data is properly prepared, the process proceeds to step S102.

In step S102, the attribute value calculation unit 151 links the input data item name list to the output data item name list. In addition, if an internal data item name list exists, the internal data item name list is also linked to the output data item name list.

Next, in step S103, the attribute value calculation unit 151, based on the result of the linking in step S102, links the attribute values linked to the data item names of the respective source input data or internal data for each data item name in the output data item name list by referencing the input attribute value information and the changed attribute value information.

The process of step S103 is executed when a first type of process (addition of a data item) or a second type of process (deletion of a data item) is performed in data processing. At this time, for data item names that have not been added or deleted, the attribute value calculation unit 151 links the attribute value of the corresponding data item name from the input attribute value information. Furthermore, for added data item names, the attribute value calculation unit 151 links the attribute value of the corresponding data item name from the changed attribute value information. Furthermore, for deleted data item names, the attribute value calculation unit 151 does not register the attribute value of the data item name.

Next, in step S104, the attribute value calculation unit 151 refers to the processing content information, and if it is necessary to update the attribute value associated with the data item name in the output data item name list for which processing up to step S103 has been completed, updates the target attribute value. The processing in step S104 is executed when a third type of processing (data modification processing) is performed in data processing. At this time, the attribute value calculation unit 151 recalculates the attribute value of the data item name for which data modification processing has been performed, in accordance with the attribute value modification policy indicated in the modified attribute value information.

Finally, in step S105, the attribute value calculation unit 151 generates updated attribute values 126, which are attribute values linked to each data item constituting the output data, by describing in JSON format the list of output data item names linked in step S102 and the attribute values of each data item name linked in steps S103 and S104. Then, the attribute value calculation unit 151 returns the generated updated attribute values 126 to the control function request unit 114, which has requested the attribute value recalculation, and ends the attribute value recalculation process.

Now, referring to FIG. 6, the linking and calculation of attribute values in steps S102 to S105 will be specifically described. In FIG. 6, input attribute value information 231 is shown as attribute value information of the input data, and specific examples of linking of attribute values when a first, second, or third type of processing is performed as data processing on this input data are shown in processing patterns (A), (B), and (C), respectively.

Processing pattern (A) is a case where a first type of data processing is performed to insert the data item "item 4" into the input data. In this case, the attribute value calculation unit 151 can link attribute values in the output data by adding changed attribute value information 232 related to the added internal data of "item 4" to the input attribute value information 231 in the processing of steps S102 to S103. Using the result of this attribute value linking, in step S105, the attribute value calculation unit 151 can generate updated attribute values 126 (updated attribute values 234 in FIG. 6) linked to each data item constituting the output data (processed data 125) after data processing.

Processing pattern (B) is a case where a second type of data processing is performed on the input data to delete the data item "item 2". In this case, the attribute value calculation unit 151 can link attribute values in the output data by deleting the entire data item of "item 2" that was deleted in the processing of steps S102 to S103 from the input attribute value information 231. Using the result of this attribute value linking, in step S105, the attribute value calculation unit 151 can generate updated attribute values 126 (updated attribute values 235 in FIG. 6) that are linked to each data item that constitutes the output data (processed data 125) after data processing.

Processing pattern (C) is a case where a third type of data processing is performed on the input data to change the internal data of "item 2". In this case, in the processing of steps S102 and S104, the attribute value calculation unit 151 can update the attribute value in the output data by changing the attribute value of "item 2" in the input attribute value information 231 based on the changed attribute value information 233 related to the change of the internal data in the data item of "item 2". Using the result of this attribute value update, in step S105, the attribute value calculation unit 151 can generate updated attribute values 126 (updated attribute values 236 in FIG. 6) linked to each data item constituting the output data (processed data 125) after data processing.

In this way, the attribute value calculation unit 151 can generate attribute value information (updated attribute value 126) in which the attribute values corresponding to each item name of the output data are updated by performing a predetermined list operation (insertion, deletion, attribute value change, etc.) according to the type of data processing content indicated in the processing content information (for example, first to third types of processing). Note that when the data processing unit 112 executes data processing that combines the above-mentioned first to third types of processing, the attribute value calculation unit 151 can generate updated attribute value 126 corresponding to the processed data 125 by executing an appropriate combination of the processing patterns (A) to (C).

As described above, in this embodiment, each time processing by the software components of the data processing unit 112 is completed, the attribute value calculation unit 151 in the data control function 150 updates the list of attribute values to record changes in data. In addition, by unifying the input and output of the software components of the control function request unit 114 and the attribute value calculation unit 151 that constitute the data control function 150 into structured data (e.g., JSON format) similar to that of the software components of the data processing unit 112, a configuration that is loosely coupled with the data processing unit 112 can be realized. Therefore, in the data control system 1, it is possible to add the data control function 150 without making major changes to an existing flow-based application 100 that includes multiple data processing units 112.

Figure 7 is a sequence diagram showing an example of the overall processing procedure in the data control system 1. Figure 7 shows the overall processing procedure from when data 121 and its attribute value 122 are input to the flow-based application 100, to when processing is executed in the flow-based application 100 and the data control function 150, to when final output is produced. Note that the attribute value recalculation process by the data control function 150 has been described with reference to the flowchart in Figure 5, and detailed description will not be repeated.

As shown in FIG. 7, first, the information input unit 111 of the flow-based application 100 accepts data 121 that is requested to be processed in the flow-based application 100 and attribute values 122 that express the attribute values of that data in JSON format as input from outside, and starts processing of the application (step S201).

Next, the information input unit 111 inputs the data 121 and attribute value 122 input in step S201 to the software component (data processing unit 112) of the processing block 101 (processing 1) (step S202).

Next, the data processing unit 112 for process 1 performs a predetermined data processing (e.g., data processing) on the data 121 input in step S202 (step S203). By performing this data processing, the data processing unit 112 outputs the processed data 125.

Next, the control function request unit 114 of process 1 sends the calculation data described in FIG. 5 to the data control function 150 based on the results of the data processing performed in step S203, and requests recalculation of the attribute values (step S204).

In the data control function 150 that received the request in step S204, the attribute value calculation unit 151 executes the attribute value recalculation process detailed in the flowchart of FIG. 5 to generate updated attribute values 126 that express a list of attribute values corresponding to the processed data 125 in JSON format (step S205). The attribute value calculation unit 151 then transmits the generated updated attribute values 126 to the control function request unit 114 that made the request in step S204 (step S206).

Next, the control function request unit 114 inputs the processed data 125 output from the data processing unit 112 in step S203 and the updated attribute value 126 received from the attribute value calculation unit 151 in step S206 to the software component (data processing unit 112) of the next processing block 102 (process 2) (step S207).

Then, the processing of steps S202 to S207 is repeatedly executed in each processing block in sequence up to the final processing block 103 (processing N) (step S208). However, in the final processing block 103 (processing N), the control function request unit 114, which has received the updated attribute value 126 from the attribute value calculation unit 151 in step S206, inputs the processed data 125 output from the data processing unit 112 in step S203 of processing N to the first information output unit 115, instead of executing the processing of step S207, and inputs the updated attribute value 126 received from the attribute value calculation unit 151 in step S206 of processing N to the second information output unit 116.

Finally, the first information output unit 115 and the second information output unit output, as the results of the flow-based application 100, the final processed data 125 obtained after data processing is performed on the data 121 input in step S201 by the software components of all the data processing units 112 in the flow-based application 100, and the final updated attribute value 126 recalculated by the data control function 150 each time each data processing process is performed (steps S209, S210).

By performing the overall processing as shown in FIG. 7, in the data control system 1, after processing by each software component (data processing unit 112) of the flow-based application 100, attribute values are recalculated by the software component (attribute value calculation unit 151) of the data control function 150, and when processing by all software components is completed, the final processed data 125 and the attribute value corresponding to that data (final updated attribute value 126) can be obtained.

In addition, in the data control system 1, when the control function request unit 114 requests the attribute value calculation unit 151 to recalculate the attribute values, the input/output data for data processing by the data processing unit 112 is not sent, and only information smaller in size than the input/output data, such as an input data item list and an output data item name list, is sent as data for calculation. This makes it possible to reduce the load on the entire system regarding the attribute value recalculation process by the data control function 150, and by connecting the data control function 150, the impact on the design of the flow-based application 100 can be minimized.

Figure 8 is a block diagram showing an example of the configuration of a data processing verification system 2 that uses the data control system 1. The data processing verification system 2 is a system in which a developer 160 of a flow-based application 100 verifies whether the processing content and processing results of the flow-based application 100 are appropriate by using the (final) updated attribute value 126 output from the data control system 1. Note that most of the configuration shown in Figure 1 is omitted in Figure 8.

As shown in FIG. 8, the data processing verification system 2 is configured by connecting the data control system 1 and a developer computer 170 operated by a developer 160 via a network. The developer computer 170 is a computer having a hardware configuration similar to that of the computer 10 illustrated in FIG. 2. The developer computer 170 includes a test data transmission unit 171 that transmits test data for verification to the flow-based application 100, and a test data reception unit 172 that receives the processed test data that has been processed by the flow-based application 100 and the data control function 150.

Figure 9 is a sequence diagram showing an example of the overall processing procedure in the data processing verification system 2. In Figure 9, the processing described in Figure 7 is omitted.

As shown in FIG. 9, when the developer 160 starts the verification process, he operates the developer computer 170 to send test data for verification from the test data sending unit 171 to the information input unit 111 of the flow-based application 100 (step S301). The test data for verification includes data (e.g., data 121) that causes the data processing unit 112 to perform data processing, and an attribute value of the data (e.g., attribute value 122). In addition, data is sent and received between the developer computer 170 and the flow-based application 100 via the communication unit 14 of each computer 10 (see FIG. 2).

Next, in the flow-based application 100, which receives the test data for verification at the information input unit 111, each data processing unit 112 sequentially processes data, starting with the data 121 included in the test data for verification as the first processing target (step S302). Also, as explained in FIG. 7 and other figures, each time the processing by each data processing unit 112 is completed, the data control function 150 recalculates the attribute value (steps S303 to S305).

When the processing in the data control system 1 (flow-based application 100 and data control function 150) is completed, the first information output unit 115 and the second information output unit 116 output the final processed data 125 and its updated attribute values 126, and the test data receiving unit 172 of the developer computer 170 receives the processed data 125 and the updated attribute values 126 as processed test data (steps S306, S307).

The developer computer 170 then outputs the processed test data (processed data 125 and updated attribute values 126) received by the test data receiving unit 172 from the output unit 16 of the developer computer 170 (see FIG. 2) (step S308). The developer 160 can evaluate whether the design of the flow-based application 100 is appropriate by comparing the processed data 125 and updated attribute values 126 output in step S308 with the processing results and attribute values of the data that the developer 160 intended in advance.

According to such a data processing verification system 2, not only the processed data 125, which is the final data processing result, but also the attribute values (updated attribute values 126) appropriately updated at each data processing stage are output as evaluation targets, making it easy to verify that there are no unexpected data changes between the software components of the flow-based application 100 that are invisible to the developer 160. In particular, the data processing verification system 2 has the advantage that verification using attribute values is possible more easily than verifying the data itself.

(2) Second embodiment Fig. 10 is a diagram showing a configuration example of a data control system 3 according to a second embodiment. The data control system 3 is configured by connecting a data storage application 300 and a storage 320 to the data control system 1 shown in the first embodiment. In addition, a user computer 190 operated by a user 180 of the flow-based application 100 is connected to the data control system 3. Note that a description of parts common to the configuration and processing described in the first embodiment will be omitted.

The user computer 190 is a computer having a hardware configuration similar to that of the computer 10 illustrated in FIG. 2, and has software for operating the flow-based application 100. The user 180 can use the flow-based application 100 by operating the user computer 190.

The user computer 190 includes an input data transmission unit 191 and a storage location receiving unit 192.

The input data transmission unit 191 transmits data 121 to be processed by the flow-based application 100 and attribute values 122 of the data to the flow-based application 100 based on the operation of the user 180. The data 121 and attribute values 122 transmitted from the input data transmission unit 191 are input to the information input unit 111 of the flow-based application 100.

The storage location receiving unit 192 receives storage location information 312 of the processed data 125 from the storage information output unit 306 of the data storage application 300.

The data storage application 300 is an application that automatically sorts and stores the processed data 125 that has been processed by the flow-based application 100, and outputs the storage location, and includes a first information input unit 301, a second information input unit 302, a data storage unit 303, a storage location determination unit 304, a setting data storage unit 305, and a storage information output unit 306.

The first information input unit 301 is connected to the first information output unit 115 of the flow-based application 100, and inputs the processed data 125 from the first information output unit 115. The second information input unit 302 is connected to the second information output unit 116 of the flow-based application 100, and inputs the updated attribute value 126 from the second information output unit 116.

The data storage unit 303 stores the processed data 125 input to the first information input unit 301 in a storage location determined by the storage location determination unit 304. After storing the processed data 125, the data storage unit 303 generates storage location information 312, which is information indicating the storage location, and transmits it to the storage information output unit 306.

The storage location determination unit 304 determines the storage location of the processed data 125 to be one of the multiple memory areas 321 of the storage 320 based on the updated attribute value 126 input to the second information input unit 302.

The setting data storage unit 305 stores information (setting rules 311) that is referenced when the storage location determination unit 304 determines the storage location.

The storage information output unit 306 transmits the storage location information 312 received from the data storage unit 303 to the user computer 190 that input the data 121 to the flow-based application 100.

Figure 11 is a sequence diagram showing an example of the overall processing procedure in data control system 3.

According to FIG. 11, the user 180 operates the user computer 190 to input data 121 and its attribute value 122 for which data processing is requested to the flow-based application 100, and requests storage of the data after data processing (processed data 125) (step S401).

Next, the flow-based application 100 and the data control function 150 perform data processing and recalculate attribute values for each data processing by the processing of the sequence diagram shown in FIG. 7 (steps S402, S403).

When the processing in the flow-based application 100 and the data control function 150 is completed, the processed data 125 and the updated attribute value 126 are sent to the data storage application 300 (step S404). In detail, the first information output unit 115 sends the processed data 125 to the first information input unit 301, and the second information output unit 116 sends the updated attribute value 126 to the second information input unit 302.

Next, in the data storage application 300, the storage location determination unit 304 evaluates the attribute value (updated attribute value 126) input to the second information input unit 302 and determines the storage location of the processed data 125 by the data storage unit 303 (step S405). When determining the storage location by the storage location determination unit 304, for example, evaluation is performed using the setting rules 311 stored in the setting data storage unit 305. A specific example of such a method of determining a storage location will be described later with reference to Figures 12 to 14.

The data referenced by the storage location determination unit 304 in determining the storage location is not limited to the setting rules 311 in the setting data storage unit 305, and other information may be referenced. Specifically, for example, in addition to the attribute value indicated in the updated attribute value 126, information on the processed data 125, the application from which the access originates (flow-based application 100), the user 180, or the user's computer 190 may be referenced. Furthermore, the data storage application 300 may be implemented to use other information that can be referenced by any method as the basis for determining the storage location.

In addition, the data referenced by the storage location determination unit 304 when determining a storage location is not limited to data prepared in advance, but may be, for example, data that is automatically generated by any method when determining the storage location or at any other time.

Next, the data storage unit 303 stores the data (processed data 125) input to the first information input unit 301 in step S404 in the storage location (memory area 321 of storage 320) determined in step S405 (step S406). Then, after storing the processed data 125 in the memory area 321, the storage 320 responds to the data storage application 300 with the data storage result (step S407).

In addition, in the data control system 3, after the storage location determination unit 304 determines the storage location in step S405, and before the data storage unit 303 stores the processed data 125 in the storage location in step S406, the data control system 3 may be implemented to confirm the storage location with the user 180 by any method.

Next, the data storage unit 303 transmits storage location information 312 indicating the location where the processed data 125 is stored to the storage information output unit 306, and the storage information output unit 306 outputs the storage location information 312 to the user computer 190 that made a request to the flow-based application 100 in step S401 (step S408).

Then, the user 180 can check the storage location of the data (processed data 125) after data processing of the input data (data 121) that he or she sent to the flow-based application 100 by checking the storage location information 312 displayed on the output unit 16 (see Figure 2) of the user computer 190 (step S409).

The above is an example of the overall processing procedure in the data control system 3. In the following, as a supplement, an example of a specific method for determining the storage location in step S405 will be described with reference to Figures 12 to 14.

Figure 12 shows an attribute value threshold list 410, Figure 13 shows an evaluation formula list 420, and Figure 14 shows a storage destination list 430. Each list shown in Figures 12 to 14 is an example of a setting rule 311 stored in the setting data storage unit 305.

The attribute value threshold list 410 shown in FIG. 12 is data in which a threshold is set for each attribute value, and a threshold is set for each of the attribute value names 411 to 413. The attribute value names 411 to 413 correspond to the attribute value names included in the updated attribute values 126 (see FIG. 4 for the data structure of the attribute value information), and for example, a threshold value "α" is set for the attribute value "attribute value a."

The evaluation formula list 420 shown in FIG. 13 is data that sets evaluation formulas for the updated attribute value 126, and has data items of evaluation order 421, evaluation formula 422, and storage destination 423. The evaluation order 421 indicates the execution order of the evaluation formulas 422, and the evaluation formulas 422 are executed with priority from the one with the lowest evaluation order, and the storage destination 423 is determined. The variables such as "α" and "β" used in the evaluation formula 422 correspond to the thresholds set in the attribute value threshold list 410 in FIG. 12. In addition, one of the memory areas 321 of the storage 320 is set as the storage location for the processed data 125 in the storage destination 423.

The storage destination list 430 shown in FIG. 14 is data that sets the access method to the storage location, and has data items of storage destination 431 and access method 432. The setting contents of storage destination 431 correspond to the setting contents of storage destination 423 of evaluation formula list 420 in FIG. 13.

The storage location determination unit 304 can determine the storage location (storage destination) of the processed data 125 by evaluating the updated attribute value 126 using each list (setting rule 311) in Figures 12 to 14 above.

As described above, according to the data control system 3, when it is necessary to use different storage methods depending on the nature of the data, the user 180 can automatically store the processed data 125 after data processing in an appropriate location based on the attribute value (updated attribute value 126) calculated by the data control function 150, without relying on the design of the flow-based application 100. This makes it possible to prevent storage of inappropriate data due to design errors in the flow-based application 100, operational errors by the user 180, or data unintentionally input by the user 180.

The present invention is not limited to the above-described embodiments, but includes various modified examples. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and are not necessarily limited to those having all of the configurations described. It is also possible to replace part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. It is also possible to add, delete, or replace part of the configuration of each embodiment with other configurations.

Furthermore, the above-mentioned configurations, functions, processing units, processing means, etc. may be realized in hardware, in part or in whole, for example by designing them as integrated circuits. Furthermore, the above-mentioned configurations, functions, etc. may be realized in software, by a processor interpreting and executing a program that realizes each function. Information on the programs, tables, files, etc. that realize each function can be stored in a memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD.

In addition, the control lines and information lines shown in the drawings are those that are considered necessary for explanation, and do not necessarily show all control lines and information lines on the product. In reality, it can be assumed that almost all components are interconnected.

1, 3 Data control system 2 Data processing verification system 10 Computer 11 CPU
12 Memory 13 Storage device 14 Communication unit 15 Input unit 16 Output unit 17 Bus 100 Flow-based application 101, 102, 103 Processing block 111 Information input unit 112 Data processing unit 113 Data storage unit 114 Control function request unit 115 First information output unit 116 Second information output unit 121 Data 122, 124 Attribute value 123 Internal data 125 Processed data 126 Updated attribute value 150 Data control function 151 Attribute value calculation unit 160 Developer 170 Developer computer 171 Test data transmission unit 172 Test data reception unit 180 User 190 User computer 191 Input data transmission unit 192 Storage location reception unit 210 Data 220 Attribute value information 231 Input attribute value information 232, 233 Changed attribute value information 234, 235, 236 Updated attribute value 300 Data storage application 301 First information input unit 302 Second information input unit 303 Data storage unit 304 Storage location determination unit 305 Setting data storage unit 306 Storage information output unit 311 Setting rule 312 Storage location information 320 Storage 321 Storage area 410 Attribute value threshold list 420 Evaluation formula list 430 Storage destination list

Claims (10)

A data control system that detects changes in data processed in a flow-based application, the whole or a part of which is composed of a combination of loosely coupled software components, comprising:
the plurality of loosely coupled software components constituting the flow-based application each include a software component having a function as a data processing unit that performs arbitrary data processing on input data and outputs processed data;
an attribute value calculation unit that calculates an attribute value that quantifies a change in data due to the data processing using predetermined calculation data;
a control function request unit that requests, each time the data processing by each of the data processing units is completed, the attribute value calculation unit to calculate the attribute value for the processed data of the data processing unit by attaching the calculation data related to the data processing unit;
Equipped with
a control function request unit that receives an updated attribute value, which is a calculation result of the attribute value for the processed data, from the attribute value calculation unit, and then transmits the processed data and the updated attribute value to a next software component of the data processing unit. 2. The data control system according to claim 1, wherein the attribute value calculation unit is implemented by a software component that is loosely coupled to a software component having a function as the flow-based application and the data processing unit. 3. The data control system according to claim 2, wherein the flow-based application, the software component having a function as the data processing unit, and the software component of the attribute value calculation unit input and output structured data of the same format. The calculation data attached by the control function request unit when making the request includes:
The input data and the processed data in the data processing are not included, but the input data includes item names of the structured data of the input data and the attribute values corresponding to each item name, the processed data includes item names of the structured data of the processed data, and processing content information indicating processing content in the data processing,
The data control system according to claim 3, characterized in that the attribute value calculation unit generates the updated attribute values by updating the attribute values corresponding to each item name of the structured data of the processed data by performing a predetermined list operation according to the type of processing content indicated in the processing content information based on the calculation data. When there is internal data added to the processed data from other than the input data in the data processing,
5. The data control system according to claim 4, wherein the control function request unit adds item names of the structured data of the internal data and the attribute values corresponding to each item name to the calculation data. A data control system comprising the flow-based application, the control function request unit, and the attribute value calculation unit,
2. The data control system according to claim 1, further comprising an information output unit that outputs the final processed data after the data processing is performed by all software components having a function as the data processing unit in the flow-based application, and the final updated attribute value calculated by the attribute value calculation unit for the final processed data. a data storage application connected to the information output unit and configured to input the final processed data and the final updated attribute value;
The data storage application includes:
a storage location determination unit that determines a storage location for the final processed data;
a data storage unit that stores the final processed data in a storage location determined by the storage location determination unit and generates storage location information indicating the storage location;
The data control system according to claim 6 , further comprising: a storage information output unit that outputs the storage location information generated by the data storage unit. The data control system according to claim 7 , wherein the storage location determination unit determines the storage location of the final processed data by evaluating the final updated attribute value using a predetermined setting rule. The data control system according to claim 7 , wherein the storage location includes a storage area of a storage device connected to the data storage application. A data control method for detecting a change in data processed in a flow-based application, the whole or a part of which is composed of a combination of loosely coupled software components, comprising:
the plurality of loosely coupled software components constituting the flow-based application each include a software component having a function as a data processing unit that performs arbitrary data processing on input data and outputs processed data;
a request step of requesting calculation of an attribute value for the processed data of the data processing by attaching calculation data related to the data processing each time the data processing by each software component having a function as the data processing unit is completed;
an attribute value calculation step of calculating, for the processed data, an attribute value that quantifies a change in data due to the data processing, using the calculation data attached to the request in the request step;
an output step of transmitting, after the calculation in the attribute value calculation step, the processed data and an updated attribute value which is the attribute value for the processed data, to a next software component of the data processing unit;
A data control method comprising:

Images