JP7087921B2 - Processing framework linkage device, processing framework linkage method and processing framework linkage program - Google Patents

Description

The present invention relates to a processing framework linkage device for linking a plurality of processing frameworks, a processing framework linkage method, and a processing framework linkage program.

In recent years, IoT (Internet of Things) has made it possible to realize a wider variety of IoT services by utilizing not only sensor data but also media such as camera video and audio. It is possible to develop such an IoT service easily and in a short time, and there are many processing frameworks for executing it in real time with low delay. For example, if users of these processing frameworks cooperate using a queuing system, it becomes possible to develop a service using a plurality of processing frameworks.

As a conventional technique, there is a technique of saving a message from a high-speed volatile storage area to a low-speed non-volatile storage area when the system is stopped to improve the processing speed and prevent message loss when the system is stopped (for example, the following patent). See Document 1). There is also a technique to prevent delay in queue processing by selecting either a high-speed small-capacity queue or a low-speed large-capacity queue based on the freeness of the high-speed small-capacity queue, the message retention rate, and the message priority. (For example, see Patent Document 2 below.).

Japanese Unexamined Patent Publication No. 2007-226385 Japanese Unexamined Patent Publication No. 2006-277483

However, in the prior art, it has not been possible to execute a service in which multiple processing frameworks are linked in real time with low delay. A configuration in which the queue is arranged in a volatile area such as a memory (in Memory, in-memory type) as in the prior art 2 and a configuration in which the queue is arranged in a non-volatile area such as a disk (not in Memory, referred to as a non-in-memory type). By combining these, the characteristics of high-speed small capacity and low-speed large capacity can be utilized. However, for example, when a high-speed in-memory type is selected for the queuing system, although the service can be executed in real time, the simultaneous use of a large number of services causes a memory shortage and the service throughput decreases.

Here, in the prior art, the delay other than the processing delay related to the processing of the processing framework is not considered. Therefore, when linking multiple processing frameworks using each queuing system, it is not possible to select an appropriate queuing system in response to other delays such as transfer delay, and multiple processing frameworks can be used. The linked service could not be executed in real time with low latency, and the overall processing throughput could not be improved. For example, even if the in-memory type has a small processing delay, if the processing frameworks are far apart, the transfer delay becomes large and approaches the non-in-memory type transfer delay.

In one aspect, it is an object of the present invention to be able to improve the throughput at the time of cooperation with processing frameworks.

According to one aspect of the present invention, the processing framework linkage device is a processing framework linkage device in which a linkage point between processing frameworks having a process included in the requested service definition is connected by a queuing system on a memory. Based on the processing delay of the queuing of each of the in-memory type queues placed in the queue and the non-in-memory type queuing systems placed on the disk, and the transfer delay of data transfer between the processing frameworks. It is a requirement that a control unit for selecting the in-memory type or the non-in-memory type as the queuing system is provided.

According to one aspect of the present invention, there is an effect that the throughput at the time of linking with the processing framework can be improved.

FIG. 1 is an explanatory diagram of processing framework linkage processing performed by the processing framework linkage device according to the embodiment. FIG. 2 is a block diagram showing a function of the processing framework cooperation device according to the embodiment. FIG. 3 is a diagram showing a hardware configuration example of the processing framework cooperation device according to the embodiment. FIG. 4 is a diagram illustrating the processing of the cooperation point extraction unit of the processing framework cooperation device according to the embodiment. FIG. 5 is a diagram illustrating the processing of the cooperation system extraction unit of the processing framework cooperation device according to the embodiment. FIG. 6 is a diagram illustrating the processing of the delay measurement unit and the cooperation system selection unit of the processing framework cooperation device according to the embodiment. FIG. 7 is a diagram illustrating the processing of the cooperation system insertion unit of the processing framework cooperation device according to the embodiment. FIG. 8 is a flowchart showing a processing example of the processing framework cooperation device according to the embodiment. FIG. 9 is a diagram showing another processing example of the processing framework cooperation device according to the embodiment. (Part 1) FIG. 10 is a diagram showing another processing example of the processing framework cooperation device according to the embodiment. (Part 2) FIG. 11 is a diagram showing another processing example of the processing framework cooperation device according to the embodiment. (Part 3) FIG. 12 is a diagram showing another processing example of the processing framework cooperation device according to the embodiment. (Part 4) FIG. 13 is a diagram showing another processing example of the processing framework cooperation device according to the embodiment. (Part 5)

Hereinafter, embodiments of the disclosed processing framework linkage device, processing framework linkage method, and processing framework linkage program will be described in detail with reference to the drawings.

FIG. 1 is an explanatory diagram of processing framework linkage processing performed by the processing framework linkage device according to the embodiment. For example, there are many processing frameworks for developing various IoT services easily and in a short time and executing them in real time with low delay. Each processing framework provides processing (called a filter) such as data and media acquisition, processing / detection / analysis as a library. The processing framework linkage device can provide various services depending on the combination of filters.

As shown in FIG. 1 (a), the existing single processing framework FW-A has filters A1 to A3, and FW-B has filters B1 and B2, respectively, which can be used. And there are restrictions on the services that can be developed. Therefore, by connecting different processing frameworks FW-A and FW-B with the queuing system Q, a service in which a plurality of processing frameworks FW-A and FW-B having a filter of the requested service are linked is linked. Can be provided.

The queuing system Q has an in-memory type and a non-in-memory type. The in-memory type has a configuration in which a queue is arranged in a volatile area such as a memory, and has a characteristic of high speed (high throughput, small processing delay) but a large memory usage. In the embodiment, the non-memory type refers to a configuration in which a queue is arranged in a non-volatile area such as a disk, and has a characteristic of low speed (low throughput and large processing delay) although the memory usage is small.

Here, when a high-speed in-memory type is selected for the queuing system Q, the service can be executed in real time, but due to the simultaneous use of a large number of services, a memory shortage occurs and the service throughput decreases.

In the embodiment, the processing frameworks FW-A and FW-B are linked by the processing shown in FIG. 1B. When the plurality of processing frameworks FW-A and FW-B are linked, the inventors utilize the non-in-memory type if the processing delay Ta of the queuing system Q is sufficiently smaller than the other delays. Attention was paid to the fact that there was no significant impact. In the embodiment, if the processing delay Ta of the queuing system Q is sufficiently smaller than the other delays, the non-in-memory queuing system Q is used. Other delays include a transfer delay Tb between the processing frameworks FW-A and FW-B, an execution delay (described later) which is the processing time of the entire service, and the like. For example, even if the processing delay Ta is a small in-memory type, if the processing frameworks FW-A and FW-B are far apart, the transfer delay Tb becomes large and the non-in-memory type transfer delay Tb. Approaching.

Here, when the transfer delay Tb is larger than the processing delay Ta, the influence is small even if the processing delay Ta is large. Therefore, by selecting a non-in-memory type with a small memory usage as the queuing system Q, the queuing system Q can be selected. Suppress the occurrence of memory shortage. On the other hand, when the transfer delay Tb is small, the influence of the processing delay is large, so that the in-memory type is selected as the queuing system Q so that high-speed processing can be executed.

First, the processing framework linkage device obtains a processing framework FW in which each filter included in the service definition S operates. In the example of FIG. 1 (b), it is assumed that the service definition S is the filter 1, the filter 2, and the filter 3. In this case, in the processing framework linkage device, the filters 1 and 2 are the processing framework FW-A, and the filter 3 is the processing framework FW-B. In addition, the processing framework linkage device extracts the linkage point P to which the processing framework FW switches. At this time, the processing framework cooperation device extracts the queuing system Q (cooperation system) that can be used (shared) by the processing frameworks FW-A and FW-B at both ends of the cooperation point P.

Here, it is assumed that the queuing system Q (cooperation system) that can be used by the processing frameworks FW-A and FW-B is a queue (Queue 1, 4, 5). It is assumed that QUEUE1 is an in-memory type and QUEUE4 and 5 are non-in-memory type. The processing framework linkage device connects the processing frameworks with the extracted queuing system Q (Queue 1, 4, 5), actually transfers data (sample data), and measures the transfer delay Tb. Then, the processing framework cooperation device selects one of Queue 1, 4, and 5 as the optimum queuing system Q using the measurement result of the transfer delay Tb.

Each queuing system Q (Queue 1, 4, 5) has a predetermined processing delay (corresponding to processing time) Ta from queue reception to queue transmission. Further, after data is transmitted from the processing framework FW-A (filter 2 (402)) on the data transmitting side, it is predetermined before data is received by the processing framework FW-B (filter 3 (413)) on the data receiving side. It has a transfer delay (corresponding to the transfer time) Tb. The transfer delay Tb includes the processing delay Ta of the queuing system Q (Queue 1, 4, 5).

Then, the processing framework cooperation device selects the queuing system Q to be used according to a predetermined condition. At this time, for example, the shortest queuing system Q is the in-memory type Queue1 and the transfer delay Tb of the non-in-memory type queuing system Q (for example, Queue5) is within a predetermined tolerance range close to Queue1. Suppose.

In this case, the processing framework linkage device selects the non-in-memory type queuing system Q (Queue5). As described above, the processing framework linkage device of the embodiment does not simply select the queuing system Q only by the processing delay Ta, but uses the transfer delay Tb when the actual data is transferred to the queuing system Q. Select. This makes it possible to select not only a high-speed in-memory type but also a non-in-memory type queuing system Q having a large capacity and a small memory usage.

After that, the processing framework linkage device performs the linkage processing (input / output processing) 701 and 702 necessary for using the selected queuing system Q (Queue5), and the processing frameworks FW-A and FW at both ends of the linkage point P. -Insert in B. The linkage process 701 is an output process for outputting the data output by the filter 2 (402) of the processing framework FW-A to the QUEUE 5. The linkage process 702 is an input process for inputting data input from the QUEUE 5 and outputting it to the filter 3 (413) of the processing framework FW-B.

Then, the processing framework linkage device outputs the modified service definition S'in which the linkage processes 701 and 702 are inserted into the initial service definition S to the user.

As described above, in the embodiment, the processing frameworks FW-A and FW-B in which the filter corresponding to the service definition S operates are extracted and linked. At this time, the transfer delay Tb between the processing frameworks FW-A and FW-B is actually measured. Further, the cooperation processes 701 and 702 necessary for using the selected queuing system Q are inserted into the processing frameworks FW-A and FW-B at both ends of the cooperation point P, respectively.

As a result, the optimum queuing system Q for connecting the processing frameworks can be selected, and not only the in-memory type but also the non-in-memory type queuing system Q can be selected. For example, when selecting a non-in-memory type queuing system Q, it is possible to avoid a memory shortage when using a large number of services at the same time, and as a result, it becomes possible to achieve low latency and improve throughput.

Further, the filters 1 and 2 operating in different processing frameworks FW-A and the filters 3 operating in FW-B can be connected by the queuing system Q, and by the cooperation of a plurality of processing frameworks FW-A and FW-B. We can flexibly respond to the required services. This makes it possible to develop and provide various required services.

FIG. 2 is a block diagram showing a function of the processing framework cooperation device according to the embodiment. The processing framework linkage device 100 includes a linkage point extraction unit 201, a filter database (DB) 202, a linkage system extraction unit 203, a framework DB 204, a delay measurement unit 205, a linkage system selection unit 206, and a linkage system insertion unit 207. ..

The cooperation point extraction unit 201 extracts from the filter DB 202 a plurality of processing frameworks that can use the filter included in the service definition S requested by the user, and extracts the cooperation points at which the processing frameworks are switched. The filter DB 202 stores and holds a processing framework for each filter.

The cooperation system extraction unit 203 refers to the framework DB 204 and extracts the queuing system Q that can be used (shared) by the processing frameworks at both ends of the cooperation point as the cooperation system. The framework DB 204 stores and holds the corresponding queuing system Q for each processing framework for each in-memory type and non-in-memory type.

The information held by the filter DB 202 and the framework DB 204 is information unique to each framework FW and can be stored and held in advance.

The delay measurement unit 205 connects the processing frameworks with the queuing system Q extracted by the cooperation system extraction unit 203, and measures the transfer delay when data is transferred. The delay measuring unit 205 actually measures the delay characteristics using appropriate sample data. The cooperative system selection unit 206 selects the optimum queuing system Q based on a predetermined determination condition using the measured transfer delay. The cooperation system selection unit 206 selects an in-memory type or non-in-memory type queuing system Q according to a predetermined determination condition.

The linkage system insertion unit 207 inserts the linkage processing necessary for using the queuing system Q selected by the linkage system selection unit 206 into the processing frameworks at both ends, and a plurality of processing frameworks can be executed in cooperation with each other. Output the information of service definition S'.

FIG. 3 is a diagram showing a hardware configuration example of the processing framework cooperation device according to the embodiment. As the processing framework linkage device 100 shown in FIG. 2, the hardware shown in FIG. 3 can be used, and for example, a server can be used.

For example, the processing framework linkage device 100 includes a CPU (Central Processing Unit) 301, a memory 302, a network interface (I / F) 303, a recording medium IF 304, and a recording medium 305. Reference numeral 300 is a bus connecting each part.

The CPU 301 is an arithmetic processing device that functions as a control unit that controls the entire processing framework cooperation device 100. The memory 302 includes a non-volatile memory and a volatile memory. The non-volatile memory is, for example, a ROM (Read Only Memory) for storing the program of the CPU 301. The volatile memory is, for example, a DRAM (Dynamic Random Access Memory) used as a work area of the CPU 301, a SRAM (Static Random Access Memory), or the like.

The network I / F 303 can also access a processing framework outside the processing framework linkage device 100, such as a server or a cloud, through the network 310, and use the external processing framework. The network IF 303 is communicated and connected to an external server, cloud, or the like via a network 310 such as a LAN (Local Area Network), a WAN (Wide Area Network), or the Internet.

The recording medium IF 304 is an interface for reading and writing the information processed by the CPU 301 to and from the recording medium 305. The recording medium 305 is a recording device that assists the memory 302. For example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), a USB (Universal Serial Bus) flash drive, or the like can be used.

By executing the program recorded in the memory 302 or the recording medium 305 by the CPU 301, each function of the processing framework linkage device 100 shown in FIG. 2 is realized. Further, the memory 302 and the recording medium 305 can hold the information of the filter DB 202 and the framework DB 204 shown in FIG. 2 in advance. Then, the CPU 301 can easily read the information of the framework FW in which the filter operates corresponding to the service and the information of the queuing system for each framework FW.

Then, in the embodiment, for example, in the in-memory type queuing system Q, the queue (Queue) is arranged on the memory 302. Further, in the non-in-memory type queuing system Q, a cue is arranged on a recording medium 305 such as a disk.

Next, the processes performed by the processing framework linkage device according to the embodiment will be described in order by function.

FIG. 4 is a diagram illustrating the processing of the cooperation point extraction unit of the processing framework cooperation device according to the embodiment. The processing performed by the cooperation point extraction unit 201 of the processing framework cooperation device 100 will be described. FIG. 4A is a diagram showing a filter corresponding to a predetermined service definition S, and FIG. 4B is a diagram showing information of a processing framework for each filter held by the filter DB 202.

As shown in FIG. 4A, it is assumed that the service definition S is the filter 1 (401) to the filter 2 (402) to the filter 3 (413). In this case, the cooperation point extraction unit 201 refers to the filter DB 202 shown in FIG. 4 (b) and can use the corresponding filter 1 (401), filter 2 (402), and filter 3 (413) as a processing framework FW. Ask for.

In the example shown in FIG. 4B, in the filter DB 202, the filters 1 and 2 (401, 402) are the processing framework FW-A, and the filter 3 (413) is the processing framework FW-B. It is being held. In this case, the linkage point extraction unit 201 determines that the processing framework FW of the linkage point P to which the processing framework FW is switched is the processing framework FW-A (filter 2) and the FW-B (filter 3) portion. Extract.

FIG. 5 is a diagram illustrating the processing of the cooperation system extraction unit of the processing framework cooperation device according to the embodiment. The cooperation system extraction unit 203 of the processing framework cooperation device 100 refers to the queuing system Q for each processing framework stored in the framework DB 204. Then, the queuing system Q (cooperation system) that can be used (shared) by the processing framework FW at both ends of the cooperation point P is extracted.

In the example shown in FIG. 5, in the framework DB 204, the in-memory types are Queue1 and Queue2, and the non-in-memory types are Queue4 and Queue5 as the queuing system Q corresponding to the processing framework FW-A. Information is stored. Further, as the queuing system Q corresponding to the processing framework FW-B, information that the in-memory type is Queue1 and the non-in-memory type is Queue3, Queue4, and Queue5 is stored.

In this case, the cooperation system extraction unit 203 extracts Queue1 as an in-memory type as a common (shared) queue between two different processing frameworks FW-A and FW-B, and is a non-in-memory type. Extracts Queue 4 and Queue 5. Here, in the linkage system extraction unit 203, the in-memory type QUEUE1 and the non-in-memory type QUEUE4, 5 are the processing frameworks FW-A and FW-B at both ends of the linkage point P (filter 2 and filter 3). Judge that it is an available queuing system Q and extract it.

The cooperation system extraction unit 203 does not extract the in-memory type QUEUE2 because it exists only in one of the processing frameworks FW-A and cannot be shared in FW-B. Similarly, the non-in-memory type QUEUE3 is not extracted because it exists only in one of the processing frameworks FW-B and cannot be shared because it is not in FW-A.

FIG. 6 is a diagram illustrating the processing of the delay measurement unit and the cooperation system selection unit of the processing framework cooperation device according to the embodiment. The delay measuring unit 205 of the processing framework linkage device 100 connects the processing frameworks with the queuing system Q extracted by the linkage system extraction unit 203, and measures the transfer delay when data (sample data) is transferred. .. Further, the cooperation system selection unit 206 selects the optimum queuing system Q by using the measurement result of the transfer delay by the delay measurement unit 205.

FIG. 6A shows an in-memory type Queue 1 which is a queuing system Q extracted by the cooperation system extraction unit 203, and a non-in-memory type Queue 4, 5. These in-memory type Queue 1 and non-in-memory type Queue 4 and 5 are located at both ends of the cooperation point P. In the processing framework FW-A, the filter 2 (402) outputs data to Queue 1, 4, and 5. The Queues 1, 4 and 5 input data to the filter 3 (413) of the processing framework FW-B.

Here, each queuing system Q (Queue 1, 4, 5) has a predetermined processing delay (corresponding to processing time) Ta from queue reception to queue transmission, respectively. Further, after data is transmitted from the processing framework FW-A (filter 2 (402)) on the data transmitting side, it is predetermined before data is received by the processing framework FW-B (filter 3 (413)) on the data receiving side. It has a transfer delay (corresponding to the transfer time) Tb. The transfer delay Tb includes the processing delay Ta of the queuing system Q (Queue 1, 4, 5).

The processing delay Ta is different for each queuing system Q. In general, the in-memory type (Queue1) has a shorter processing delay Ta than the non-in-memory type (Queue4, 5). Further, although the transfer delay Tb includes the processing delay Ta, it is affected by the actual data transfer state, so that the in-memory type is not always short, and the non-in-memory type may also be short. Therefore, the delay measurement unit 205 actually transfers sample data for each of the in-memory type Queue 1 and the non-in-memory type Queue 4, 5 which are the queuing system Q extracted by the cooperation system extraction unit 203. And measure the transfer delay Tb. As the sample data, for example, a random binary data string can be used.

FIG. 6B is a chart showing transfer delay measurement results 600 of the in-memory type Queue 1 which is the queuing system Q extracted by the cooperation system extraction unit 203 and the transfer delay Tb of the non-in-memory type Queues 4 and 5. be. The processing delay Ta is a fixed characteristic value (reference value).

The delay measuring unit 205 connects the processing frameworks with each queuing system Q (Queue 1, 4, 5), transfers predetermined sample data, and measures the transfer delay Tb. The transfer delay Tb is a period from the transmission of data by the filter 402 of the processing framework FW-A of the previous stage to the reception of data by the filter 413 of the processing framework FW-B of the subsequent stage. The delay measurement unit 205 stores and holds the transfer delay measurement result 600 of the transfer delay Tb for each queuing system Q (Queue 1, 4, 5) in, for example, a memory 302, a recording medium 305, or the like. By storing and holding the transfer delay Tb for each queuing system Q, it can be read from the memory 302, the recording medium 305, etc. and used without measuring the transfer delay Tb at the time of connection of the same processing framework FW. ..

Then, the cooperation system selection unit 206 has the following conditions 1. ~ 3. Therefore, select the queuing system Q to be used.
1. 1. The queuing system Q having the shortest transfer delay Tb is defined as the shortest transfer delay Dmin.
2. 2. If there is a non-in-memory type queuing system Q in which the transfer delay Tb is Dmin × N or less, the one with the shortest transfer delay Tb is selected. N is the tolerance (constant) from the shortest
3. 3. If there is no non-in-memory type queuing system Q in which the transfer delay Tb is Dmin × N or less, the in-memory type queuing system Q having the shortest transfer delay Tb is selected.

For example, in the transfer delay measurement result 600 of FIG. 6B, the one with the shortest transfer delay Tb (Dmin) is the in-memory type QUEUE1 (700 msec). It is assumed that the next shortest is the non-in-memory type QUEUE5 (900 msec), and the last is the non-in-memory type QUEUE4 (1200 msec).

The cooperative system selection unit 206 refers to the transfer delay measurement result 600 in FIG. 6B, and when N = 1.3 is set, Queue 5 is selected as the queuing system Q. In this case, (Transfer delay Tb of Queen 5 = 900 msec) <(Transfer delay Tb of Queen 1 = 700 × 1.3 = 910).

When N = 1.2 is set, the cooperation system selection unit 206 selects Queue 1 as the queuing system Q. In this case, (Transfer delay Tb of Queen 5 = 900 msec)> (Transfer delay Tb of Queen 1 = 700 × 1.2 = 840).

N is a constant indicating how much delay may be made with respect to the shortest (best effort), and is arbitrarily set by the user.

In the transfer delay measurement result 600 of FIG. 6B, the transfer delay Tb (900 msec) of the non-in-memory type QUEUE 5 is close to the transfer delay Tb (700 msec) of the in-memory type QUEUE 1. In the transfer using QUEUE5, for example, the transfer delay Tb may be shortened because the transfer distance from the data transmission to the data reception is short. As described above, in the embodiment, the optimum queuing system Q is selected by using the transfer delay Tb when the data is actually transferred between the processing framework FW.

FIG. 7 is a diagram illustrating the processing of the cooperation system insertion unit of the processing framework cooperation device according to the embodiment. The cooperation system insertion unit 207 of the processing framework cooperation device 100 performs a process of connecting the processing frameworks with the optimum queuing system Q selected by the cooperation system selection unit 206.

FIG. 7A shows the selection state of the queuing system Q selected by the cooperation system selection unit 206 corresponding to the service definition. In the example of FIG. 7A, the queuing 5 optimal as the queuing system Q is arranged at the cooperation point P between the processing frameworks FW-A (filters 1 and 2) and FW-B (filter 3).

Then, the cooperation system insertion unit 207 inserts the cooperation processing necessary for using the selected queuing system Q (Queue5) into the processing frameworks FW-A and FW-B at both ends of the cooperation point P, respectively.

In the example of FIG. 7B, the linkage system insertion unit 207 inserts the linkage processing 701 of the FW-A after the filters 1 (401) and the filters 2 (402) of the processing framework FW-A. Further, the linkage system insertion unit 207 inserts the linkage processing 702 of the FW-B in front of the filter 3 (413) of the processing framework FW-B. The linkage process 701 is a process of outputting the data output by the filter 2 (402) of the processing framework FW-A to the QUEUE 5. The linkage process 702 is a process of outputting the data input from the QUEUE 5 to the filter 3 (413) of the processing framework FW-B.

As described above, the cooperation system insertion unit 207 outputs the modified service definition S'in which the cooperation processes 701 and 702 are inserted into the initial service definition S shown in FIG. 7A to the user. As a result, information on services that can be executed in cooperation with a plurality of processing frameworks FW-A and FW-B (modified service definition S') can be output to the user.

The modified service definition S'is linked via the queuing system Q, which minimizes the transfer delay between the plurality of processing frameworks FW-A and FW-B. As a result, according to the embodiment, it becomes possible to improve the throughput of a predetermined service in which a plurality of processing frameworks FW-A and FW-B are linked. In the above explanation, the case where there is one cooperation point P has been described as an example, but even when there are a plurality of cooperation points P, the optimum queuing system Q in which the transfer delay is the shortest for each cooperation point P is provided. You can select it.

FIG. 8 is a flowchart showing a processing example of the processing framework cooperation device according to the embodiment. The processing of each functional unit (see FIG. 2) executed by the CPU 301 of the processing framework linkage device 100 will be described in order.

First, the processing framework linkage device 100 (linkage point extraction unit 201) refers to the filter DB 202 and obtains a processing framework FW in which each filter of the requested service definition S operates (step S801). Next, the processing framework linkage device 100 (linkage point extraction unit 201) extracts the linkage point P to which the processing framework FW switches (step S802).

Next, the processing framework linkage device 100 executes the following processing for one linkage point P. First, the processing framework linkage device 100 (linkage point extraction unit 201) refers to the framework DB 204 and extracts the queuing system Q (Queue) that can be used by the processing framework FW at both ends of the linkage point P (step S803). ).

Next, the processing framework linkage device 100 (delay measurement unit 205) connects with the queuing system Q (Queue) extracted between the processing framework FWs, and measures each transfer delay Tb (step S804). The transfer delay Tb of each queuing system Q (Queue) when data is actually transferred using the plurality of queuing systems Q extracted in advance in step S803 is measured, and the transfer delay measurement result 600 is stored in the memory 302. You may store it.

Next, the processing framework linkage device 100 (linkage system selection unit 206) determines the above-mentioned predetermined conditions 1. for the transfer delay Tb of each queuing system Q (Queue). ~ 3. Is applied to select the optimum queuing system Q (Queue).

For example, the processing framework linkage device 100 (linkage system selection unit 206) determines whether the shortest transfer delay Tb ≦ the shortest transfer delay Dmin × N of the non-in-memory type queuing system Q (Queue) ( Step S805).

As a result of the determination in step S805, it is assumed that the condition of the shortest transfer delay Tb ≦ the shortest transfer delay Dmin × N of the non-in-memory type queuing system Q (Queue) is satisfied (step S805: Yes). In this case, the processing framework linkage device 100 (linkage system selection unit 206) selects the non-in-memory type queuing system Q (Queue) having the shortest transfer delay Tb (step S806).

On the other hand, as a result of the determination in step S805, it is assumed that the condition of the shortest transfer delay Tb ≦ the shortest transfer delay Dmin × N of the non-in-memory type queuing system Q (Queue) is not satisfied (step S805: No). In this case, the processing framework linkage device 100 (linkage system selection unit 206) selects the in-memory type queuing system Q (Queue) having the shortest transfer delay Tb (step S807).

After that, the processing framework linkage device 100 (linkage system insertion unit 207) operates at both ends of the linkage point P with the processing framework FW at each end, and performs linkage processing using the selected queuing system Q (Queue). Insert (step S808). The processing framework linkage device 100 (linkage system selection unit 206) outputs a service (modified service definition S') that can be executed in cooperation with the processing framework.

According to the above processing, actual data transfer is performed to a plurality of queuing systems Q that can be used when the processing frameworks FW are connected at the cooperation point P, and the transfer delay Tb is obtained. At this time, if a short transfer delay Tb is obtained even in the non-in-memory type, the non-in-memory type queuing system Q can be selected rather than the in-memory type. When the non-in-memory type is adopted, the queue can be arranged on a disk (recording medium 305) or the like, and the amount of memory 302 used can be reduced while preventing a decrease in throughput. Further, even when a large number of services are used at the same time, a large capacity disk area can be used, and it is possible to prevent a memory shortage of the memory 302.

Next, various processing examples other than the above-mentioned processing examples will be described. 9 to 13 are diagrams showing other processing examples of the processing framework cooperation device according to the embodiment. In each of these figures, based on the flowchart shown in FIG. 8, only different processes are extracted and described. The control unit (CPU301) of the processing framework linkage device 100 is based on the processing shown in FIG. ~ 5. Any of the processes of FIGS. 9 to 13 can be executed in response to the change in the state of the service shown in FIG.

(1. When the frequency of data handled by the service is high)
When the occurrence frequency of the data handled by the service is higher than the predetermined occurrence frequency, the non-in-memory type cannot process in time, so that the in-memory type capable of higher-speed processing than the non-in-memory type is effective. In such a case, the processing framework linkage device 100 (delay measurement unit 205) measures the throughput when data is continuously transferred instead of the transfer delay described in the above-described embodiment.

Then, the processing framework cooperation device 100 (cooperation system selection unit 206) has the following conditions a. ~ C. Select the queuing system Q according to.
a. Let Tmax be the throughput of the queuing system Q with the maximum throughput.
b. If there is a non-in-memory type queuing system Q having a throughput of Tmax ÷ N'or more, the non-in-memory type queuing system Q having the maximum throughput is selected. N'is the tolerance (constant) from the maximum.
c. If there is no non-in-memory type queuing system Q having a throughput of Tmax ÷ N'or more, the queuing system Q having the maximum throughput is selected from the in-memory type queuing systems Q.

FIG. 9 is a flowchart showing a processing example when the data handled by the service is frequently generated. In FIG. 9, the processes of steps S804 to S807 of FIG. 8 are replaced. After the processing of step S803 in FIG. 8, the processing framework cooperation device 100 (delay measuring unit 205) is connected by the queuing system Q (Queue) extracted between the processing framework FWs. Then, the throughput of each queuing system Q is measured (step S804a).

Next, the processing framework linkage device 100 (linkage system selection unit 206) determines whether the maximum throughput ≥ maximum throughput / N'of the non-in-memory type queuing system Q (Queue) (step S805a).

As a result of the determination in step S805, it is assumed that the condition of maximum throughput ≥ maximum throughput / N'of the non-in-memory type queuing system Q (Queue) is satisfied (step S805a: Yes). In this case, the processing framework linkage device 100 (linkage system selection unit 206) selects the non-in-memory type queuing system Q (Queue) having the maximum throughput (step S806a).

On the other hand, as a result of the determination in step S805a, it is assumed that the condition of maximum throughput ≥ maximum throughput / N'of the non-in-memory type queuing system Q (Queue) is not satisfied (step S805a: No). In this case, the processing framework linkage device 100 (linkage system selection unit 206) selects the in-memory type queuing system Q (Queue) having the maximum throughput (step S807a). After that, the process proceeds to the process of step S808 of FIG.

In this way, when the frequency of data handled by the service is high, the throughput can be improved by preferentially using the in-memory type that can perform high-speed processing over the non-in-memory type. ..

(2. When the data size handled by the service is large)
When the data size handled by the service is larger than the predetermined data size, the in-memory type uses a large amount of the memory 302, so that the non-in-memory type that can take a large data area is effective. In such a case, the processing framework linkage device 100 (delay measurement unit 205) measures the transfer delay Tb using sample data having a large data size corresponding to the designated data size used in the service.

FIG. 10 is a flowchart showing a processing example when the data size handled by the service is large. FIG. 10 replaces the process of step S804 of FIG. After the processing of step S803 in FIG. 8, the processing framework cooperation device 100 (delay measuring unit 205) is connected by the queuing system Q (Queue) extracted between the processing framework FWs. Then, the transfer delay Tb of each queuing system Q is measured using the designated data size (step S804b). After that, the process proceeds to the process of step S805 of FIG.

As described above, when the data size handled by the service is large, the transfer delay Tb is measured using a large data size corresponding to the designated data size used in the service. As a result, for example, when transferring a large data size, the non-in-memory type that can secure the memory capacity is preferentially used over the in-memory type, so that the data transfer throughput can be improved.

(3. When there are multiple cooperation points in the service)
In the above embodiment, the case where there is one cooperation point P has been described, but the processing example when there are a plurality of cooperation points P in the service will be described below. When there are a plurality of cooperation points P in the service, the cooperation point P having a large transfer delay Tb affects the processing delay of the entire service (becomes a bottleneck). Further, as the number of queuing systems Q used increases, the amount of memory used increases. Therefore, in this processing example, the queuing system Q is selected from the cooperation point P having the longest transfer delay Dmin (the shortest transfer delay is large). Further, the same system is used for the cooperation points that can be used by the selected queuing system Q.

FIG. 11 is a flowchart showing a processing example when there are a plurality of cooperation points in the service. As shown in FIG. 11, after the processing performed for each cooperation point by the processing of step S803 and step S804 of FIG. 8 is executed, the processing performed in order from the cooperation point P having the shortest transfer delay Tb is executed.

After the processing of step S804 of FIG. 8, the processing framework linkage device 100 (linkage system selection unit 206) determines whether or not there is a selected and available queuing system Q (Queue) (step S1101). As a result of the determination, if there is a selected and available queuing system Q (step S1101: Yes), the delay measuring unit 205 has the shortest transfer delay Tb ≤ the shortest transfer delay Dmin of the non-in-memory queuing system Q. It is determined whether it is × N (step S805).

As a result of the determination in step S805, it is assumed that the condition of the shortest transfer delay Tb ≦ the shortest transfer delay Dmin × N of the non-in-memory type queuing system Q (Queue) is satisfied (step S805: Yes). In this case, the processing framework linkage device 100 (linkage system selection unit 206) selects the non-in-memory type queuing system Q (Queue) having the shortest transfer delay Tb (step S806).

On the other hand, as a result of the determination in step S805, it is assumed that the condition of the shortest transfer delay Tb ≦ the shortest transfer delay Dmin × N of the non-in-memory type queuing system Q (Queue) is not satisfied (step S805: No). In this case, the processing framework linkage device 100 (linkage system selection unit 206) selects the in-memory type queuing system Q (Queue) having the shortest transfer delay Tb (step S807).

If there is no queuing system Q that has been selected and can be used as a result of the determination in step S1101 (step S1101: No), the processing framework linkage device 100 (linkage system selection unit 206) executes step S1102. In step S1102, the processing framework linkage device 100 (linkage system selection unit 206) selects the one with the shortest transfer delay Tb among the selected and available queuing systems Q (step S1102). Then, after the execution of step S806, step S807, or step S1102, the process proceeds to step S808.

In this way, when there are a plurality of cooperation points P in the service, the queuing system Q is sequentially selected from the cooperation point P having the longest transfer delay Tb among the queuing systems Q connecting the plurality of processing frameworks FW. I will choose. Then, if there is a selected queuing system Q, the queuing system Q having the shortest delay is selected from the selected and available queuing systems Q. If there is no queuing system Q that has been selected and can be used, the non-in-memory type is preferentially selected based on the shortest transfer delay Tb, as in the processing example described in the above embodiment. As a result, even when there are a plurality of cooperation points P, the throughput of the entire system can be improved.

(4. When multiple services are running)
When a plurality of services are executed, the queuing system Q to be used increases and the memory usage increases. In such a case, the queuing system Q already used by other services is preferentially selected. At this time, if the transfer delay Tb of another service affects the service, the transfer delay Tb may be measured again and the queuing system Q may be reselected. Further, the transfer delay Tb may be remeasured and the queuing system Q may be reselected based on the priority of the service.

FIG. 12 is a flowchart showing a processing example when a plurality of services are executed. After executing the process of step S803 in FIG. 8, the processing framework linkage device 100 (linkage system extraction unit 203) is using the queuing system Q, if there is a queuing system Q already used by another service. Narrow down to Q (step S1201). After that, the process proceeds to step S804.

In this way, when a plurality of services are executed, the queuing system Q already used by the other services is preferentially selected. This makes it possible to reduce the amount of memory used by using the queuing system Q used by other services when executing multiple services, for example, when the same video is used by multiple services. Become.

(5. When the service execution delay is sufficiently larger than the transfer delay or processing delay)
FIG. 13A is a diagram showing a modified service definition S'. When the execution delay Tc, which is the execution time of the entire service, is sufficiently larger than the transfer delay Tb or the processing delay Ta, the effect of the delay by the queuing system Q is small, so that even if the transfer delay Tb is large, there is no effect. In this case, by measuring the execution time (execution delay Tc) of the entire service in addition to the transfer delay Tb, if the execution delay Tc is sufficiently larger than the transfer delay Tb, the queuing system Q having a large transfer delay Tb can be obtained. select.

FIG. 13B is a flowchart showing a processing example when the execution delay of the service is sufficiently larger than the transfer delay or the processing delay. After executing the processing in step S803 of FIG. 8, the processing framework cooperation device 100 (delay measurement unit 205) is connected by the queuing system Q (Queue) extracted between the processing framework FWs. Then, the transfer delay Tb and the execution delay Tc of each queuing system Q are measured (step S804c).

After that, the processing framework linkage device 100 (delay measurement unit 205) determines whether the execution delay Tc ≧ the shortest transfer delay Tb × M (constant) (step S1301). Here, the determination is made for the queuing system Q of the shortest transfer delay Tb. As a result of the determination, if the execution delay Tc ≧ the shortest transfer delay Dmin × M (constant) (step S1301: Yes), the process proceeds to step S1302. In this case, the processing framework linkage device 100 (linkage system selection unit 206) preferentially selects from the queuing system Q having the longest transfer delay Tb (step S1302). On the other hand, it is assumed that the execution delay Tc ≧ the shortest transfer delay Dmin × M (constant) (step S1301: No). In this case, the processing framework cooperation device 100 (cooperation system selection unit 206) shifts to the processing of step S805.

In step S805, the delay measuring unit 205 determines whether the shortest transfer delay Tb ≦ the shortest transfer delay Dmin × N of the non-in-memory queuing system Q (step S805). As a result of the determination in step S805, it is assumed that the condition of the shortest transfer delay Tb ≦ the shortest transfer delay Dmin × N of the non-in-memory type queuing system Q (Queue) is satisfied (step S805: Yes). In this case, the processing framework linkage device 100 (linkage system selection unit 206) selects the non-in-memory type queuing system Q (Queue) having the shortest transfer delay Tb (step S806).

On the other hand, as a result of the determination in step S805, it is assumed that the condition of the shortest transfer delay Tb ≦ the shortest transfer delay Dmin × N of the non-in-memory type queuing system Q (Queue) is not satisfied (step S805: No). In this case, the processing framework linkage device 100 (linkage system selection unit 206) selects the in-memory type queuing system Q (Queue) having the shortest transfer delay Tb (step S807). Then, after the processing of step S806, step S807, and step S1302, the process proceeds to the processing of step S808.

As described above, when the execution delay Tc of the service is sufficiently larger than the transfer delay Tb or the processing delay Ta, the influence of the delay due to the queuing system Q becomes relatively small, and even if the transfer delay Tb is large, the influence is exerted. Does not occur. Therefore, the execution delay Tc is measured, and if the execution delay Tc is sufficiently larger than the transfer delay Tb, the queuing system Q having a large transfer delay Tb is selected. This allows other services to utilize the low latency queuing system Q.

According to the embodiment described above, the transfer delay between the processing frameworks is measured when the processing frameworks in which the filters corresponding to the service definitions operate are linked. As a result, the optimum queuing system for connecting the processing frameworks can be selected, and not only the in-memory type but also the non-in-memory type queuing system can be selected. For example, for the shortest transfer delay among a plurality of queuing systems, if the non-in-memory type transfer delay is within a predetermined range, the shortest transfer delay among the non-in-memory types is selected. When selecting a non-in-memory queuing system, it is possible to avoid running out of memory when using a large number of services at the same time, and as a result, it is possible to improve the throughput of services with low latency.

In addition, the linkage processing required to use the selected queuing system is inserted into the processing framework at both ends of the linkage point. This makes it possible to connect filters in different processing frameworks with a queuing system, flexibly respond to the services required by linking multiple processing frameworks, and develop and provide various services. Become.

Also, paying attention to the fact that if the processing delay of the queuing system is sufficiently smaller than other delays, there is no effect even if it is a non-in-memory type, and the queuing system is used to actually transfer data between processing frameworks. The transfer delay at the time of transfer is measured. As a result, not only the processing delay of the queuing system itself but also the measurement result of the transfer delay can be used to improve the throughput with low delay when actually providing the service.

In addition, the processing framework in which the processing included in the service definition operates is obtained, and the cooperation points between the processing frameworks are extracted. In addition, the queuing system that can be shared by the queuing systems at both ends of the extracted cooperation point is extracted. At this time, information on the processing framework in which the processing included in the service definition operates and the queuing system that can be shared can be stored in advance in the database, and can be easily obtained by referring to the database. .. In addition, filters in different processing frameworks can be connected by a queuing system, and multiple processing frameworks can be linked.

Further, among a plurality of queuing systems, if the non-in-memory type transfer delay is not within a predetermined range for the shortest transfer delay, the in-memory type with the shortest transfer speed is selected. As described above, in the embodiment, the non-in-memory type is preferentially used as much as possible, but various queuing systems including the in-memory type can be selected.

Further, by outputting the service definition S as a modified service definition S'corresponding to the linked processing inserted between the processing frameworks, the processing frameworks can be linked and executed.

Further, when the frequency of occurrence of data handled by the service is higher than a predetermined frequency, the throughput of a plurality of queuing systems when the data is continuously transferred may be measured. Then, if the non-in-memory type throughput is within a predetermined range with respect to the maximum throughput of the plurality of queuing systems, the non-in-memory type with the maximum throughput is selected. In this way, by selecting a high-speed in-memory type corresponding to the high frequency of data generation, it is possible to suppress a decrease in the overall processing throughput.

Further, when the size of the data handled by the service is larger than the predetermined size, the predetermined data size used in the service may be transferred in advance and the transfer delay of each of the queuing systems may be measured. This makes it possible to prevent a decrease in the overall processing throughput by selecting the non-in-memory type in response to the case where a large data size is used in the actual service.

If there are multiple linkage points in the service, the queuing system is selected from the linkage points with the largest shortest transfer delay, and the other linkage points are the same as selected if the selected queuing system is available. A queuing system may be used. As a result, it is possible to suppress the occurrence of a linkage point having a large transfer delay during execution of a service having a plurality of linkage points, and it is possible to prevent a decrease in the overall processing throughput.

Further, when a plurality of services are executed, the queuing system used by other services may be preferentially selected. For example, when the same video is used by a plurality of different services, the same queuing system is used for each service. As a result, it is possible to suppress an increase in the number of queuing systems to be used, suppress memory usage, and prevent a decrease in the overall processing throughput.

In addition, the execution delay of the entire service may be measured, and if the execution delay is sufficiently larger than the transfer delay or the processing delay, the queuing system with the longest transfer delay may be selected first. In such a case, even if the processing delay or transfer delay due to the queuing system is large, the effect of the delay on the entire service is small, so it is possible to select from the queuing system with the longest transfer delay such as non-in-memory type. can. This makes it possible for other services to utilize a queuing system with low delay (processing delay and transfer delay), and it is possible to prevent a decrease in the overall processing throughput when executing multiple services.

The processing framework linkage program described in the embodiment of the present invention can be realized by causing a processor such as a server to execute a program prepared in advance. This control method is recorded on a computer-readable recording medium such as a hard disk, a flexible disk, a CD-ROM (Compact Disc-Read Only Memory), or a flash memory, and is read from the recording medium and executed by the computer. Further, this control method may be distributed via a network such as the Internet.

The following additional notes are further disclosed with respect to the above-described embodiment.

(Appendix 1) In the processing framework cooperation device that connects the cooperation points between the processing frameworks having the processing included in the service definition of the requested service by the queuing system.
The processing delay of queuing for each of the in-memory type queuing system in which the queue is arranged on the memory and the non-in-memory type queuing system in which the queue is arranged on the disk, and the transfer delay for data transfer between the processing frameworks. A control unit that selects the in-memory type or the non-in-memory type as the queuing system based on
A processing framework cooperation device characterized by being equipped with.

(Appendix 2) The control unit is
A linkage point extraction unit that extracts the linkage points between the processing frameworks from the service definition,
A linkage system extraction unit that extracts a plurality of available queuing systems for each of the extracted linkage points.
A delay measurement unit that connects the cooperation points with the extracted queuing system and measures the data transfer delay between the processing frameworks.
A selection unit that selects the in-memory type or the non-in-memory type as the queuing system to be used based on the measured transfer delay.
A linkage system insertion unit that inserts a linkage process that connects the linkage points in the selected queuing system, and a linkage system insertion unit.
The processing framework linkage device according to Appendix 1, wherein the processing framework linkage device is provided.

(Appendix 3) The selection unit is
If the non-in-memory type transfer delay is within a predetermined range with respect to the queuing system having the shortest transfer delay among the plurality of queuing systems, the shortest transfer delay among the non-in-memory types is said. The processing framework linkage device according to Appendix 2, wherein a queuing system is selected.

(Appendix 4) The cooperation point extraction unit is
The processing framework linkage device according to Appendix 2 or 3, wherein the processing framework in which the processing included in the service definition operates is obtained, and the linkage point between the processing frameworks is extracted.

(Appendix 5) The cooperation system extraction unit is
The processing framework linkage device according to any one of Supplementary note 2 to 4, wherein the queuing system that can be shared by the queuing system at both ends of the extracted linkage point is extracted.

(Appendix 6) The delay measuring unit is
Described in any one of Supplementary Provisions 2 to 5, wherein the linked points are connected by the extracted queuing system, sample data is actually transferred between the processing frameworks, and the transfer delay is measured. Processing framework cooperation device.

(Appendix 7) The selection unit is
If the non-in-memory type transfer delay is not within a predetermined range with respect to the queuing system having the shortest transfer delay among the plurality of queuing systems, the in-memory type has the shortest transfer speed. The processing framework linkage device according to any one of Supplementary note 2 to 6, wherein a queuing system is selected.

(Appendix 8) The processing framework linkage according to any one of Supplementary note 2 to 7, wherein the linkage system insertion unit outputs the service definition modified in response to the insertion of the linkage process. Device.

(Appendix 9) The control unit is
When the frequency of occurrence of the data handled by the service is higher than the predetermined frequency, the throughput of the plurality of the queuing systems when the data is continuously transferred is measured, and the maximum throughput of the plurality of the queuing systems is measured. On the other hand, if the throughput of the non-in-memory type is within a predetermined range, one of the appendices 1 to 8 is characterized in that the one having the maximum throughput is selected from the non-in-memory type. Described processing framework linkage device.

(Appendix 10) The control unit is
When the size of the data handled by the service is larger than the predetermined size, the predetermined data size used in the service is transferred in advance, and the transfer delay of each of the queuing systems is measured. The processing framework linkage device according to any one of 9.

(Appendix 11) The control unit is
When a plurality of the cooperation points exist in the service, the queuing system is selected from the cooperation point having the shortest transfer delay, and the other cooperation points are selected if the selected queuing system is available. The processing framework linkage device according to any one of Supplementary note 1 to 10, wherein the same queuing system that has already been completed is used.

(Appendix 12) The control unit is
The processing framework linkage device according to any one of Supplementary note 1 to 11, wherein when a plurality of services are executed, the queuing system used in the other services is preferentially selected. ..

(Appendix 13) The control unit is
It is characterized in that the execution delay related to the execution of the entire service is measured, and when the execution delay is sufficiently larger than the transfer delay or the processing delay, the queuing system having the longest transfer delay is preferentially selected. The processing framework linkage device according to any one of Supplementary note 1 to 12.

(Appendix 14) In the processing framework cooperation method of connecting the cooperation points between the processing frameworks having the processing included in the service definition of the requested service by the queuing system.
The processing delay of queuing for each of the in-memory type queuing system in which the queue is arranged on the memory and the non-in-memory type queuing system in which the queue is arranged on the disk, and the transfer delay for data transfer between the processing frameworks. Select the in-memory type or the non-in-memory type as the queuing system based on the above.
A processing framework linkage method characterized in that processing is executed by a computer.

(Appendix 15) In a processing framework linkage program that connects linkage points between processing frameworks that have processing included in the service definition of the requested service with a queuing system.
The processing delay of queuing for each of the in-memory type queuing system in which the queue is arranged on the memory and the non-in-memory type queuing system in which the queue is arranged on the disk, and the transfer delay for data transfer between the processing frameworks. Select the in-memory type or the non-in-memory type as the queuing system based on the above.
A processing framework linkage program characterized by having a computer execute processing.

100 Processing framework linkage device 201 Linkage point extraction unit 202 Filter database (DB)
203 Linked system extraction unit 204 Framework database (DB)
205 Delay measurement unit 206 Coordination system selection unit 207 Coordination system insertion unit 301 CPU
302 Memory 305 Recording medium 310 Network 401, 402, 413 Filter 600 Transfer delay measurement result 701,702 Linkage processing (input / output processing)
Dmin Shortest transfer delay FW (FW-A, FW-B) Processing framework P Linkage point Q Queuing system Ta Processing delay Tb Transfer delay Tc Execution delay

Claims (14)

In the processing framework linkage device that connects the linkage points between the processing frameworks that have the processing included in the service definition of the requested service with the queuing system.
The processing delay of queuing for each of the in-memory type queuing system in which the queue is arranged on the memory and the non-in-memory type queuing system in which the queue is arranged on the disk, and the transfer delay for data transfer between the processing frameworks. A control unit that selects the in-memory type or the non-in-memory type as the queuing system based on
A processing framework cooperation device characterized by being equipped with. The control unit
A linkage point extraction unit that extracts the linkage points between the processing frameworks from the service definition,
A linkage system extraction unit that extracts a plurality of available queuing systems for each of the extracted linkage points.
A delay measurement unit that connects the cooperation points with the extracted queuing system and measures the data transfer delay between the processing frameworks.
A selection unit that selects the in-memory type or the non-in-memory type as the queuing system to be used based on the measured transfer delay.
A linkage system insertion unit that inserts a linkage process that connects the linkage points in the selected queuing system, and a linkage system insertion unit.
The processing framework linkage device according to claim 1, wherein the processing framework linkage device is provided. The selection unit is
If the non-in-memory type transfer delay is within a predetermined range with respect to the queuing system having the shortest transfer delay among the plurality of queuing systems, the shortest transfer delay among the non-in-memory types is said. The processing framework linkage device according to claim 2, wherein a queuing system is selected. The cooperation point extraction unit
The processing framework cooperation device according to claim 2 or 3, wherein the processing framework in which the processing included in the service definition operates is obtained, and the cooperation point between the processing frameworks is extracted. The cooperation system extraction unit
The processing framework linkage device according to any one of claims 2 to 4, wherein the queuing system that can be shared by the queuing system at both ends of the extracted linkage point is extracted. The delay measuring unit is
One of claims 2 to 5, wherein the linked points are connected by the extracted queuing system, sample data is actually transferred between the processing frameworks, and the transfer delay is measured. The processing framework linkage device described. The selection unit is
If the non-in-memory type transfer delay is not within a predetermined range with respect to the queuing system having the shortest transfer delay among the plurality of queuing systems, the in-memory type has the shortest transfer speed. The processing framework linkage device according to any one of claims 2 to 6, wherein a queuing system is selected. The control unit
When the frequency of occurrence of the data handled by the service is higher than the predetermined frequency, the throughput of the plurality of the queuing systems when the data is continuously transferred is measured, and the maximum throughput of the plurality of the queuing systems is measured. Any one of claims 1 to 7, characterized in that, if the non-in-memory type has the throughput within a predetermined range, the non-in-memory type having the maximum throughput is selected. The processing framework linkage device described in. The control unit
Claim 1 is characterized in that when the size of the data handled by the service is larger than the predetermined size, the predetermined data size used in the service is transferred in advance and the transfer delay of each of the queuing systems is measured. The processing framework linkage device according to any one of 8 to 8. The control unit
When a plurality of the cooperation points exist in the service, the queuing system is selected from the cooperation point having the shortest transfer delay, and the other cooperation points are selected if the selected queuing system is available. The processing framework linkage device according to any one of claims 1 to 9, wherein the same queuing system that has already been completed is used. The control unit
The processing framework linkage according to any one of claims 1 to 10, wherein when a plurality of services are executed, the queuing system used by the other services is preferentially selected. Device. The control unit
It is characterized in that the execution delay related to the execution of the entire service is measured, and when the execution delay is sufficiently larger than the transfer delay or the processing delay, the queuing system having the longest transfer delay is preferentially selected. The processing framework linkage device according to any one of claims 1 to 11. In the processing framework linkage method that connects the linkage points between processing frameworks that have the processing included in the service definition of the requested service with the queuing system.
The processing delay of queuing for each of the in-memory type queuing system in which the queue is arranged on the memory and the non-in-memory type queuing system in which the queue is arranged on the disk, and the transfer delay for data transfer between the processing frameworks. Select the in-memory type or the non-in-memory type as the queuing system based on the above.
A processing framework linkage method characterized in that processing is executed by a computer. In the processing framework linkage program that connects the linkage points between the processing frameworks that have the processing included in the service definition of the requested service with the queuing system.
The processing delay of queuing for each of the in-memory type queuing system in which the queue is arranged on the memory and the non-in-memory type queuing system in which the queue is arranged on the disk, and the transfer delay for data transfer between the processing frameworks. Select the in-memory type or the non-in-memory type as the queuing system based on the above.
A processing framework linkage program characterized by having a computer execute processing.

Images