JP2020071729A - Processing framework cooperation device, processing framework cooperation method and processing framework cooperation program - Google Patents

Abstract

To improve a throughput upon processing framework cooperation.SOLUTION: A processing framework cooperation device connects a cooperation point P between processing frameworks FW-A and FW-B having filters 402 and 413, respectively, included in a service definition S of a required service by using a queuing system Q. On the basis of processing delays Ta and transfer delays Tb measured upon actual data transfer of the queuing systems Q of an in-memory type and a non-in-memory type, if the transfer delay Tb of Queue 5 of the non-in-memory type to the shortest transfer delay Tb of Queue 1 falls under a predetermined range, the device selects the Queue 5 as the queuing system Q.SELECTED DRAWING: Figure 6

Description

  The present invention relates to a processing framework cooperation device, a processing framework cooperation method, and a processing framework cooperation program for coordinating a plurality of processing frameworks.

  In recent IoT (Internet of Things), more diverse IoT services can be realized by utilizing not only sensor data but also media such as video and audio of a camera. There are many processing frameworks for developing such an IoT service easily and in a short time and executing it in real time with low delay. For example, users of these processing frameworks can develop a service using a plurality of processing frameworks by cooperating with each other using a queuing system.

  As a conventional technique, there is a technique of saving a message from a high-speed volatile storage area to a low-speed non-volatile storage area when the system is stopped to improve processing speed and prevent message loss when the system is stopped (for example, the following patents: Reference 1.). In addition, there is a technology to prevent a delay in processing of the queue by selecting either a high-speed small-capacity queue or a low-speed large-capacity queue based on the free space of the high-speed small-capacity queue, the message retention speed, and the priority of the message. (For example, refer to the following patent document 2.).

JP, 2007-226385, A JP, 2006-277483, A

  However, in the related art, it has been impossible to execute a service in which a plurality of processing frameworks are linked in real time with low delay. A configuration in which the queue is arranged in a volatile area such as a memory (in memory, in-memory type) as in the related art 2 and a configuration in which the queue is arranged in a non-volatile area such as a disk (not in memory, called a non-in-memory type) By combining the above, the characteristics of high-speed small capacity and low-speed large capacity can be utilized. However, for example, when a high-speed in-memory type is selected for the queuing system, although the service can be executed in real time, a shortage of memory occurs due to the simultaneous use of a large number of services, and the service throughput decreases.

  Here, the related art does not consider any delay other than the processing delay associated with the processing of the processing framework. For this reason, when using multiple queuing systems to cooperate with each other, it is not possible to select an appropriate queuing system in response to other delays such as transfer delays. The linked service could not be executed in real time with low latency, and the throughput of the entire process could not be improved. For example, even if the processing delay is small in the in-memory type, when the distance between the processing frameworks is large, the transfer delay becomes large and approaches the non-in-memory type transfer delay.

  In one aspect, it is an object of the present invention to improve throughput when linking processing frameworks.

  According to one aspect of the present invention, the processing framework cooperation device is a processing framework cooperation device for connecting a cooperation point between processing frameworks having a process included in a requested service definition with a queuing system. Based on a queuing processing delay of each of the in-memory type queuing system in which a queue is arranged and a non-in-memory type queuing system in which a queue is arranged on a disk, and a transfer delay required for data transfer between the processing frameworks. , A control unit for selecting the in-memory type or the non-in-memory type as the queuing system.

  According to one aspect of the present invention, it is possible to improve the throughput when the processing frameworks are linked.

FIG. 1 is an explanatory diagram of processing framework cooperation processing performed by the processing framework cooperation apparatus according to the embodiment. FIG. 2 is a block diagram showing the functions of the processing framework cooperation device according to the embodiment. FIG. 3 is a diagram illustrating a hardware configuration example of the processing framework cooperation device according to the embodiment. FIG. 4 is a diagram illustrating a process of the cooperation point extraction unit of the processing framework cooperation device according to the embodiment. FIG. 5 is a chart for explaining the processing of the cooperation system extraction unit of the processing framework cooperation apparatus according to the embodiment. FIG. 6 is a diagram illustrating processing of the delay measuring unit and the cooperation system selecting unit of the processing framework cooperation apparatus according to the embodiment. FIG. 7 is a diagram illustrating a process of the cooperation system insertion unit of the processing framework cooperation device according to the embodiment. FIG. 8 is a flowchart illustrating a processing example of the processing framework cooperation device according to the embodiment. FIG. 9 is a diagram illustrating another processing example of the processing framework cooperation device according to the embodiment. (Part 1) FIG. 10 is a diagram illustrating another processing example of the processing framework cooperation device according to the embodiment. (Part 2) FIG. 11 is a diagram illustrating another processing example of the processing framework cooperation device according to the embodiment. (Part 3) FIG. 12 is a diagram illustrating another processing example of the processing framework cooperation device according to the embodiment. (Part 4) FIG. 13 is a diagram illustrating another processing example of the processing framework cooperation device according to the embodiment. (Part 5)

  Embodiments of the disclosed processing framework cooperation apparatus, processing framework cooperation method, and processing framework cooperation program will be described in detail below with reference to the drawings.

  FIG. 1 is an explanatory diagram of processing framework cooperation processing performed by the processing framework cooperation apparatus according to the embodiment. For example, there are many processing frameworks for developing various IoT services easily and in a short time and executing them in real time with low delay. Each processing framework provides processing (referred to as a filter) such as acquisition of data and media, processing / detection / analysis, etc. as a library. The processing framework cooperation device can provide various services by combining filters.

  As shown in FIG. 1A, the existing single processing framework FW-A has filters A1 to A3, and FW-B has filters B1 and B2. And the services that can be developed are limited. Therefore, by connecting the different processing frameworks FW-A and FW-B with the queuing system Q, a service in which a plurality of processing frameworks FW-A and FW-B having a filter of a requested service are linked with each other. Can be provided.

  The queuing system Q has an in-memory type and a non-in-memory type. The in-memory type has a configuration in which queues are arranged in a volatile area such as a memory, and has a characteristic that the memory usage amount is large although the speed is high (high throughput, processing delay is small). In the embodiment, the non-in-memory type refers to a configuration in which queues are arranged in a non-volatile area such as a disk, and has a characteristic of low speed (low throughput and large processing delay) although the memory usage is small.

  Here, when a high-speed in-memory type is selected for the queuing system Q, although services can be executed in real time, the simultaneous use of a large number of services causes a shortage of memory and reduces the throughput of services.

  In the embodiment, the processing for linking the processing frameworks FW-A and FW-B is performed by the processing shown in FIG. When the processing delays Ta of the queuing system Q are sufficiently smaller than other delays when the processing frameworks FW-A and FW-B are linked, the inventors use a non-in-memory type. Also paid attention to the fact that there is no significant effect. In the embodiment, if the processing delay Ta of the queuing system Q is sufficiently smaller than other delays, the non-in-memory type queuing system Q is used. Other delays include a transfer delay Tb between the processing frameworks FW-A and FW-B, an execution delay (which will be described later) that is the processing time of the entire service, and the like. For example, even if the processing delay Ta is a small in-memory type, if the distance between the processing frameworks FW-A and FW-B is large, the transfer delay Tb becomes large, and the non-in-memory type transfer delay Tb. Approach.

  Here, when the transfer delay Tb is larger than the processing delay Ta, the degree of influence is small even if the processing delay Ta becomes large. Therefore, by selecting a non-in-memory type with a small memory usage as the queuing system Q, Suppress the occurrence of memory shortage. On the other hand, when the transfer delay Tb is small, the influence of the processing delay is large, and therefore the in-memory type is selected as the queuing system Q to enable high-speed processing.

  First, the processing framework cooperation apparatus obtains the processing framework FW in which each filter included in the service definition S operates. In the example of FIG. 1B, it is assumed that the service definition S is filter 1, filter 2, and filter 3. In this case, in the processing framework cooperation device, the filters 1 and 2 are the processing framework FW-A, and the filter 3 is the processing framework FW-B. In addition, the processing framework cooperation device extracts a cooperation point P at which the processing framework FW switches. At this time, the processing framework cooperation device extracts a queuing system Q (cooperation system) that can be used (shared) by the processing frameworks FW-A and FW-B at both ends of the cooperation point P.

  Here, it is assumed that the queuing system Q (cooperative system) that can be used by the processing frameworks FW-A and FW-B is the queue (Queue1, 4, 5). It is assumed that Queue1 is an in-memory type and Queues 4 and 5 are non-in-memory types. The processing framework cooperation device connects the processing frameworks by the extracted queuing system Q (Queue 1, 4, 5), and actually transfers the data (sample data) to measure the transfer delay Tb. Then, the processing framework cooperation device selects one of Queue 1, 4, or 5 as the optimum queuing system Q using the measurement result of the transfer delay Tb.

  Each queuing system Q (Queue 1, 4, 5) has a predetermined processing delay (corresponding to processing time) Ta from queue reception to queue transmission. Further, after the data is transmitted from the processing framework FW-A (filter 2 (402)) on the data transmitting side, a predetermined period is set until the data is received by the processing framework FW-B (filter 3 (413)) on the data receiving side. It has a transfer delay (corresponding to transfer time) Tb. The transfer delay Tb includes the processing delay Ta of the queuing system Q (Queues 1, 4, 5).

  Then, the processing framework cooperation device selects the queuing system Q to be used according to a predetermined condition. At this time, for example, the shortest queuing system Q is the in-memory type Queue1, and the transfer delay Tb of the non-in-memory type queuing system Q (for example, Queue5) is within a predetermined tolerance range close to Queue1. Suppose

  In this case, the processing framework cooperation apparatus selects the non-in-memory type queuing system Q (Queue5). As described above, the processing framework cooperation apparatus of the embodiment does not select the queuing system Q only by the processing delay Ta, but uses the transfer delay Tb when the actual data transfer is performed and uses the queuing system Q. Select. This makes it possible to select not only a high-speed in-memory type queuing system Q of a large capacity and a small memory usage amount, but also a queuing system Q.

  After that, the processing framework cooperation device performs the cooperation processing (input / output processing) 701 and 702 necessary for using the selected queuing system Q (Queue5) on the processing frameworks FW-A and FW at both ends of the cooperation point P. -Insert in B. The cooperation process 701 is an output process of outputting the data output by the filter 2 (402) of the processing framework FW-A to the Queue 5. The cooperation process 702 is an input process of inputting data input from the Queue 5 and outputting the data to the filter 3 (413) of the processing framework FW-B.

  Then, the processing framework cooperation apparatus outputs the corrected service definition S ′ in which the cooperation processes 701 and 702 are inserted to the original service definition S to the user.

  As described above, in the embodiment, the processing frameworks FW-A and FW-B in which the filter corresponding to the service definition S operates are extracted and linked. At this time, the transfer delay Tb between the processing frameworks FW-A and FW-B is actually measured. Further, the cooperative processes 701 and 702 necessary for using the selected queuing system Q are inserted into the processing frameworks FW-A and FW-B at both ends of the cooperative point P, respectively.

  As a result, the optimum queuing system Q for connecting the processing frameworks can be selected, and not only the in-memory type queuing system Q but also the non-in-memory type queuing system Q can be selected. For example, when the non-in-memory type queuing system Q is selected, it is possible to avoid a memory shortage when using a large number of services at the same time, and as a result, it is possible to achieve a low delay and an improvement in throughput.

  Further, the filters 1 and 2 which operate in different processing frameworks FW-A and the filter 3 which operates in FW-B can be connected by the queuing system Q, and by the cooperation of a plurality of processing frameworks FW-A and FW-B. Can flexibly respond to required services. As a result, various required services can be developed and provided.

  FIG. 2 is a block diagram showing the functions of the processing framework cooperation device according to the embodiment. The processing framework cooperation device 100 includes a cooperation point extraction unit 201, a filter database (DB) 202, a cooperation system extraction unit 203, a framework DB 204, a delay measurement unit 205, a cooperation system selection unit 206, and a cooperation system insertion unit 207. ..

  The cooperation point extraction unit 201 extracts from the filter DB 202 a plurality of processing frameworks that can use the filters included in the service definition S requested by the user, and extracts cooperation points at which the processing frameworks are switched. The filter DB 202 stores and holds the processing framework for each filter.

  The cooperation system extraction unit 203 refers to the framework DB 204 and extracts a queuing system Q that can be used (shared) by the processing frameworks at both ends of the cooperation point as a cooperation system. The framework DB 204 stores and holds the corresponding queuing system Q for each processing framework by in-memory type and non-in-memory type.

  The information held by the filter DB 202 and the framework DB 204 is information unique to each framework FW and can be stored and held in advance.

  The delay measuring unit 205 connects the processing frameworks by the queuing system Q extracted by the cooperation system extracting unit 203, and measures the transfer delay when data is transferred. The delay measuring unit 205 actually measures the delay characteristic using appropriate sample data. The cooperation system selection unit 206 selects the optimum queuing system Q based on a predetermined determination condition using the measured transfer delay. The cooperation system selection unit 206 selects an in-memory type or non-in-memory type queuing system Q according to a predetermined determination condition.

  The cooperation system insertion unit 207 inserts the cooperation process required for using the queuing system Q selected by the cooperation system selection unit 206 into the processing frameworks at both ends, and a plurality of processing frameworks can execute them in cooperation. The information of the service definition S'is output.

  FIG. 3 is a diagram illustrating a hardware configuration example of the processing framework cooperation device according to the embodiment. The processing framework cooperation apparatus 100 shown in FIG. 2 can use the hardware shown in FIG. 3, and for example, can use a server.

  For example, the processing framework cooperation device 100 includes a CPU (Central Processing Unit) 301, a memory 302, a network interface (I / F) 303, a recording medium IF 304, and a recording medium 305. A bus 300 connects each unit.

  The CPU 301 is an arithmetic processing device that functions as a control unit that controls the overall control of the processing framework cooperation device 100. The memory 302 includes a non-volatile memory and a volatile memory. The non-volatile memory is, for example, a ROM (Read Only Memory) that stores a program of the CPU 301. The volatile memory is, for example, a DRAM (Dynamic Random Access Memory) or an SRAM (Static Random Access Memory) used as a work area of the CPU 301.

  The network I / F 303 can access a processing framework outside the processing framework cooperation device 100, such as a server or a cloud, through the network 310 and use the external processing framework. The network IF 303 is communicatively connected to an external server, a cloud, or the like via a network 310 such as a LAN (Local Area Network), a WAN (Wide Area Network), or the Internet.

  The recording medium IF 304 is an interface for reading / writing information processed by the CPU 301 from / to the recording medium 305. The recording medium 305 is a recording device that assists the memory 302. For example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), a USB (Universal Serial Bus) flash drive, or the like can be used.

  When the CPU 301 executes the program recorded in the memory 302 or the recording medium 305, each function of the processing framework cooperation device 100 shown in FIG. 2 is realized. Further, the memory 302 and the recording medium 305 can hold in advance the information of the filter DB 202 and the framework DB 204 shown in FIG. Then, the CPU 301 can easily read the information on the framework FW in which the filter operates corresponding to the service and the information on the queuing system for each framework FW.

  In the embodiment, for example, the in-memory type queuing system Q arranges queues on the memory 302. In addition, in the non-in-memory type queuing system Q, a queue is arranged on a recording medium 305 such as a disk.

  Next, processing performed by the processing framework cooperation device according to the embodiment will be described in order of functions.

  FIG. 4 is a diagram illustrating a process of the cooperation point extraction unit of the processing framework cooperation device according to the embodiment. The processing performed by the cooperation point extraction unit 201 of the processing framework cooperation apparatus 100 will be described. FIG. 4A is a diagram showing a filter corresponding to a predetermined service definition S, and FIG. 4B is a diagram showing information on a processing framework for each filter held by the filter DB 202.

  As shown in FIG. 4A, it is assumed that the service definition S is filter 1 (401) to filter 2 (402) to filter 3 (413). In this case, the cooperation point extraction unit 201 refers to the filter DB 202 shown in FIG. 4B and can use the corresponding filter 1 (401), filter 2 (402), and filter 3 (413) FW Ask for.

  In the example illustrated in FIG. 4B, the filters 1 and 2 (401 and 402) are the processing framework FW-A, and the filter 3 (413) is the processing framework FW-B in the filter DB 202. Is held. In this case, the cooperation point extraction unit 201 determines that the processing framework FW of the cooperation point P at which the processing framework FW is switched is the processing framework FW-A (filter 2) and the FW-B (filter 3) part. Extract.

  FIG. 5 is a chart for explaining the processing of the cooperation system extraction unit of the processing framework cooperation apparatus according to the embodiment. The cooperation system extraction unit 203 of the processing framework cooperation apparatus 100 refers to the queuing system Q for each processing framework stored in the framework DB 204. Then, a queuing system Q (cooperative system) that can be used (shared) by the processing frameworks FW at both ends of the cooperation point P is extracted.

  In the example shown in FIG. 5, in the framework DB 204, as the queuing system Q corresponding to the processing framework FW-A, in-memory types are Queue1 and Queue2, and non-in-memory types are Queue4 and Queue5. Information is stored. Further, as the queuing system Q corresponding to the processing framework FW-B, the in-memory type is Queue1, and the non-in-memory type is Queue3, Queue4, and Queue5.

  In this case, the cooperation system extraction unit 203 extracts Queue1 as an in-memory type as a common (sharable) queue between two different processing frameworks FW-A and FW-B, and a non-in-memory type. Extracts Queue4 and Queue5. Here, in the cooperation system extraction unit 203, the in-memory type Queue1 and the non-in-memory type Queues 4 and 5 have processing frameworks FW-A and FW-B at both ends (filter 2 and filter 3) of the cooperation point P. It is judged as an available queuing system Q and extracted.

  The cooperation system extraction unit 203 does not extract the in-memory type Queue2 because it exists only in one processing framework FW-A and cannot be shared because it is not in FW-B. Similarly, the non-in-memory type Queue3 exists only in one of the processing frameworks FW-B and cannot be shared because it is not in FW-A and is not extracted.

  FIG. 6 is a diagram illustrating processing of the delay measuring unit and the cooperation system selecting unit of the processing framework cooperation apparatus according to the embodiment. The delay measuring unit 205 of the processing framework cooperation apparatus 100 connects the processing frameworks by the queuing system Q extracted by the cooperation system extracting unit 203, and measures the transfer delay when the data (sample data) is transferred. .. Further, the cooperation system selection unit 206 selects the optimum queuing system Q using the measurement result of the transfer delay by the delay measurement unit 205.

  FIG. 6A shows an in-memory type Queue 1, which is the queuing system Q extracted by the cooperation system extraction unit 203, and non-in-memory type Queues 4, 5. The in-memory type Queue 1 and the non-in-memory type Queues 4 and 5 are located at both ends of the cooperation point P. In the processing framework FW-A, the filter 2 (402) outputs data to Queues 1, 4, and 5. The Queues 1, 4, and 5 input data to the filter 3 (413) of the processing framework FW-B.

  Here, each queuing system Q (Queue 1, 4, 5) has a predetermined processing delay (corresponding to processing time) Ta from queue reception to queue transmission. Further, after the data is transmitted from the processing framework FW-A (filter 2 (402)) on the data transmitting side, a predetermined period is set until the data is received by the processing framework FW-B (filter 3 (413)) on the data receiving side. It has a transfer delay (corresponding to transfer time) Tb. The transfer delay Tb includes the processing delay Ta of the queuing system Q (Queues 1, 4, 5).

  The processing delay Ta differs for each queuing system Q. In general, the in-memory type (Queue1) has a shorter processing delay Ta than the non-in-memory type (Queues 4, 5). Although the transfer delay Tb includes the processing delay Ta, it is affected by the actual data transfer state, so that the in-memory type is not always short, and the non-in-memory type may be short. Therefore, the delay measuring unit 205 actually transfers sample data for each of the queuing system Q extracted by the cooperation system extracting unit 203, that is, the in-memory type Queue1 and the non-in-memory type Queues 4 and 5. Then, the transfer delay Tb is measured. As the sample data, for example, a random binary data string can be used.

  FIG. 6B is a chart showing a transfer delay measurement result 600 of the transfer delay Tb of the in-memory type Queue1 which is the queuing system Q and the non-in-memory type Queues 4 and 5 extracted by the cooperation system extracting unit 203. is there. The processing delay Ta is a fixed characteristic value (reference value).

  The delay measuring unit 205 connects the processing frameworks in each queuing system Q (Queue 1, 4, 5), transfers predetermined sample data, and measures the transfer delay Tb. The transfer delay Tb is a period from when the filter 402 of the processing framework FW-A in the previous stage transmits data to when the filter 413 of the processing framework FW-B in the subsequent stage receives the data. The delay measuring unit 205 stores and holds the transfer delay measurement result 600 of the transfer delay Tb for each queuing system Q (Queue 1, 4, 5) in, for example, the memory 302 or the recording medium 305. By storing and holding the transfer delay Tb for each queuing system Q, the transfer delay Tb at the time of connection of the similar processing framework FW can be read from the memory 302, the recording medium 305, or the like without using it. ..

Then, the cooperation system selection unit 206 uses the following condition 1. ~ 3. Therefore, the queuing system Q to be used is selected.
1. The queuing system Q with the shortest transfer delay Tb is set as the shortest transfer delay Dmin.
2. If there is a non-in-memory type queuing system Q having a transfer delay Tb of Dmin × N or less, the one having the shortest transfer delay Tb is selected. N is the tolerance from the shortest (constant)
3. If there is no non-in-memory type queuing system Q having a transfer delay Tb of Dmin × N or less, the in-memory type queuing system Q having the shortest transfer delay Tb is selected.

  For example, in the transfer delay measurement result 600 of FIG. 6B, the shortest transfer delay Tb (Dmin) is the in-memory type Queue1 (700 msec). It is assumed that the next shortest is the non-in-memory type Queue 5 (900 msec), and the last is the non-in-memory type Queue 4 (1200 msec).

  The cooperation system selection unit 206 refers to the transfer delay measurement result 600 of FIG. 6B, and when N = 1.3 is set, selects Queue 5 as the queuing system Q. In this case, (Transfer delay Tb of Queue 5 = 900 msec) <(Transfer delay Tb of Queue 1 = 700 × 1.3 = 910).

  Further, when N = 1.2 is set, the cooperation system selection unit 206 selects Queue1 as the queuing system Q. In this case, (Queue 5 transfer delay Tb = 900 msec)> (Queue 1 transfer delay Tb = 700 × 1.2 = 840).

  N is a constant indicating how much delay may be given to the shortest (best effort), and is arbitrarily set by the user.

  In the transfer delay measurement result 600 of FIG. 6B, the transfer delay Tb (900 msec) of the non-in-memory type Queue 5 is close to the transfer delay Tb (700 msec) of the in-memory type Queue 1. In the transfer using Queue 5, the transfer delay Tb may be short because the transfer distance from data transmission to data reception is short, for example. As described above, in the embodiment, the optimum queuing system Q is selected using the transfer delay Tb when data is actually transferred between the processing frameworks FW.

  FIG. 7 is a diagram illustrating a process of the cooperation system insertion unit of the processing framework cooperation device according to the embodiment. The cooperation system insertion unit 207 of the processing framework cooperation apparatus 100 performs a process of connecting the processing frameworks with the optimum queuing system Q selected by the cooperation system selection unit 206.

  FIG. 7A shows a selected state of the queuing system Q selected by the cooperation system selection unit 206 in correspondence with the service definition. In the example of FIG. 7A, an optimum Queue 5 as the queuing system Q is arranged at the cooperation point P between the processing frameworks FW-A (filters 1 and 2) and FW-B (filter 3).

  Then, the cooperation system insertion unit 207 inserts the cooperation processing required for using the selected queuing system Q (Queue 5) into the processing frameworks FW-A and FW-B at both ends of the cooperation point P, respectively.

  In the example of FIG. 7B, the cooperation system insertion unit 207 inserts the FW-A cooperation process 701 in the subsequent stage of the filter 1 (401) and the filter 2 (402) of the processing framework FW-A. Further, the cooperation system inserting unit 207 inserts the FW-B cooperation process 702 in the preceding stage of the filter 3 (413) of the processing framework FW-B. The cooperation process 701 is a process of outputting the data output by the filter 2 (402) of the processing framework FW-A to Queue 5. The cooperation process 702 is a process of outputting the data input from the Queue 5 to the filter 3 (413) of the processing framework FW-B.

  As described above, the cooperation system insertion unit 207 outputs the corrected service definition S ′ obtained by inserting the cooperation processes 701 and 702 to the original service definition S shown in FIG. 7A to the user. As a result, it is possible to output to the user the information on the service that can be executed by the plurality of processing frameworks FW-A and FW-B in cooperation with each other (corrected service definition S ′).

  The corrected service definition S'is linked through the queuing system Q having the shortest transfer delay among the processing frameworks FW-A and FW-B. Thereby, according to the embodiment, it is possible to improve the throughput of a predetermined service in which a plurality of processing frameworks FW-A and FW-B cooperate with each other. In the above description, the case where there is one cooperation point P has been described as an example. However, even when there are a plurality of cooperation points P, an optimum queuing system Q that minimizes the transfer delay for each cooperation point P is provided. Just select it.

  FIG. 8 is a flowchart illustrating a processing example of the processing framework cooperation device according to the embodiment. The processing of each functional unit (see FIG. 2) executed by the CPU 301 of the processing framework cooperation device 100 will be sequentially described.

  First, the processing framework cooperation device 100 (cooperation point extraction unit 201) refers to the filter DB 202 and obtains the processing framework FW in which each filter of the requested service definition S operates (step S801). Next, the processing framework cooperation device 100 (cooperation point extraction unit 201) extracts a cooperation point P at which the processing framework FW switches (step S802).

  Next, the processing framework cooperation device 100 executes the following processing for one cooperation point P. First, the processing framework cooperation device 100 (cooperation point extraction unit 201) refers to the framework DB 204 and extracts a queuing system Q (Queue) that can be used by the processing frameworks FW at both ends of the cooperation point P (step S803). ).

  Next, the processing framework cooperation device 100 (delay measurement unit 205) connects the processing frameworks FW by the extracted queuing system Q (Queue) and measures the transfer delay Tb of each (step S804). The transfer delay Tb of each queuing system Q (Queue) when data is actually transferred using the plurality of queuing systems Q extracted in advance in step S803 is measured, and the transfer delay measurement result 600 is stored in the memory 302. It may be stored.

  Next, the processing framework cooperation device 100 (cooperation system selection unit 206) determines the above-mentioned predetermined conditions 1. with respect to the transfer delay Tb of each queuing system Q (Queue). ~ 3. To select the optimum queuing system Q (Queue).

  For example, the processing framework cooperation apparatus 100 (cooperation system selection unit 206) determines whether or not the shortest transfer delay Tb of the non-in-memory type queuing system Q (Queue) ≦ the shortest transfer delay Dmin × N ( Step S805).

  As a result of the determination in step S805, it is assumed that the condition of the shortest transfer delay Tb ≦ the shortest transfer delay Dmin × N of the non-in-memory type queuing system Q (Queue) is satisfied (step S805: Yes). In this case, the processing framework cooperation device 100 (cooperation system selection unit 206) selects the non-in-memory type queuing system Q (Queue) with the shortest transfer delay Tb (step S806).

  On the other hand, as a result of the determination in step S805, it is assumed that the condition of the shortest transfer delay Tb ≦ the shortest transfer delay Dmin × N of the non-in-memory type queuing system Q (Queue) is not satisfied (step S805: No). In this case, the processing framework cooperation device 100 (cooperation system selection unit 206) selects the in-memory type queuing system Q (Queue) with the shortest transfer delay Tb (step S807).

  After that, the processing framework cooperation apparatus 100 (cooperation system insertion unit 207) operates at both ends of the cooperation point P with the processing framework FW at each end, and performs cooperation processing using the selected queuing system Q (Queue). Insert (step S808). The processing framework cooperation apparatus 100 (cooperation system selection unit 206) outputs a service (corrected service definition S ′) that the processing frameworks can execute in cooperation with each other.

  According to the above processing, the transfer delay Tb is obtained by actually transferring the data to the plurality of queuing systems Q that can be used when the processing frameworks FW are connected at the cooperation point P. At this time, if a short transfer delay Tb is obtained even with the non-in-memory type, the non-in-memory type queuing system Q can be selected over the in-memory type. When the non-in-memory type is adopted, the queue can be arranged on the disk (recording medium 305) or the like, and the use amount of the memory 302 can be reduced while preventing a decrease in throughput. Further, even when a large number of services are used at the same time, a large-capacity disk area can be used, and the memory shortage of the memory 302 can be prevented.

  Next, various processing examples other than the above-described processing examples will be described. 9 to 13 are diagrams illustrating other processing examples of the processing framework cooperation device according to the embodiment. In each of these figures, based on the flowchart shown in FIG. 8, only different processes are extracted and described. The control unit (CPU 301) of the processing framework cooperation device 100 is based on the processing shown in FIG. ~ 5. 9 to 13 can be executed in response to the change in the service state shown in FIG.

(1. When the frequency of data handled by the service is high)
When the frequency of data handled by the service is higher than a predetermined frequency of occurrence, the non-in-memory type will not be able to process in time, so the in-memory type, which allows faster processing than the non-in-memory type, is effective. In such a case, the processing framework cooperation device 100 (delay measurement unit 205) measures the throughput when data is continuously transferred, instead of the transfer delay described in the above embodiment.

Then, the processing framework cooperation device 100 (cooperation system selection unit 206) has the following condition a. ~ C. Select the queuing system Q according to.
a. Let Tmax be the throughput of the queuing system Q with the maximum throughput.
b. If there is a non-in-memory type queuing system Q having a throughput of Tmax ÷ N ′ or more, the non-in-memory type queuing system Q having the maximum throughput is selected. N'is the tolerance from the maximum (constant).
c. If there is no non-in-memory type queuing system Q having a throughput of Tmax ÷ N ′ or more, the queuing system Q having the maximum throughput is selected from the in-memory type queuing system Q.

  FIG. 9 is a flowchart showing a processing example in the case where the frequency of data handled by the service is high. In FIG. 9, the processes of steps S804 to S807 of FIG. 8 are replaced. After the processing of step S803 in FIG. 8, the processing framework cooperation apparatus 100 (delay measurement unit 205) connects the processing frameworks FW by the queuing system Q (Queue) extracted. Then, the throughput of each queuing system Q is measured (step S804a).

  Next, the processing framework cooperation apparatus 100 (cooperation system selection unit 206) determines whether or not the maximum throughput of the non-in-memory type queuing system Q (Queue) ≧ maximum throughput / N ′ (step S805a).

  As a result of the determination in step S805, it is assumed that the condition of maximum throughput ≧ maximum throughput / N ′ of the non-in-memory type queuing system Q (Queue) is satisfied (step S805a: Yes). In this case, the processing framework cooperation device 100 (cooperation system selection unit 206) selects the non-in-memory type queuing system Q (Queue) having the maximum throughput (step S806a).

  On the other hand, as a result of the determination in step S805a, it is assumed that the condition of maximum throughput ≧ maximum throughput / N ′ of the non-in-memory type queuing system Q (Queue) is not satisfied (step S805a: No). In this case, the processing framework cooperation apparatus 100 (cooperation system selection unit 206) selects the in-memory type queuing system Q (Queue) having the highest throughput (step S807a). After this, the process proceeds to step S808 in FIG.

  In this way, when the frequency of data handled by the service is high, the throughput can be improved by preferentially using the in-memory type, which enables high-speed processing, over the non-in-memory type. ..

(2. When the data size handled by the service is large)
When the data size handled by the service is larger than a predetermined data size, the in-memory type uses a large amount of the memory 302, and thus the non-in-memory type that can take a large data area is effective. In such a case, the processing framework cooperation device 100 (delay measurement unit 205) measures the transfer delay Tb using sample data having a large data size corresponding to the designated data size used in the service.

  FIG. 10 is a flowchart showing a processing example when the data size handled by the service is large. In FIG. 10, the process of step S804 of FIG. 8 is replaced. After the processing of step S803 in FIG. 8, the processing framework cooperation apparatus 100 (delay measurement unit 205) connects the processing frameworks FW by the queuing system Q (Queue) extracted. Then, the transfer delay Tb of each queuing system Q is measured using the designated data size (step S804b). After this, the process proceeds to step S805 in FIG.

  Thus, when the data size handled by the service is large, the transfer delay Tb is measured using a large data size corresponding to the designated data size used by the service. Thus, for example, when transferring a large data size, the non-in-memory type that can secure the memory capacity is preferentially used over the in-memory type, so that the throughput of data transfer can be improved.

(3. When there are multiple cooperation points within the service)
In the above embodiment, the case where there is one cooperation point P has been described, but below, a processing example when there are a plurality of cooperation points P in the service will be described. When there are a plurality of cooperation points P in the service, the cooperation point P having a large transfer delay Tb affects the processing delay of the entire service (becomes a bottleneck). Further, the memory usage increases as the number of queuing systems Q used increases. Therefore, in this processing example, the queuing system Q is selected from the cooperation point P with the shortest transfer delay Dmin (the shortest transfer delay is large). Further, the same system is used as the cooperation points that can use the selected queuing system Q.

  FIG. 11 is a flowchart showing a processing example when there are a plurality of cooperation points in the service. As shown in FIG. 11, after performing the processing performed for each cooperation point by the processing of steps S803 and S804 of FIG. 8, the processing is sequentially performed from the cooperation point P having the shortest transfer delay Tb.

  After the processing of step S804 in FIG. 8, the processing framework cooperation device 100 (cooperation system selection unit 206) determines whether or not there is a selected and available queuing system Q (Queue) (step S1101). As a result of the determination, if there is a selected and available queuing system Q (step S1101: Yes), the delay measuring unit 205 causes the non-in-memory type queuing system Q to have the shortest transfer delay Tb ≦ the shortest transfer delay Dmin. It is determined whether it is × N (step S805).

  As a result of the determination in step S805, it is assumed that the condition of the shortest transfer delay Tb ≦ the shortest transfer delay Dmin × N of the non-in-memory type queuing system Q (Queue) is satisfied (step S805: Yes). In this case, the processing framework cooperation device 100 (cooperation system selection unit 206) selects the non-in-memory type queuing system Q (Queue) with the shortest transfer delay Tb (step S806).

  On the other hand, as a result of the determination in step S805, it is assumed that the condition of the shortest transfer delay Tb ≦ the shortest transfer delay Dmin × N of the non-in-memory type queuing system Q (Queue) is not satisfied (step S805: No). In this case, the processing framework cooperation device 100 (cooperation system selection unit 206) selects the in-memory type queuing system Q (Queue) with the shortest transfer delay Tb (step S807).

  If the result of determination in step S1101 is that there is no selected and available queuing system Q (step S1101: No), the processing framework cooperation device 100 (cooperation system selection unit 206) executes step S1102. In step S1102, the processing framework cooperation apparatus 100 (cooperation system selection unit 206) selects the one with the shortest transfer delay Tb among the selected and available queuing systems Q (step S1102). Then, after the execution of step S806, step S807, or step S1102, the process proceeds to step S808.

  In this way, when there are a plurality of cooperation points P in the service, among the queuing systems Q that connect a plurality of processing frameworks FW, the queuing system Q is selected in order from the cooperation point P with the longest transfer delay Tb. Select. Then, if there is a selected queuing system Q, the queuing system Q with the shortest delay is selected from the selected and available queuing systems Q. If there is no selected and available queuing system Q, the non-in-memory type is preferentially selected based on the shortest transfer delay Tb as in the processing example described in the above embodiment. As a result, the throughput of the entire system can be improved even when there are a plurality of cooperation points P.

(4. When multiple services are running)
When a plurality of services are executed, the queuing system Q to be used increases and the memory usage increases. In such a case, the queuing system Q already used by another service is preferentially selected. At this time, when the service is affected by the transfer delay Tb of another service, the transfer delay Tb may be measured again and the queuing system Q may be reselected. Further, the transfer delay Tb may be remeasured and the queuing system Q may be reselected based on the priority of the service.

  FIG. 12 is a flowchart showing a processing example when a plurality of services are being executed. After executing the process of step S803 in FIG. 8, if the processing framework cooperation apparatus 100 (cooperation system extraction unit 203) has a queuing system Q already used by another service, the queuing system in use Narrow down to Q (step S1201). After this, the process proceeds to step S804.

  In this way, when a plurality of services are being executed, the queuing system Q already used by another service is preferentially selected. Thus, for example, when the same video is used by a plurality of services, it is possible to reduce the memory usage by using the queuing system Q used by another service when executing a plurality of services. Become.

(5. When service execution delay is sufficiently larger than transfer delay or processing delay)
FIG. 13A is a diagram showing the corrected service definition S ′. When the execution delay Tc, which is the execution time of the entire service, is sufficiently larger than the transfer delay Tb or the processing delay Ta, the effect of the delay due to the queuing system Q is small, so even if the transfer delay Tb is large, there is no effect. In this case, by measuring not only the transfer delay Tb but also the execution time of the entire service (execution delay Tc), if the execution delay Tc is sufficiently larger than the transfer delay Tb, a queuing system Q with a large transfer delay Tb is set. select.

  FIG. 13B is a flowchart showing a processing example when the execution delay of the service is sufficiently larger than the transfer delay or the processing delay. After the process of step S803 in FIG. 8 is executed, the processing framework cooperation device 100 (delay measurement unit 205) connects the processing frameworks FW by the queuing system Q (Queue) that is extracted. Then, the transfer delay Tb and the execution delay Tc of each queuing system Q are measured (step S804c).

  After that, the processing framework cooperation device 100 (delay measurement unit 205) determines whether or not execution delay Tc ≧ shortest transfer delay Tb × M (constant) (step S1301). Here, the queuing system Q with the shortest transfer delay Tb is judged. As a result of the determination, if the execution delay Tc ≧ the shortest transfer delay Dmin × M (constant) (step S1301: Yes), the process proceeds to step S1302. In this case, the processing framework cooperation device 100 (cooperation system selection unit 206) preferentially selects from the queuing system Q with the longest transfer delay Tb (step S1302). On the other hand, it is assumed that execution delay Tc ≧ shortest transfer delay Dmin × M (constant) is not satisfied (step S1301: No). In this case, the processing framework cooperation device 100 (cooperation system selection unit 206) proceeds to the process of step S805.

  In step S805, the delay measuring unit 205 determines whether or not the shortest transfer delay Tb of the non-in-memory type queuing system Q ≦ the shortest transfer delay Dmin × N (step S805). As a result of the determination in step S805, it is assumed that the condition of the shortest transfer delay Tb ≦ the shortest transfer delay Dmin × N of the non-in-memory type queuing system Q (Queue) is satisfied (step S805: Yes). In this case, the processing framework cooperation device 100 (cooperation system selection unit 206) selects the non-in-memory type queuing system Q (Queue) with the shortest transfer delay Tb (step S806).

  On the other hand, as a result of the determination in step S805, it is assumed that the condition of the shortest transfer delay Tb ≦ the shortest transfer delay Dmin × N of the non-in-memory type queuing system Q (Queue) is not satisfied (step S805: No). In this case, the processing framework cooperation device 100 (cooperation system selection unit 206) selects the in-memory type queuing system Q (Queue) with the shortest transfer delay Tb (step S807). Then, after the processes of steps S806, S807, and S1302, the process proceeds to step S808.

  As described above, when the service execution delay Tc is sufficiently larger than the transfer delay Tb or the processing delay Ta, the influence of the delay due to the queuing system Q becomes relatively small, and even if the transfer delay Tb is large, the influence becomes large. Does not happen. Therefore, the execution delay Tc is measured, and when the execution delay Tc is sufficiently larger than the transfer delay Tb, the queuing system Q having the large transfer delay Tb is selected. This allows other services to utilize the low delay queuing system Q.

  According to the embodiment described above, the transfer delay between the processing frameworks is measured when the processing frameworks in which the filters corresponding to the service definitions operate are linked. This makes it possible to select an optimum queuing system for connecting the processing frameworks, and it is possible to select a non-in-memory type queuing system as well as an in-memory type. For example, if the non-in-memory type transfer delay is within a predetermined range with respect to the shortest transfer delay of the plurality of queuing systems, the shortest transfer delay of the non-in-memory type is selected. When a non-in-memory type queuing system is selected, it is possible to avoid a memory shortage when using a large number of services at the same time, and as a result, it is possible to improve throughput with low delay of services.

  In addition, the cooperation processing required to use the selected queuing system is inserted into the processing frameworks at both ends of the cooperation point. As a result, filters in different processing frameworks can be connected by a queuing system, services required by cooperation of multiple processing frameworks can be flexibly handled, and various services can be developed and provided. Become.

  Also, note that if the processing delay of the queuing system is sufficiently smaller than other delays, it will not affect even the non-in-memory type, and use the queuing system to actually transfer data between processing frameworks. The transfer delay when transferring is measured. As a result, by using not only the processing delay of the queuing system itself but also the measurement result of the transfer delay, it becomes possible to improve the throughput with low delay when the actual service is provided.

  Further, the processing framework in which the processing included in the service definition operates is obtained, and the cooperation points between the processing frameworks are extracted. Also, a queuing system that can be shared by the queuing systems at both ends of the extracted cooperation point is extracted. At this time, the processing framework in which the processing included in the service definition operates and the information of the queuing system that can be shared can be held in advance in the database and can be easily obtained by referring to the database. .. In addition, it becomes possible to connect filters in different processing frameworks with a queuing system and to link a plurality of processing frameworks.

  Further, among the plurality of queuing systems, if the non-in-memory type transfer delay is not within a predetermined range with respect to the shortest transfer delay, the in-memory type having the shortest transfer speed is selected. As described above, in the embodiment, the non-in-memory type is preferentially used, but various queuing systems including the in-memory type can be selected.

  Further, by outputting the service definition S as a modified service definition S'corresponding to the cooperation process inserted between the processing frameworks, the processing frameworks can be executed in cooperation with each other.

  Further, when the frequency of data handled by the service is higher than a predetermined frequency, the throughput of a plurality of queuing systems at the time of continuous data transfer may be measured. If the throughput of the non-in-memory type is within a predetermined range with respect to the maximum throughput of the plurality of queuing systems, the maximum throughput of the non-in-memory type is selected. As described above, by selecting the high-speed in-memory type in response to the high frequency of data generation, it is possible to suppress the decrease in the throughput of the entire process.

  Further, when the size of data handled by the service is larger than a predetermined size, a predetermined data size used in the service may be transferred in advance and the transfer delay of each queuing system may be measured. As a result, by selecting the non-in-memory type corresponding to the case where a large data size is used in an actual service, it is possible to prevent the throughput of the entire process from decreasing.

  If there are multiple cooperation points in the service, select the queuing system from the cooperation points with the largest shortest transfer delays, and select the same queuing system for the other cooperation points if the selected queuing system is available. A queuing system may be used. As a result, it is possible to suppress the occurrence of a cooperation point having a large transfer delay when a service having a plurality of cooperation points is executed, and prevent a decrease in the throughput of the entire process.

  Further, when a plurality of services are being executed, the queuing system used by another service may be preferentially selected. For example, when the same video is used for different services, the same queuing system is used for each service. As a result, it is possible to suppress an increase in the queuing system to be used, suppress the memory usage, and prevent a decrease in the throughput of the entire process.

  Alternatively, the execution delay required to execute the entire service may be measured, and if the execution delay is sufficiently larger than the transfer delay or the processing delay, the queuing system with the longest transfer delay may be preferentially selected. In such a case, even if the processing delay or transfer delay by the queuing system is large, the effect of the delay on the entire service is small, so it is possible to select from the queuing system with the longest transfer delay of non-in-memory type. it can. As a result, other services can use the queuing system with low delay (processing delay and transfer delay), and it is possible to prevent a decrease in the throughput of the entire processing when a plurality of services are executed.

  The processing framework cooperation program described in the embodiments of the present invention can be realized by causing a processor such as a server to execute a prepared program. This control method is recorded on a computer-readable recording medium such as a hard disk, a flexible disk, a CD-ROM (Compact Disc-Read Only Memory), and a flash memory, and the computer reads and executes the recording medium. The control method may be distributed via a network such as the Internet.

  Regarding the above-described embodiment, the following supplementary notes are further disclosed.

(Supplementary Note 1) In a processing framework cooperation device for connecting a cooperation point between processing frameworks having processing included in a service definition of a requested service with a queuing system,
A queuing processing delay of each of the in-memory type queuing system in which a queue is arranged on a memory and a non-in-memory type where a queue is arranged on a disk, and a transfer delay required for data transfer between the processing frameworks. A control unit for selecting the in-memory type or the non-in-memory type as the queuing system,
A processing framework cooperating device comprising:

(Supplementary Note 2) The control unit
A cooperation point extraction unit that extracts the cooperation points between the processing frameworks from the service definition;
A cooperation system extraction unit for extracting a plurality of usable queuing systems for each of the extracted cooperation points,
A delay measuring unit that connects the cooperation points in the extracted queuing system and measures a data transfer delay between the processing frameworks,
A selection unit for selecting the in-memory type or the non-in-memory type as a queuing system to be used based on the measured transfer delay;
A cooperation system insertion unit that inserts a cooperation process for connecting the cooperation points in the selected queuing system,
The processing framework cooperation device according to appendix 1, further comprising:

(Supplementary Note 3) The selection unit is
If the transfer delay of the non-in-memory type is within a predetermined range with respect to the queuing system having the shortest transfer delay of the plurality of queuing systems, the shortest transfer delay of the non-in-memory type is The processing framework cooperation device according to appendix 2, wherein a queuing system is selected.

(Supplementary Note 4) The cooperation point extraction unit
4. The processing framework cooperation device according to appendix 2 or 3, wherein the processing framework in which the processing included in the service definition operates is obtained, and the cooperation points between the processing frameworks are extracted.

(Supplementary Note 5) The cooperation system extraction unit
The processing framework cooperation device according to any one of appendices 2 to 4, wherein the queuing system that can be shared by the queuing systems at both ends of the extracted cooperation point is extracted.

(Supplementary Note 6) The delay measuring unit
The connection point is connected by the extracted queuing system, sample data is actually transferred between the processing frameworks, and the transfer delay is measured. Processing framework cooperation device.

(Supplementary Note 7) The selection unit is
Among the plurality of queuing systems, if the transfer delay of the non-in-memory type is not within a predetermined range with respect to the queuing system of the shortest transfer delay, the transfer speed of the in-memory type is the shortest. The processing framework cooperation device according to any one of appendices 2 to 6, wherein a queuing system is selected.

(Additional remark 8) The cooperation system inserting unit outputs the service definition modified in response to insertion of the processing of the cooperation, and the processing framework cooperation according to any one of additional remarks 2 to 7. apparatus.

(Supplementary Note 9) The control unit
When the occurrence frequency of the data handled by the service is higher than a predetermined frequency, the throughput of the plurality of queuing systems when the data is continuously transferred is measured, and the maximum throughput of the plurality of queuing systems is measured. On the other hand, if the throughput of the non-in-memory type is within a predetermined range, the maximum throughput of the non-in-memory type is selected. The described processing framework cooperation device.

(Supplementary Note 10) The control unit
When the size of data handled by the service is larger than a predetermined size, a predetermined data size used in the service is transferred in advance, and the transfer delay of each of the queuing systems is measured. 9. The processing framework cooperation device according to any one of 9.

(Supplementary Note 11) The control unit
When there are a plurality of the cooperation points in the service, the queuing system is selected from the cooperation points having the shortest transfer delay and the other cooperation points are selected if the selected queuing system is available. The processing framework cooperation device according to any one of appendices 1 to 10, wherein the same queuing system is used.

(Supplementary Note 12) The control unit
When a plurality of services are executed, the queuing system used in another service is preferentially selected, and the processing framework cooperation device according to any one of appendices 1 to 11. ..

(Supplementary Note 13) The control unit
The execution delay required to execute the entire service is measured, and if the execution delay is sufficiently larger than the transfer delay or the processing delay, the queuing system having the longest transfer delay is preferentially selected. The processing framework cooperation device according to any one of Supplementary Notes 1 to 12.

(Supplementary Note 14) In the processing framework cooperation method for connecting the cooperation points between the processing frameworks having the processing included in the service definition of the requested service with the queuing system,
A queuing processing delay of each of the in-memory type queuing system in which a queue is arranged on a memory and a non-in-memory type where a queue is arranged on a disk, and a transfer delay required for data transfer between the processing frameworks. Based on, select the in-memory type or the non-in-memory type as the queuing system,
A processing framework cooperation method, wherein a processing is executed by a computer.

(Supplementary Note 15) In the processing framework cooperation program for connecting the cooperation points between the processing frameworks having the processing included in the service definition of the requested service with the queuing system,
A queuing processing delay of each of the in-memory type queuing system in which a queue is arranged on a memory and a non-in-memory type where a queue is arranged on a disk, and a transfer delay required for data transfer between the processing frameworks. Based on, select the in-memory type or the non-in-memory type as the queuing system,
A processing framework cooperation program characterized by causing a computer to execute processing.

100 processing framework cooperation device 201 cooperation point extraction unit 202 filter database (DB)
203 Cooperation System Extraction Unit 204 Framework Database (DB)
205 Delay Measurement Unit 206 Cooperative System Selection Unit 207 Cooperative System Insertion Unit 301 CPU
302 memory 305 recording medium 310 network 401, 402, 413 filter 600 transfer delay measurement result 701, 702 cooperative processing (input / output processing)
Dmin Shortest transfer delay FW (FW-A, FW-B) Processing framework P Cooperation point Q Queuing system Ta Processing delay Tb Transfer delay Tc Execution delay

Claims (14)

In a processing framework cooperation device for connecting a cooperation point between processing frameworks having processing included in the service definition of a requested service with a queuing system,
A queuing processing delay of each of the in-memory type queuing system in which a queue is arranged on a memory and a non-in-memory type where a queue is arranged on a disk, and a transfer delay required for data transfer between the processing frameworks. A control unit for selecting the in-memory type or the non-in-memory type as the queuing system,
A processing framework cooperating device comprising: The control unit is
A cooperation point extraction unit that extracts the cooperation points between the processing frameworks from the service definition;
A cooperation system extraction unit for extracting a plurality of usable queuing systems for each of the extracted cooperation points,
A delay measuring unit that connects the cooperation points in the extracted queuing system and measures a data transfer delay between the processing frameworks,
A selection unit for selecting the in-memory type or the non-in-memory type as a queuing system to be used based on the measured transfer delay;
A cooperation system insertion unit that inserts a cooperation process for connecting the cooperation points in the selected queuing system,
The processing framework cooperation device according to claim 1, further comprising: The selection unit,
If the transfer delay of the non-in-memory type is within a predetermined range with respect to the queuing system having the shortest transfer delay of the plurality of queuing systems, the shortest transfer delay of the non-in-memory type is The processing framework cooperation device according to claim 2, wherein a queuing system is selected. The cooperation point extraction unit,
The processing framework cooperation device according to claim 2 or 3, wherein the processing framework in which the processing included in the service definition operates is obtained, and the cooperation point between the processing frameworks is extracted. The cooperation system extraction unit,
The processing framework cooperation device according to claim 2, wherein the queuing system that can be shared by the queuing systems at both ends of the extracted cooperation point is extracted. The delay measuring unit,
The connection point is connected by the extracted queuing system, sample data is actually transferred between the processing frameworks, and the transfer delay is measured. The described processing framework cooperation device. The selection unit,
Among the plurality of queuing systems, if the transfer delay of the non-in-memory type is not within a predetermined range with respect to the queuing system of the shortest transfer delay, the transfer speed of the in-memory type is the shortest. The processing framework cooperation device according to claim 2, wherein a queuing system is selected. The control unit is
When the occurrence frequency of the data handled by the service is higher than a predetermined frequency, the throughput of the plurality of queuing systems when the data is continuously transferred is measured, and the maximum throughput of the plurality of queuing systems is measured. On the other hand, if the throughput of the non-in-memory type is within a predetermined range, the maximum throughput of the non-in-memory type is selected. The processing framework cooperation device described in. The control unit is
When the size of data handled by the service is larger than a predetermined size, a predetermined data size used by the service is transferred in advance, and the transfer delay of each of the queuing systems is measured. ~ The processing framework cooperation device according to any one of 8 above. The control unit is
When there are a plurality of the cooperation points in the service, the queuing system is selected from the cooperation points having the shortest transfer delay and the other cooperation points are selected if the selected queuing system is available. The processing framework cooperation device according to claim 1, wherein the same queuing system is used. The control unit is
When a plurality of services are being executed, the queuing system used by another service is preferentially selected, and the processing framework cooperation according to any one of claims 1 to 10. apparatus. The control unit is
The execution delay required to execute the entire service is measured, and if the execution delay is sufficiently larger than the transfer delay or the processing delay, the queuing system having the longest transfer delay is preferentially selected. The processing framework cooperation device according to any one of claims 1 to 11. In a processing framework cooperation method for connecting a cooperation point between processing frameworks having processing included in a service definition of a requested service with a queuing system,
A queuing processing delay of each of the in-memory type queuing system in which a queue is arranged on a memory and a non-in-memory type where a queue is arranged on a disk, and a transfer delay required for data transfer between the processing frameworks. Based on, select the in-memory type or the non-in-memory type as the queuing system,
A processing framework cooperation method, wherein a processing is executed by a computer. In the processing framework linkage program that connects the linkage points between the processing frameworks that have the processing included in the service definition of the requested service with the queuing system,
A queuing processing delay of each of the in-memory type queuing system in which a queue is arranged on a memory and a non-in-memory type where a queue is arranged on a disk, and a transfer delay required for data transfer between the processing frameworks. Based on, select the in-memory type or the non-in-memory type as the queuing system,
A processing framework cooperation program characterized by causing a computer to execute processing.

Images