JP4265559B2 - Data processing apparatus, data processing server, data processing system, and data processing program - Google Patents

Description

  The present invention relates to a data processing device and a data processing server that perform processing on data received from a plurality of partners, a data processing system including these partners and a data processing device or a data processing server, and a data processing program thereof.

  In recent years, network devices and network connection services have been increased in speed and price, and many communications using networks have been performed. Along with this, there is a demand for improving the processing capacity of network servers. Examples of the network server include an application server that provides various applications via a network, a transaction server that performs transaction processing as a batch process such as deposit and withdrawal at a financial institution, and a database server. These services are provided by receiving a request from a client connected to a network and performing processing on the request. In such a network server, it is important not only to improve the arithmetic processing capability but also to efficiently perform communication processing such as data reception regarding processing requests from a plurality of clients that can occur simultaneously and data transmission as a result of the processing. .

  In view of this, a method has been proposed in which multiple processes of communication are realized by generating a plurality of tasks (see, for example, Patent Document 1). The proposed computer includes a task generation unit. For example, when data is transmitted to another computer, a new task is generated, and data transmission processing is left to the task. Therefore, in this computer, even before this data transmission process is completed, different processes can be performed in parallel. That is, by generating a plurality of tasks in the network server, a communication multiplex process that simultaneously performs a communication process with a plurality of clients is realized. This makes it possible to receive more processing requests and provide services according to the requests.

  In such multitask processing that performs different processing by multiple tasks, the processing of the CPU (Central Processing Unit) is time-divided and sequentially assigned to multiple processing targets, and each is performed simultaneously. It looks like this. A series of processes divided at this time is called a task or a process. The OS (Operating System) operating in multitasking is actually switched as a series of processes that can process this task or process at the same time, in units of threads divided into processes such as input, calculation, and file operation. It is operating as a multi-thread. These tasks, processes and threads are called contexts. Each context is uniquely assigned to indicate an execution state of processing such as a stack, a register, and an execution counter. As a result, the processing can be performed independently without being influenced by the processing progressing in another context. Further, by increasing the context, the number of objects that can be processed simultaneously in a computer having a plurality of CPUs can be increased, and more processes can be assigned to more CPUs.

FIG. 10 shows how the network server controls a plurality of connections in one context. Before describing processing by a plurality of contexts, first, processing by one context will be described. The first to third connection 101 _{1-101 3} each network server 102 transmits the data 103 ₁ to 103 _3, receives the processing result 104 _{1-104 3} against it after waiting for processing 105 _{1-105 3} as It has become. It will be described in detail for the first connection 101 _1. The first connection 101 ₁ transmits the data 103 ₁ to the network server 102 and receives the processing result 104 ₁ for the data 103 ₁ after the processing wait 105 ₁ . The data 103 ₁ is once buffered 107 ₁ in the buffer memory (buffer) 106 and transferred to the context (C) 109 of the processing means 108. In the context 109, data processing 111 ₁ is performed, and the processing result 104 ₁ is sent back to the first connection 101 ₁ of the request source. In the context 109, if the data 103 ₁ has not arrived from the first connection 101 ₁ , the data waiting 110 ₁ is performed, and processing for other connections is not performed until the data processing 111 ₁ is completed after the arrival. A state exclusively used for processing consisting of data waiting and data processing for one connection is called a block, and such a reception method is called a block reception method. That is, the block 112 ₁ by the first connection is a combination of the data waiting 110 ₁ and the data processing 111 ₁ as shown in FIG. Here, in order to focus on the receiving system, the transmission of the processing result 104 ₁ are described as not through the processing by the OS in order to simplify the description. Similarly, in order to simplify the explanation, the processing of the buffer memory 106 is described with one flow. However, in practice, a plurality of buffering operations can be performed simultaneously by a multitasking OS. The same processing is performed for the second and third connections 101 ₂ and 101 ₃ . The processing means 108 is usually constituted by a program, and such a network server 102 is called a server program.

Here, there is one context 109 for executing the server program, and it is determined in advance that the processing for the _first to third connections 101 _{1 to} 101 ₃ is performed in this order. Accordingly, as shown in this figure, for example, data 103 ₃ from the third connection 101 ₃ are buffered 107 ₃ by the buffer memory 106, also ready to begin processing immediately, the context 109 is determined in advance After the processing for the first connection 101 _{1 in} the order described, the processing for the second connection 101 ₂ is performed. Therefore, when the arrival of the data 103 ₂ from the second connection 101 ₂ is delayed, a third connection 101 ₃ will also delay time obtained by the processing result 104 _3. In other words, due to the influence of other connections, inequality occurs in the waiting time until each connection obtains a processing result. Therefore, a network server that handles a plurality of connections at the same time by the block reception method is realized by generating a plurality of contexts as in the first proposal.

FIG. 11 shows how the network server controls a plurality of connections in a plurality of contexts. In FIG. 11, the same parts as those in FIG. 10 are denoted by the same reference numerals, and description thereof will be omitted as appropriate. Processing means 108A of network server 102A simultaneously in the first to third context 109 _{1-109 3,} perform respective data processing 111 _{1-111 3} for the first to third connection 101 _{1-101 3} separately It comes to let you. Therefore, the processing waiting time is caused by the corresponding connection, and is not affected by other connections. In the block reception method, communication multiple processing can be realized by providing a plurality of contexts.

  In this way, in the block reception method, multiple processing of communication is realized by providing a plurality of contexts, and processing that does not affect other connections is possible. However, in order to perform communication multiplex processing by the block reception method, a plurality of contexts corresponding to the number of connection destinations to be handled are required. Each context is independently influenced by the processing of other contexts. For this reason, a dedicated memory area that is not rewritten by the progress state of another program is required for each context. Therefore, the number of connections that can be handled in terms of resources is limited.

  Therefore, it is usual to use a non-block reception method in the network server. In this method, multiple processing of communication is possible in combination with means for scanning whether received data that can be processed from each connection destination has been buffered in the buffer memory. This means is called a selector and is provided by the OS. The selector is called by the server program of the network server and repeatedly scans until the scan result includes one or more events. When an event occurs, the result is returned to the server program.

FIG. 12 shows a state in which the network server controls a plurality of connections by using one context and a selector by a non-block receiving method. In FIG. 12, the same parts as those in FIG. 10 are denoted by the same reference numerals, and description thereof will be omitted as appropriate. In the network server 102B, the selector 120 scans the buffer memory 106 for data that has been buffered among the data 103 _{1 to} 103 ₃ from the _first to third connections 101 _{1 to} 101 ₃ . . As shown in this figure, in the processing means 108B, scanning 121 _{1 to} 121 ₆ by the selector 120 is performed when the context 109 is ready to perform a new process. Depending on the first scan 121 ₁ of the selector 120, no data is found. Therefore, scanning 121 ₂ is continued. Data 103 ₁ of the first connection 101 ₁ at this stage is found. Based on the scanning result of the selector 120, the processing unit 108B causes the context 109 to receive the data 103 ₁ and perform data processing 111 ₁ thereof. Thereafter, the processing result 104 ₁ is sent back to the first connection 101 ₁ . By scanning 121 ₃ next carried out, the third connection 101 _third data 103 ₃ is found, are processed similarly. The selector 120 repeats the scanning 121 _{4-121 6} until again the data. When the second connection 101 _{and second} data 103 ₂ are found by scanning 121 _6, processing is performed in the same manner again. When a plurality of pieces of data are buffered in the buffer memory 106, the selector 120 may find only the one buffered earliest among them or may find all of them. In the latter case, the processing unit 108B looks at the processing state of the context 109, and causes each process to be performed each time one process is completed. Block reception waits until received data arrives. In this way, the non-block reception method uses a selector to obtain data that can be processed immediately without waiting for reception on a specific connection. Therefore, it is possible to perform processing in a certain short time.
JP 2003-58384 A (paragraph 0023, FIG. 1)

  By the way, in the non-block reception method, although a plurality of connections can be controlled, if the number of connections scanned by the selector becomes a large number such as thousands or tens of thousands, scanning overhead becomes a problem. come. In other words, when viewed from the connection, even if data is transmitted, the data is transferred to the processing means and processed after the data is found by scanning, so that there is a time lag in scanning time. Therefore, as compared with the block method in which processing is performed as soon as data is buffered in the buffer memory by the OS, the time until the processing result is obtained increases. This is not a problem for connections with no communication or a low communication frequency, but the processing time increases with the number of data to be transmitted, so a problem is large in terms of processing efficiency for a connection with a high communication frequency. In addition, when data is found in the buffer memory by scanning, processing for reporting the result is performed. Therefore, if there is even one connection that frequently transmits data, the scanning result must be reported each time. . Therefore, the burden is increased compared to the case where scanning is repeated until data is found only for connections with low communication frequency.

  The network server communication control has been described above as an example. In addition to this, there are many examples in which it is necessary to efficiently receive and process processing requests from these partners and data related to them that can occur at the same time in an apparatus that performs processing for a plurality of partners. In any case, when the non-block reception method is used in order to limit the number of opponents, there is a similar problem with respect to processing efficiency for an opponent who frequently requests processing.

  Accordingly, an object of the present invention is to provide a data processing device, a data processing server, a data processing system, and a data processing method that can reduce a decrease in processing efficiency without limiting the number of counterparts to be processed. .

In the present invention, (a) unit data receiving means for sequentially receiving unit data as data for each processing unit via a communication network, and (b) determining the transmission source of unit data received by the unit data receiving means (C) a reception frequency monitoring unit that monitors the reception frequency of the unit data described above for each transmission source; and (d) temporarily storing unit data received by the unit data reception unit. Data storage means, and (e) processing of any number of the unit data described above that are transmitted intermittently or continuously from the same transmission source in a form that one transmission source occupies for a desired time. When the data waiting for the transmission source occurs, the unit data from the same transmission source is not processed until the data for the transmission source is processed. Motor 1 or a plurality of first data processing means for enabling a high throughput communication when the receiving frequency is high, in the form of proprietary by each source is time division as a unit the unit data mentioned above (to) , one or a plurality of second data processing means for processing the unit data described above, according to (g) the result of the determination to the source discriminating means each time the above-mentioned unit data receiving means receives unit data described above First, the unit data stored in the data storage unit is assigned to the first data processing unit in order from the transmission source having the high reception frequency of the unit data received by the unit data receiving unit, and the first data is transferred. Data transfer means; and (h) a unit stored in the data storage means without being transferred to the first data processing means by the first data transfer means. A data presence / absence determining means for determining the presence / absence of data; and (i) when the determination result of the data presence / absence determining means is data, the unit data is transferred from the data storage means to the second data processing means. The data processing device includes second data transfer means.

In the present invention, (a) unit data receiving means for sequentially receiving unit data as data for each processing unit from the client via the communication network; and (b) unit data received by the unit data receiving means. Client discrimination means for discriminating clients, (c) reception frequency monitoring means for monitoring the reception frequency of the unit data for each client determined by the client discrimination means, and (d) received by the unit data reception means Data storage means for temporarily storing unit data; and (e) any number of the above-mentioned units that are intermittently or continuously sent from the same client in a form that each client occupies for a desired time. Data processing to the client when data waiting for that client occurs One or a plurality of first data processing means for enabling high-throughput communication when the frequency of receiving unit data from the same client is high by not performing processing for other clients until the current data is processed (F) One or a plurality of second data processing means for processing the unit data in a form that each client occupies in a time-sharing manner with the unit data as one unit, and (g) reception of the unit data as described above Each time the unit receives the unit data, the unit data stored in the data storage unit according to the determination result of the client determination unit is changed to the reception frequency of the unit data received by the unit data reception unit. First data transfer means for assigning and transferring to the first data processing means in order from the highest client; Data presence / absence determining means for determining presence / absence of unit data stored in the data storage means not transferred to the first data processing means by the first data transfer means; The data processing server includes a second data transfer means for transferring the unit data from the data storage means to the second data processing means when the determination result of the determination means is data.

The present invention further includes (a) a communication network, (b) a data processing device according to claim 1, and (c) a data transmission means for transmitting data to the data processing device via the communication network. The data processing system includes one or more data processing requesting devices.

Furthermore, according to the present invention, (b) unit data reception processing for sequentially receiving unit data as data for each processing unit via a communication network as a data processing program; A transmission source determination process for determining the transmission source of the unit data received in the process; (c) a reception frequency monitoring process for monitoring the reception frequency of the unit data for each transmission source; and (d) the unit data described above. Data storage processing for temporarily storing unit data received in the reception processing, and (e) each transmission source occupies a desired time and is sent intermittently or continuously from the same transmission source. The processing of an arbitrary number of the above unit data is performed until other data is processed until the data for the transmission source is processed when the data waiting for the transmission source occurs. One or a plurality of first data processing that enables high-throughput communication when the frequency of receiving unit data from the same transmission source is high by not performing processing on the source; Each transmission source as a unit is dedicated in a time-sharing manner, one or a plurality of second data processes for processing the unit data, and (g) each time the unit data is received in the unit data reception process. First, the unit data stored in the data storage process according to the determination result of the transmission source determination process described above in order from the transmission source having the highest reception frequency of the unit data received in the unit data reception process. And (h) the first data transfer process which is not transferred to the first data process described above. Data presence / absence determination processing for determining the presence / absence of unit data as stored in the data storage processing, and (i) unit data stored in the data storage processing described above when the determination result of the data presence / absence determination processing is data And a second data transfer process for transferring the data to the second data process described above.

As described above, according to the present invention, not only can data be received and processed from a plurality of transmission sources, but a processing means that can be used exclusively by the transmission source is assigned to a transmission source that frequently transmits data. It is also possible to process efficiently. In addition, since the frequency of communication with each transmission source can be checked and the assignment can be changed according to the frequency, for example, even if many of the transmission sources are not communicating or have low transmission frequency, data is frequently transmitted. If even one transmission source exists, it is possible to avoid a situation in which the overhead for scanning whether data has been received becomes a problem .

  Hereinafter, the present invention will be described in detail with reference to examples.

FIG. 1 shows an outline of the configuration of a data processing system according to an embodiment of the present invention. The data processing system 200 is comprised of a server 201 and a LAN (Local Area Network) or the first to the client 203 ₁ ~203 _n of the n connected through the communication network 202 such as the Internet. As described above, the server 201 communicates with the _first to n-th clients 203 _{1 to} 203 _n, and in order to establish these communications simultaneously, multiple processing of communication is performed.

The configuration of the server 201 will be described. The server 201 includes first to m-th CPUs (central processing units) 211 _{1 to} 211 _m . These first to m-th CPUs 211 ₁ to 211 _m can perform processing in parallel. Therefore, in the present specification, these will be described simply as “CPU 211”. The CPU 211 is connected to each unit in the communication multiprocessing server 201 through a bus 212 such as a data bus. Of these, a ROM (Read Only Memory) 213 stores nonvolatile data or a control program. A RAM (Random Access Memory) 214 temporarily stores data required when the CPU 211 executes various controls. A hard disk drive (HDD) 215 is a magnetic storage device in which an OS (Operating System) and application programs for controlling the server 201 are stored. The display unit 216 includes a monitor using a liquid crystal or an organic panel, and displays information visually. The operation unit 217 includes a keyboard, a mouse, and a switch button, and accepts various operations. The clock circuit 218 measures the time so that the time when the server 201 communicates can be known. The communication control unit 219 is connected to the communication network 202. The hard disk drive 215, the display unit 216, and the operation unit 217 may be connected to the outside of the server 201 via the communication control unit 219.

In the server 201, in the case of performing a plurality of independent first to processing to the client 203 ₁ ~203 _n of the n as a communication partner at the same time, it executes the contents of each process, affecting the execution contents of another process As a result, processing is performed in multiple contexts. In order to realize a plurality of contexts, there are a method using a multi-thread and a method using a shared memory in a multi-process. By sharing data among a plurality of threads or between a plurality of processes, a plurality of threads or a plurality of processes can execute the same program by referring to the same data. In this way, communication multiplex processing is realized by a plurality of contexts. Since threads in the same process share memory, it is relatively easy to share data between threads. For this reason, in the present embodiment, description will be made with a method using multi-threads, but the same implementation is possible with a method using shared memory in multi-processes.

FIG. 2 shows a server program unit composed of a server program that operates on a server constituting the data processing system. A description will be given with FIG. The server program unit 301 is a circuit part realized by a program installed in the server 201 and can perform multiple processing of communication. The communication between the server 201 and the first to nth clients 203 _{1 to} 203 _n is actually realized by the client program installed in the server program unit 301 and the first to nth clients 203 _{1 to} 203 _n. The data processing system 200 is realized by the server program unit 301 and the client program unit.

  The server program unit 301 performs corresponding processing on a plurality of threads, that is, an event waiting thread 311, a non-block receiving thread 312, a block receiving thread 313, and an application thread 314. Further, a user object 321 that collects a program and data necessary for operating the server 201 for each user, and a program and data related to a connection with the client for each client with which the server 201 is communicating are collected. The connection object 322 includes two types of objects. Each of these holds a data part 323 of a user object and a data part 324 of a connection object as data parts. Based on the user object data part 323 and the connection object data part 324, the user object program part 325 and the connection object program part 326 operate. At the same time, the communication control access library 330, the execution thread control unit 331, the event waiting control unit 332, the non-block reception control unit 333, the block reception control unit 334, and the user program unit 335 realized by the user program also operate. ing. In this embodiment, only the minimum necessary objects for carrying out the present invention of receiving data from a plurality of connection destinations by multiplex communication processing are described. Actually, objects for other purposes can be added and used individually. For example, in the case of a server that handles a deposit / withdrawal process of a financial institution, an object that summarizes the calculation process and data necessary for the calculation process is added for each client.

  Each program part and thread will be described.

  The communication control access library 330 is a program part that can be called from all programs operating on the server program part 301. The communication control access library 330 provides an interface for exchanging data with an external program for all programs running on the server 201. Specifically, the connection as a communication path between the server 201 and the client 203 is connected and disconnected, and data is transmitted and received. The communication control access library 330 can be realized based on, for example, BSD SOCKET (Berkeley Software Distribution socket).

  The execution thread control unit 331 is also realized by a program that can be called from all programs operating on the server program unit 301. The execution thread control unit 331 can transmit a processing request from a program unit operating on a different thread to the program unit. In addition, when all the threads that can be used by the requested program unit are in use and a new processing request cannot be received, the execution thread control unit 331 stores the processing request in a queue (not shown). .

  The event waiting control unit 332 is realized by a program operating on the event waiting thread 311. The event waiting control unit 332 receives a connection request from each client program unit, and causes the communication control access library 330 to connect the connection. In addition, a connection object 322 corresponding to each connection is created. Then, it is determined whether the non-block reception control unit 333 or the block reception control unit 334 receives data according to the availability of the block reception thread 313 and the communication frequency with each client program unit (not shown). This determination method will be specifically described later. The event wait control unit 332 also waits for multiple reception for creating a list of connections that can receive data for the client program unit assigned to the non-block reception control unit 333. The multiple reception waiting will be specifically described later. Furthermore, the event waiting control unit 332 also receives a communication disconnection request from the client program unit, and performs a disconnection process.

  The reference to the connection object 322 of the connection for which the block reception method is selected by the event waiting control unit 332 is stored in the block reception connection management table 342. The block reception connection management table 342 manages the time at which data is received from the connection for each connection. This reception time is updated with the time measured by the clock circuit 218 every time reception is performed, and is always information on the time of reception at the end.

  Further, the reference to the connection object 322 to the connection for which the non-block reception method is selected is stored in the non-block reception connection management table 341. Similarly to the block reception connection management table 342, the latest reception time is managed for each connection in the non-block reception connection management table 341 as well. The event wait control unit 332 waits for multiple reception for creating a list of receivable connections for the connections stored in this table.

  The block reception control unit 334 is realized by a program that operates on the block reception thread 313, and performs a reception process on the connection assigned by the event wait control unit 332. The block reception control unit 334 operates so as to continuously receive data from a connection once allocated. The block reception control unit 334 passes the data received by the block reception thread 313 to the user program unit 335 via the execution thread control unit 331. This allows the user program unit 335 to perform other processing subsequent to the reception processing by the application thread 314, and the block reception control unit 334 can immediately perform the next reception processing on the block reception thread 313. Therefore, when continuous communication is performed, communication processing can be performed with little overhead.

  The non-block reception control unit 333 is realized by a program that operates on the non-block reception thread 312. When reception data exists for the connection assigned by the event wait control unit 332, the reception processing is performed. In this embodiment, the event waiting control unit 332 allows the non-block reception control unit 333 to perform a reception operation after reception is always possible. If reception is performed when there is no reception data, the result that there is no reception data is immediately returned in the non-block reception method. Similar to the block reception control unit 334, the non-block reception control unit 333 passes the data received by the non-block reception thread 312 to the user program unit 335 and does not perform any processing other than the reception process.

  The user program unit 335 is realized by a program operating on the application thread 314, and received from the client program unit received from the block reception control unit 334 and the non-block reception control unit 333 via the execution thread control unit 331. Processing is performed according to data and processing requests. There is a possibility that data having an extremely long processing time other than reception may exist. Even when the processing time is not uniform for each data, the user program unit 335 performs processing in a thread different from reception in this way. The thread used for reception is not affected. Of course, such data processing may be performed by the block reception thread 313 and the non-block reception thread 312 without being performed by the application thread 314. When there is a program operating on the server 201 using resources such as a CPU and a memory other than the server program unit 301, the number of threads, that is, the number of execution multiplicity, for example, is started by the server program unit 301. In such a case, it is determined based on the usage rate of the CPU 211 at the time, and the processing capacity of the entire server 201 is prevented from being lowered. Also, in the data processing in the user program unit 335, as in the case of data reception, the application thread 314 can be dedicated to one connection for processing and the other can be separated.

  FIG. 3 shows the processing flow of the entire server program unit. The description will be made with reference to FIGS. When the server program unit 301 receives a request to start connection control processing as a server from the OS of the server 201 (step S401: Y), the server program unit 301 first starts the operation of the server program unit 301 on one thread (step S402). ). This thread will be referred to as the initial thread for purposes of explanation. The initial thread may be generated when the OS starts processing of the server program unit 301, or is one of threads originally reserved for various processing performed by the OS. Also good. The server program unit 301 generates threads such as an event waiting thread 311, a block reception thread 313, a non-block reception thread 312, and an application thread (step S403), and each program unit starts operating on each thread (step S403). Step S404). From here, the processing of the server program unit 301 is performed in parallel by a plurality of program units across each thread (step S405), and continues to operate until a stop request is received from the OS (step S406: N). The processing of each program unit performed in parallel will be separately described later. The initial thread may be reserved for controlling the entire server program unit 301 as a thread (not shown in FIG. 2), or may be used for processing other programs if not necessary. Good. When the server program unit 301 receives a stop request from the OS (step S406: Y), the server program unit 301 stops each program unit (step S407) and ends the operation as a whole (end). Depending on the number of threads, the number of processes that can be performed simultaneously increases, but if the number of threads is increased more than necessary, the processing capacity of the CPU will be distributed, and the processing capacity of each thread will decrease. End up. Therefore, the server program unit 301 may be configured to generate a thread only when necessary and delete it when unnecessary. However, even in this case, it is desirable that the event waiting thread 311 is always present in order to receive a connection request from the client. By the way, there are generally many processes necessary for creating and deleting threads, and if this is done frequently, the load on the CPU 211 will eventually increase. For this reason, if all the threads are generated at the time of starting the server program unit 301, it is more efficient to continue the operation until it stops. Also in this specification, a case where a thread is not generated or deleted during the operation of the server program unit 301 will be described.

  FIG. 4 shows the flow of processing of the event waiting control unit among the processing performed by the server program unit. The description will be made with reference to FIGS. When the event waiting control unit 332 is started on the event waiting thread 311 by the server program unit 301, the event waiting control unit 332 first prepares to receive a connection request from the client 203 and starts waiting for an incoming call (step 451). The start of waiting for an incoming call is executed by activating the communication control access library 330. Actually, for example, when the TCP / IP (Transmission Control Protocol / Internet Protocol) protocol is used, a server socket is created with a designated port number, and a state of waiting for an incoming call from the client 203 is created. In TCP / IP, one IP address is composed of a plurality of ports, and in order to perform communication, an IP address and a port number are designated. For example, communication may be performed by a plurality of program units such as a browser and a mailer on one computer. At this time, a port number is assigned to the program in order to designate the program data for which program unit. A socket refers to a set of an IP address and a port number. That is, the socket operates as a virtual interface when the program unit communicates with a program unit realized by a program of another device. As described above, the communication control access library 330 can be realized based on BSD SOCKET, which is currently a mainstream socket. Those waiting for an incoming call are also handled as a file descriptor or a socket object indicating a server socket. Here, these are collectively called an incoming call waiting object.

  When receiving the stop request (step S452: Y), the event waiting control unit 332 ends the processing (end). While the stop request is not received (step S452: N), the processing is continued on the event waiting thread 311, and first, multiple reception waiting for non-block reception is performed (step S453). The wait for multiple reception is a process performed for a connection assigned to non-block reception registered in the non-block reception connection management table 341 and waiting until data can be received. For example, in UNIX (UNiplexed Information and Computing System) OS, as a system call or library named “select ()” or “pol ()”, a programming language of Sun Microsystems, Inc. JAVA (registered trademark) is provided as a service named selector. Specifically, when these functions and services are called by the server program unit 301, the data from the connection buffered by the execution of the OS is scanned in the buffer memory to scan one or more data. When it is found, it is operated so as to return it to the server program unit 301. If no data is found, the scan is repeated until it is found. Here, the description will be made on the assumption that the received data from the connection is buffered in the buffer memory by executing the OS using this selector, and the result of waiting for multiple reception is obtained.

  FIG. 5 shows an example of a result of waiting for multiple reception. This will be described with reference to FIG. The result of waiting for multiple reception is composed of information indicating the presence / absence of data buffered by execution of the OS in the buffer memory for each connection being connected. The scanning result by the selector is always reported when there is data that the server program unit 301 can immediately receive from one or more connections. The example of FIG. 5 represents that data from the client x already exists in the buffer memory. Also, there is no data from client y yet. Such information can be obtained for the connected clients.

  Returning to FIG. 4, the description will be continued based on the example of the result of waiting for multiple reception in FIG. As a result of starting the waiting for incoming call in step S451 and waiting for multiple reception in step S453, when a connection is requested from a new client or a result of waiting for multiple reception is reported, processing corresponding to each is performed. Therefore, when a processing target is generated, a determination process is performed to determine which of the two is to be processed (step S454). The processing according to these processing targets and the data related thereto are collectively set as a processing target object. If the object to be processed is an incoming call waiting object (Y), a connection object 322 is created for the connection requesting client, and the reference is stored in the non-blocking reception connection management table 341 (step S455). On the other hand, when the object to be processed is not an incoming call waiting object (step S454: N), that is, when a scanning result is received by the selector, taking the example of FIG. Or the process which determines which of the block reception control parts 334 performs is performed. First, it is checked whether or not the block reception thread 313 has a vacancy (step S456). If there is a vacancy (Y), a processing request is sent to the block reception control unit 334 via the execution thread control unit 331 and processed on the block reception thread 313 (step S457), and the connection with this client x is established. Register in the block reception connection management table 342. If there is no empty block reception thread 313 (step S456: N), reception processing is performed by the non-block reception thread 312. The non-block reception control unit 333 sequentially performs reception processing with a plurality of threads, but when waiting for connections exceeding the number of threads, a waiting time occurs until actual reception processing is started. Therefore, the reception time is updated before the reception processing request is transmitted to the non-block reception control unit 334 (step S458). Thereafter, a request for reception processing is transmitted to the non-block reception thread 312 via the execution thread control unit 331 (step S459).

  In this way, after creating the connection object and storing the reference in the non-block reception connection management table or assigning the process to the block reception control unit 334 or the non-block reception control unit 333 (steps S455 and S457). In step S459), if all the processing target objects have been processed (step S460: Y), the process is repeated from the restart of waiting for multiple reception in step S453 unless a thread stop request is received (step S452: N). If there are still objects to be processed (step S460: N), the processing from step S454 is repeated until all the objects are removed. Here, when a connection request is received from a client, a reference is stored in the non-block reception connection management table 341 after creating a connection object in step S455. However, if there is an empty block reception thread 313, the connection object 322 may be registered in the block reception connection management table 342 and performed by the block reception control unit 334. However, since there is a case where only connection is performed in advance and communication is not performed, it is desirable to perform communication control by the non-block receiving thread 312 at the time of connection. In general, connection processing requires a larger amount of processing than communication itself because it is necessary to recognize the other party and determine whether or not connection is possible. For this reason, as the server 201, when the connection is made in advance and the connection is maintained, a high throughput (Throughput) is realized with respect to more connections, and an efficient operation becomes possible. Therefore, in the present invention that improves the overall performance by applying the block reception method to some connections while maintaining the degree of freedom for the number of connections that can be handled simultaneously by the non-block reception method, in the present invention, the block is blocked at the time of connection. It is effective to select a non-block reception method instead of selecting a reception method, and to select block reception control when actual communication occurs.

  FIG. 6 shows the flow of processing of the block reception control unit among the processing performed by the server program unit. The description will be made with reference to FIGS. When the server program unit 301 starts on the block reception thread 313, the block reception control unit 334 first waits for a processing request to be sent from the event wait control unit 332 via the execution thread control unit 331 (step S501). When receiving the stop request (step S502: Y), the block reception control unit 334 ends the processing (end). While the stop request is not received (step S502: N), the processing is continued on the block reception thread 313 and waits until the processing request is received (step S503: N). When the processing request is sent to the block reception control unit 334 so as to receive the receivable data by block reception by the processing of step S457 of FIG. 4 of the event waiting control unit 332, this processing request is received (step S503: Y ), Data reception is performed (step S504). Since this is performed by the normal block reception method, it is a reception process having a waiting time for waiting for data. A predetermined value (time limit) is set for this waiting time.

  The event waiting control unit 332 determines that the receivable data exists as a processing target in step S454 in FIG. 4, reaches step S457, and transmits a processing request to the block reception control unit 334. Therefore, data reception immediately after receiving a processing request from the event wait control unit 332 is performed immediately. However, in the normal operation of the block reception control unit 334, it may take time to receive data. When this exceeds a predetermined value and a time-out occurs (step S505: Y), reference to the connection is made from the block reception connection management table 342 so that the data reception from the client 203 is performed in the non-block reception method. Move to the management table 341 (step S506). As a result, the block reception thread 313 becomes an empty thread, and the process returns to step S501 to wait for the next connection to be allocated.

  On the other hand, if reception is performed without time-out (step S505: N), subsequently, a determination process is performed to determine whether the received data is normal (step S507). If the reception result is not normal (N), an error log recording the cause is created (step S508), and the reference to the connection is deleted from the block reception connection management table 342 (step S509). As a result, the block reception thread 313 becomes an empty thread, and the process returns to step S501. Possible causes of the reception being abnormal include, for example, the case where the client 203 as the data transmission source disconnects the connection before the data reception processing is performed, but the reception result is abnormal although received. . The error log created in this way can be used by notifying the administrator of the server 201 or the user of the client 203.

  If the reception is performed normally (step S507: Y), the reception time is updated (step S510). Subsequently, the received data is transferred together with the processing request to the user program 335 on the application thread 314 via the execution thread control unit 331 (step S511). If a thread stop request is received (step S512: Y), the process ends (end). If not received (step S512: N), the process returns to step S504 to repeat the process of receiving the next data.

  FIG. 7 shows the flow of processing of the non-block reception control unit among the processing performed by the server program unit. The description will be made with reference to FIGS. When the non-block reception control unit 333 is started on the non-block reception thread 312 by the server program unit 301, the non-block reception control unit 333 first waits for a processing request from the event wait control unit 332 via the execution thread control unit 331 (step S551). When receiving the stop request (step S552: Y), the non-block reception control unit 333 ends the process (end). Until the stop request is received (step S552: N), the processing is continued on the non-blocking reception thread 312 and waits until the processing request is received (step S553: N).

  When the processing request is sent to the non-blocking reception control unit 333 so as to receive the receivable data by non-blocking reception by the processing in step S459 of FIG. 4 of the event waiting control unit 332, the processing request is received (step S553). : Y), data reception is performed (step S554). The event waiting control unit 332 assumes that the receivable data exists as a processing target in step S454 of FIG. 4, and has reached step S459 and has transmitted a processing request to the non-block reception control unit 334. Therefore, data reception by the non-block reception control unit 334 is performed as soon as a processing request is received.

  If the reception is not performed normally (step S555: N), an error log recording the cause is created (step S556). Further, the reference to this connection is deleted from the non-blocking reception connection management table 341 (step S557), and the process returns to the process of waiting for the processing request from the event waiting control unit 332 in step S551, and again in the non-blocking reception method. Repeat the receiving process.

  On the other hand, if the reception is normally performed (step S555: Y), the received data is transferred to the user program 335 on the application thread 314 via the execution thread control unit 331 together with the processing request (step S558). Then, a process of determining whether or not the received data is the last data from the connection is performed (step S559). This can be understood from whether or not an EOF (End of File) code representing the end of the file is included in the information representing the outline of the reception result received together with the received data. If it is the last data (Y), after performing the process of deleting the reference to this connection from the non-block reception connection management table 341 in step S557, on the other hand, if it is not the last data (step S559: N), Returning to the process of waiting for a process request from the event wait control unit 332 in step S551 as it is, the reception process by the non-block reception method is repeated again in the same manner.

  As described above, the non-block reception control unit 333 switches the connection to be processed by the thread every time a processing request is received. As a result, a small number of threads can be released to many connections without occupying a specific connection. The reason why the process of updating the reception time is not included in the process of the non-block reception control unit 333 is that the process has already been performed in step S458 of the process of the event waiting control unit 332 in FIG. As described above, the number of non-block reception threads 312 may be smaller than the number of connections assigned to the non-block reception method. At this time, the processing of the non-block reception control unit 333 as shown in FIG. Not started. Therefore, the processing request transmitted to the non-blocking reception control unit 333 by the processing of the event waiting control unit 332 is executed until the processing being executed on the non-blocking reception thread 312 ends and an empty thread is generated. It is stored in the queue 331. For this reason, when reception events occur simultaneously in connections exceeding the number of threads, waiting occurs. However, since these are sequentially received, they are not dedicated to one connection unlike the block reception method. In other words, since the number of connection partners is not limited, the server system can be used in a wider range of fields.

  As described above, according to the present embodiment, the server program unit 301 that handles a large number of connections simultaneously can select the block reception control unit 334 and the non-block reception control unit 333 according to the communication frequency. did. As a result, it is possible for the block reception control unit 334 to perform high-throughput communication for connections with high communication frequency, and the non-block reception control unit 333 can perform communication multiplex processing for other connections. I made it. In addition, by preventing the number of connections controlled by the block reception control unit 334 from exceeding a certain number, it is possible to suppress an increase in the number of contexts and prevent a decrease in processing capacity of the server program unit 301 as a whole. Furthermore, a time limit is set for data reception from the connection allocated to the block reception control unit 334, and the allocation is changed to the non-block reception control unit 333 when a timeout occurs. When a server program of either a block reception system or a non-block reception system is installed in the server and the number of connection partners is not limited, the non-block reception system is generally combined with a selector. However, if one of the connection partners communicates at a high frequency, the overhead due to selector scanning becomes a problem. However, according to the present embodiment, since the block reception method can be used for a connection partner with high communication frequency, scanning by the selector is performed only for connection partners with no communication or low communication frequency. For this reason, scanning overhead can be reduced. In this way, by dynamically changing the connection allocation, the data processing server of the present invention can simultaneously realize high communication performance while maintaining a degree of freedom with respect to the number of connections that can be connected. .

  Conventionally, a non-block reception method using an event type program has been proposed. In an event-type program, when a certain event occurs, another process of the program is induced. For example, data reception and data processing are started by being triggered by an event that data from the connection is buffered in the buffer memory by the OS. According to this, it is not necessary to scan the presence / absence of data buffered by the selector. For this reason, overhead due to scanning can be eliminated, and the problem of deterioration in processing efficiency affected by scanning time can be eliminated even for connections with high communication frequency. As an implementation of the event-driven communication program, for example, “BSD” (Berkeley Software Distribution) compatible with UNIX (registered trademark) is “kqueue”, and Linux (Linux) is also compatible with “epool” (epool). In addition, functions such as “BeginRecv” and “EndRecv” are provided in Windows (registered trademark), which is an OS of Microsoft Corporation. Such an event-type program is significantly different in structure from a block reception type program and a non-block reception type program in which the calling relationship of each process is clear. However, many existing applications are realized by a program designed by a non-block reception method using a selector. It takes a lot of effort to change these to event-type programs with unclear calling relations, but if the communication control method of the present invention is used, it will be comparable to event-type programs with less effort. Efficient data reception. In an event type program, the OS or library preferentially processes connections to be processed. A library is a collection of general-purpose data and functions over a plurality of programs. Like a OS, a library is configured separately from a server program, and thus its operation cannot be freely changed. According to the data processing method of the present invention, the calculation of the connection usage frequency can be implemented on the server program, and not only the priority connection can be dynamically changed in the same manner as the event type, but also the user can arbitrarily You can also change the connection to give priority to.

  Here, the server program is described among the programs that use a communication protocol such as TCP / IP and communicate with each other via a LAN, the Internet, or the like. The present invention is more effective in a program that processes a large number of connections simultaneously, but it is of course possible to implement the same implementation in a client program.

<Modification of the present invention>

  FIG. 8 shows the flow of processing of the event waiting control unit among the processing performed by the server program unit of the modification of the present invention. FIG. 8 corresponds to FIG. 4 of the embodiment, and the same parts as those in FIG. 4 are denoted by the same step numbers, and description thereof will be omitted as appropriate. In addition, the processes before step S454 are not shown. The event waiting control unit 332 of the embodiment is an event waiting control unit 332A in this modification. Further, the block reception control unit 334 of the embodiment is a block reception control unit 334A in this modification. This is because each of these is realized by a program that performs processing different from the embodiment. This will be described with reference to FIG. When there is receivable data, the event waiting control unit 332A determines whether the data reception is performed by the block reception control unit 334A or the non-block reception control unit 333 based on the latest reception time of each connection. It comes to judge. If there is no empty block reception thread 313 in step S456 of FIG. 8 (N), reception times are compared (step S601). The reception times of the block reception connection management table 342 and the non-block reception connection management table 341 are used for comparison of the reception times. Here, it is determined whether or not the reception time of the connection assigned to the non-block reception control unit 333 having receivable data is newer than the reception time of any one of the connections assigned to the block reception control unit 334A. (Step S602). If it is newer (Y), the connection exchanged with the connection with the oldest reception time among the connections allocated to the block reception control unit 334A is performed (step S603), and the process proceeds to step S460. If it is old (step S602: N), the subsequent processing is performed while being assigned to the non-block reception control unit 333 (steps S458 to S460).

  FIG. 9 shows the flow of processing of the block reception control unit among the processing performed by the server program unit of the modification. FIG. 9 corresponds to FIG. 6 of the embodiment, and the same parts as those in FIG. 6 are denoted by the same step numbers, and description thereof will be omitted as appropriate. This will be described with reference to FIG. When the block reception control unit 334A is performing data reception processing including data waiting (step S504), even if the timeout has not occurred (step S505: N), the event waiting control unit 332A assigns it to the non-block reception thread 312. Is requested (step S610: Y), the reference is moved to the non-block reception connection management table 342 and the assignment is changed (step S506). If no change is requested (step S610: N), data reception is performed as it is using the block reception method.

  According to the data processing server of this modification, even if the connection assigned to the block reception control unit 334A does not time out and the communication is frequently performed, the connection assigned to the non-block reception control unit 333 If there is something that communicates more frequently, the assignment can be changed. However, it is conceivable that the assignment is also changed in the case of a connection assigned to the non-block reception control unit 333 that originally received data frequently and happens to be continuously transmitted. Therefore, information on the number of communications in a period such as the past day or the past hour may be recorded in the block reception connection management table 342 and the non-block reception connection management table 341. It is also possible to assign reception control according to this number of times.

It is a block diagram showing the outline | summary of the structure of the data processing system by one Example of this invention. It is a block diagram showing the server program part comprised by the server program which operate | moves on the server which comprises a data processing system. It is a flowchart showing the flow of processing of the whole server program part. It is a flowchart showing the flow of the process of the event waiting control part among the processes which a server program part performs. It is explanatory drawing showing the example of the result of waiting for multiple reception. It is a flowchart showing the flow of the process of a block reception control part among the processes which a server program part performs. It is a flowchart showing the flow of the process of a non-block reception control part among the processes which a server program part performs. It is a flowchart showing the flow of the process of the event waiting control part among the processes which the server program part by the modification of this invention performs. It is a flowchart showing the flow of the process of a block reception control part among the processes which the server program part by the modification of this invention performs. It is explanatory drawing showing a mode that a network server controls a some connection by one context. It is explanatory drawing showing a mode that a network server controls a some connection by a some context. It is explanatory drawing showing a mode that a network server controls a some connection by the non-block receiving system by one context and a selector.

Explanation of symbols

200 Data Processing System 201 Server 203 Client 211 CPU
301 server program unit 311 event waiting thread 312 non-block receiving thread 313 block receiving thread 314 application thread 321 user object 322 connection object 330 communication control access library 331 execution thread control unit 332 event waiting control unit 333 non-block receiving control 334 block receiving control 335 User program part 341 Non-block reception connection management table 342 Block reception connection management table

Claims (7)

Unit data receiving means for sequentially receiving unit data as data for each processing unit via a communication network;
Transmission source determination means for determining the transmission source of the unit data received by the unit data reception means;
Reception frequency monitoring means for monitoring the reception frequency of the unit data for each transmission source;
Data storage means for temporarily storing unit data received by the unit data receiving means;
Processing of an arbitrary number of unit data sent intermittently or continuously from the same transmission source in a form that one transmission source occupies for each desired time causes data waiting for that transmission source. One or a plurality of communications enabling high-throughput communication when the frequency of receiving unit data from the same transmission source is high by not performing processing for other transmission sources until the data for the transmission source is processed . First data processing means;
One or a plurality of second data processing means for processing the unit data in a form that each transmission source occupies in a time-sharing manner with the unit data as one unit;
Reception frequency of the received data unit of the transmission unit data stored in the data storage means according to the determination result of the original determination unit, said unit data receiving means each time the said unit data receiving means receives said unit data First data transfer means for assigning and transferring to the first data processing means in order from the highest transmission source;
Data presence / absence determining means for determining presence / absence of unit data stored in the data storage means without being delivered to the first data processing means by the first data delivery means;
A second data transfer means for transferring the unit data from the data storage means to the second data processing means when the determination result of the data presence / absence determination means is data. A data processing device. The first data processing means and the second data processing means further include one or a plurality of third data processing means for receiving unit data for which the respective processing has been completed and performing further processing. The data processing apparatus according to claim 1 . The first data transfer unit includes an exclusive transmission source table indicating a transmission source dedicated to each of the one or more first data processing units, and the transmission is performed each time the data reception unit receives the unit data. 2. The data according to claim 1, wherein a search is made as to whether or not a transmission source indicated by the determination result of the original determination means is included in the exclusive transmission source table, and when included, the unit data is transferred to the transmission source. Processing equipment. A priority is assigned to each of the transmission sources, and the first data transfer unit is configured to use the first data according to the usage efficiency of each of the first data processing unit and the second data processing unit. The data processing apparatus according to claim 1, further comprising a transfer adjustment unit that adjusts a transmission source to be transferred to the processing unit. Unit data receiving means for sequentially receiving unit data as data for each processing unit from a client via a communication network;
Client determination means for determining a client of unit data received by the unit data receiving means;
A reception frequency monitoring means for monitoring the reception frequency of the unit data for each client determined by the client determination means;
Data storage means for temporarily storing unit data received by the unit data receiving means;
When each client waits for data to be processed, any number of the unit data sent intermittently or continuously from the same client in a form that each client occupies for a desired time. One or a plurality of first data processing means for enabling high-throughput communication when the frequency of receiving unit data from the same client is high by not performing processing for other clients until data about the client is processed When,
One or a plurality of second data processing means for processing the unit data in a form that each client occupies in a time-sharing manner with the unit data as one unit;
Each time the unit data receiving unit receives the unit data, the unit data stored in the data storage unit according to the determination result of the client determining unit is changed to the reception frequency of the unit data received by the unit data receiving unit. First data transfer means for assigning and transferring to the first data processing means in order from the highest client;
Data presence / absence determining means for determining presence / absence of unit data stored in the data storage means without being delivered to the first data processing means by the first data delivery means;
Second data transfer means for transferring the unit data from the data storage means to the second data processing means when the determination result of the data presence / absence determination means is data present
And a data processing server . A communication network;
A data processing device according to claim 1;
One or more data processing requesting devices provided with data transmitting means for transmitting data to the data processing device via the communication network
And a data processing system . On the computer,
Unit data reception processing for sequentially receiving unit data as data for each processing unit via a communication network;
A transmission source determination process for determining the transmission source of the unit data received in the unit data reception process;
A reception frequency monitoring process for monitoring the reception frequency of the unit data for each transmission source;
A data storage process for temporarily storing the unit data received in the unit data reception process;
Processing of an arbitrary number of unit data sent intermittently or continuously from the same transmission source in a form that one transmission source occupies for each desired time causes data waiting for that transmission source. One or a plurality of communications enabling high-throughput communication when the frequency of receiving unit data from the same transmission source is high by not performing processing for other transmission sources until the data for the transmission source is processed. First data processing;
One or a plurality of second data processing for processing the unit data in such a manner that each transmission source occupies the unit data as a unit in a time division manner;
The reception frequency of the unit data received in the unit data reception process, the unit data stored in the data storage process according to the determination result of the transmission source determination process every time the unit data is received in the unit data reception process A first data transfer process that assigns and transfers to the first data process in order from the highest transmission source;
A data presence / absence determination process for determining the presence / absence of unit data that is not transferred to the first data process and stored in the data storage process by the first data transfer process;
A second data transfer process for transferring the unit data stored in the data storage process to the second data process when the determination result of the data presence / absence determination process is data present
And a data processing program .

Images