CN110795254A - Method for processing high-concurrency IO based on PHP - Google Patents

Abstract

The invention discloses a method for processing high-concurrency IO based on PHP, which comprises the following steps: the Swoole creates a single or a plurality of processes according to the configuration; the Master process creates N Reactor threads; creating coroutines in each Reactor thread; and asynchronously processing the IO tasks through coroutine scheduling. The method comprises the steps of using a Swoole model, creating a coroutine in each thread through a multithread Reactor model (based on epoll) and a multiprocess Worker, and asynchronously processing IO tasks through coroutine scheduling, namely when IO time consumption is met, processing other code logics through coroutine scheduling until IO is completed and then continuing to execute the previous codes; the system resources are fully utilized, IO blocking waiting and waste of the system resources are reduced, the IO processing capacity under high concurrency is greatly improved, and the problems of Accept performance bottleneck and surprise groups can be solved.

Description

Method for processing high-concurrency IO based on PHP

Technical Field

The invention relates to the technical field of high-concurrency IO processing, in particular to a method for processing high-concurrency IO based on PHP.

Background

High concurrency, which generally refers to the concurrent parallel processing of many requests, is one of the factors that must be considered in the design of a distributed system architecture. When the time for the system to respond to the request is too long, the user experience of the internet product is greatly reduced.

In the internet era, high concurrency generally refers to concurrent access, that is, a plurality of accesses arrive at the same time at a certain time point. Typically, a system has a daily PV of more than ten million, and may be a highly concurrent system.

At present, the most of solutions for concurrent IO use an IO multiplexing technology, and Linux provides select system call for a long time and can maintain 1024 connections in one process. Later, a poll system call is added, and some improvements are made to poll, so that the problem of 1024 limitation is solved, and any number of connections can be maintained. However, select/poll also has a problem: select/poll requires a loop to check the connection for events. At this time, if the server has 100 thousands of connections, only one connection sends data to the server at a certain time, and select/poll needs to be cycled 100 thousands of times, wherein only 1 time is hit, and the remaining 999999 times are invalid, which wastes CPU resources.

This does not really solve the C10K problem until the Linux 2.6 kernel provides a new epoll system call, an unlimited number of connections can be maintained, and no polling is required. Currently, various server programs with high concurrent asynchronous IO are implemented based on epoll, such as Nginx, node. Js, a single process, single threaded program, can maintain over 1 million TCP connections, all due to epoll technology.

Disclosure of Invention

Aiming at the defects existing in the problems, the invention provides a method for processing high-concurrency IO based on PHP.

The invention discloses a method for processing high-concurrency IO based on PHP, which comprises the following steps:

the Swoole creates a single or a plurality of processes according to the configuration;

taking the current process as a Master process, wherein the Master process creates N Reactor threads;

creating coroutines in each Reactor thread;

and asynchronously processing the IO tasks through coroutine scheduling.

As a further improvement of the invention, N is jointly determined by the configured reactivor _ num, worker _ num and the number of CPU cores.

As a further improvement of the present invention, a Reactor object is run in each of the Reactor threads;

the Reactor thread is used for receiving data sent by the client, sending response data from the server and processing a Swoole timer event.

As a further improvement of the present invention, the creating coroutines in each Reactor thread includes:

when each Reactor thread receives a request of a client, a coroutine is created through the Go function of the swoole.

As a further improvement of the present invention, the asynchronous processing of the IO task by coroutine scheduling includes:

when IO blockage occurs in the coroutine, the coroutine gives out control, and the coroutine enters a coroutine scheduling queue to perform asynchronous processing on a blocked task.

Compared with the prior art, the invention has the beneficial effects that:

the method comprises the steps of using a Swoole model, creating a coroutine in each thread through a multithread Reactor model (based on epoll) and a multiprocess Worker, and asynchronously processing IO tasks through coroutine scheduling, namely when IO time consumption is met, processing other code logics through coroutine scheduling until IO is completed and then continuing to execute the previous codes; the system resources are fully utilized, IO blocking waiting and waste of the system resources are reduced, the IO processing capacity under high concurrency is greatly improved, and the problems of Accept performance bottleneck and surprise groups can be solved.

Drawings

FIG. 1 is a flowchart illustrating a method for PHP-based high-concurrency IO processing according to an embodiment of the present invention;

FIG. 2 is a diagram of a Reactor model disclosed in one embodiment of the present invention;

fig. 3 is a flowchart of coroutine scheduling according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.

The invention is described in further detail below with reference to the attached drawing figures:

swoole: PHP asynchronous network communication engine facing production environment.

Coordinating the program: the method is an asynchronous IO Reactor model, the application layer automatically realizes task scheduling, and each currently executed user mode thread is switched by means of the Reactor.

The invention provides a method for processing high-concurrency IO based on PHP, which is characterized in that the IO tasks are processed asynchronously through coroutine scheduling by adding a Reactor model (based on epoll) and a multi-process Worker, so that the IO tasks under the high-concurrency condition are processed efficiently; the method comprises the following steps:

as shown in fig. 1, the present invention provides a method for processing high-concurrency IO based on PHP, which includes:

s1, the Swoole can create single or multiple processes according to the configuration; each process is internally provided with a Master process and a plurality of sub-processes;

s2, taking the current process as a Master process, and creating N Reactor threads by the Master process;

the method specifically comprises the following steps:

when the Swool Server is started, the Master process can create N Reactor threads, wherein N is determined by the actual configuration option, namely, the reactiver _ num, the worker _ num and the actual core number of the CPU;

running a Reactor object in each Reactor thread; and the Reactor thread is used for receiving data sent by the client, sending response data from the server and processing the Swoole timer event.

Further, in the present invention,

as shown in fig. 2, the Reactor model has 4 core operations:

add socket to monitor the reactivor, which may be a listen socket or a client socket, or a pipe, eventfd, signal, etc

The Set modifies the event snooping, and can Set the type of snooping, such as readable and writable. It is well understood that with listen socket is the need for accept when a new client connection arrives. Recv is required for client connections to be data received. A SOCKET is buffered, and if 2M data is to be sent to a client connection, it is not sent out at one time, and the operating system defaults to a TCP buffer of only 256K. 256K can be sent at a time, and the send returns an EAGAIN error after the buffer is full. At this time, the writeable event is to be snooped, and in pure asynchronous programming, the writeable must be snooped to ensure that the send operation is completely non-blocking.

Del is removed from the reactivor and is no longer listening for events

Callback is the corresponding processing logic after the event occurs, and is generally established at add/set.

S2, creating coroutines in each Reactor thread;

the method specifically comprises the following steps:

when each Reactor thread receives a request of a client, a coroutine is created through the Go function of the swoole.

S3, asynchronously processing IO tasks through coroutine scheduling;

the method specifically comprises the following steps:

when IO blockage occurs in the coroutine, the coroutine gives out control, and the coroutine enters a coroutine scheduling queue to perform asynchronous processing on a blocked task, so that the time loss caused by IO blockage is reduced.

As shown in fig. 3, the method of scheduling includes:

1) creating Tsak and transmitting the Tsak into the coroutine;

2) calling a run method;

3) scheduling in a Scheduler cycle;

4) scheduling a return value type;

5) if the type of the return value is Tsak _ Continue, returning to the step 3);

the return value type is Tsak _ Wait, and asynchronous execution is carried out; returning to the step 3) after the execution is finished;

the return value type is Tsak _ Done, and the step 6) is skipped;

6) and ending.

Example (b):

the scheduling of threads and processes is regulated by the operating system, while the scheduling of coroutines is regulated by the user himself. The coroutine scheduler of the swoole can suspend the coroutine before the coroutine A is about to enter the blocked IO operation, such as the read of the socket (which is already set as asynchronous IO), save the current stack information StackA, then switch to the coroutine B, and switch back to the current state of the coroutine A according to the StackA when the IO operation of the coroutine A returns.

S4, ending the coroutine:

when the callback function is finished executing, the coro _ close method is used to finish the coroutine.

The invention has the advantages that:

the method comprises the steps of using a Swoole model, creating a coroutine in each thread through a multithread Reactor model (based on epoll) and a multiprocess Worker, and asynchronously processing IO tasks through coroutine scheduling, namely when IO time consumption is met, processing other code logics through coroutine scheduling until IO is completed and then continuing to execute the previous codes; the system resources are fully utilized, IO blocking waiting and waste of the system resources are reduced, the IO processing capacity under high concurrency is greatly improved, and the problems of Accept performance bottleneck and surprise groups can be solved.

The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes will occur to those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (5)

1. A method for processing high-concurrency IO based on PHP is characterized by comprising the following steps:

the Swoole creates a single or a plurality of processes according to the configuration;

taking the current process as a Master process, wherein the Master process creates N Reactor threads;

creating coroutines in each Reactor thread;

and asynchronously processing the IO tasks through coroutine scheduling.

2. The method of claim 1, wherein N is determined by a configured reactivnum, worker num and a number of CPU cores.

3. The method of claim 1, wherein one Reactor object runs in each of the Reactor threads;

the Reactor thread is used for receiving data sent by the client, sending response data from the server and processing a Swoole timer event.

4. The method of claim 1, wherein creating coroutines in each Reactor thread comprises:

when each Reactor thread receives a request of a client, a coroutine is created through the Go function of the swoole.

5. The method of claim 1, wherein the asynchronously processing IO tasks through coroutine scheduling comprises:

when IO blockage occurs in the coroutine, the coroutine gives out control, and the coroutine enters a coroutine scheduling queue to perform asynchronous processing on a blocked task.

Images