CN117687763A

CN117687763A - High concurrency data weak priority processing method and device, electronic equipment and storage medium

Info

Publication number: CN117687763A
Application number: CN202410152183.2A
Authority: CN
Inventors: 朱智源; 刘畅; 徐建; 李阳
Original assignee: Chengdu Yixing Technology Co ltd
Current assignee: Chengdu Yixing Technology Co ltd
Priority date: 2024-02-03
Filing date: 2024-02-03
Publication date: 2024-03-12
Anticipated expiration: 2044-02-03
Also published as: CN117687763B

Abstract

A method, a device, an electronic device and a storage medium for processing high concurrency data with weak priority, wherein the method comprises the following steps: when the queuing data exceeds a threshold value, setting delay; when delay exists, each data is processed as follows in sequence according to the queuing sequence: judging the current count value: if the count value is less than or equal to 0, adding 1 to the count value and then judging consistency of the priority levels; if the count value is greater than 0, judging whether the priority level of the current data is superior or equal to the priority level recorded in the current recorder, if not, queuing again, and if so, adding 1 to the count value and then judging consistency of the priority levels; when judging that the priority levels are consistent, carrying out service processing; when the record is inconsistent, modifying the recorder and then carrying out service processing; the counter value is decremented by 1 after the service processing. The invention processes the emergency data preferentially when the concurrency is high and the resources are limited, maintains the real-time performance of other data, reduces the waiting of the emergency data and avoids the waiting of other data all the time.

Description

High concurrency data weak priority processing method and device, electronic equipment and storage medium

Technical Field

The invention belongs to the technical field of computers, relates to a data processing technology under the condition of high concurrency, and particularly relates to a method and a device for processing high concurrency data with weak priority, electronic equipment and a storage medium.

Background

In high concurrency situations, data processing services always accompany resource preemption problems, especially in situations where hardware resources are limited, often making data unable to be processed in time, resulting in data stacking. However, the data also has priority, some data needs urgent transmission, the real-time requirement is extremely high, some data does not need extremely high real-time, and under the condition of limited hardware resources, all data are piled together to wait for processing, so that the data with extremely high real-time requirement is very unfriendly, and finally, the real-time requirement cannot be met, thereby causing other problems.

Disclosure of Invention

In order to solve the defects of the related prior art, the invention provides a method, a device, an electronic device and a storage medium for processing high concurrency data with weak priority, which can process set emergency data with priority when the high concurrency data is high and no more hardware resources exist, maintain the real-time performance of other data, reduce the waiting time of the emergency data and avoid the problem that other data is always waiting.

In order to achieve the object of the invention, the following scheme is adopted:

a method for processing high concurrency data with weak priority comprises the following steps:

when the quantity of the data queued in the buffer pool exceeds a preset threshold value, setting delay; wherein, the data has a preset priority level;

when the delay exists and the delay is not finished, the following processing is sequentially carried out on each data according to the queuing sequence:

judging whether the count value of the current counter is smaller than or equal to 0: if the count value is less than or equal to 0, the count value of the counter is added with 1, and then priority consistency judgment is carried out; if the count value is greater than 0, judging whether the priority level of the current data is superior or equal to the priority level recorded in the current recorder, if not, re-inputting the current data into a buffer pool for queuing, and if so, adding 1 to the count value of the counter for priority level consistency judgment;

judging consistency of priority levels, and comparing whether the priority level of the current data is consistent with the priority level recorded in the recorder or not: if the data are consistent, carrying out service processing on the data; if the priority levels are inconsistent, modifying the priority levels recorded in the recorder into the priority levels of the current data, and then carrying out service processing on the data;

after the service processing, the count value of the counter is decremented by 1, and the operation is ended.

Further, when the delay does not exist/end, judging whether the quantity of the data queued in the buffer pool exceeds a preset threshold value:

if the time delay exceeds the preset time delay value, setting time delay;

if not, directly carrying out service processing on the first data according to the queuing sequence, and ending the processing flow of the first data; the step of jumping to the decision threshold is performed before processing the next data.

Further, the queued data includes medical data including one or more of out-patient registration data, out-patient charge data, and in-patient order data, the out-patient registration data having a priority level that is superior to a priority level of out-patient charge data, the out-patient charge data having a priority level that is superior to a priority level of in-patient order data.

A high concurrency data weak priority processing apparatus, comprising:

the delay setting module is used for setting delay when the number of the data queued in the buffer pool exceeds a preset threshold value; wherein, the data has a preset priority level;

the weak priority execution module is used for sequentially carrying out the following processing on each data according to the queuing sequence when delay exists and the delay is not finished:

Further, the system also comprises a threshold judgment module and a direct processing module;

the threshold judging module is used for judging whether the quantity of the data queued in the buffer pool exceeds a preset threshold value or not when the delay does not exist/is over:

if yes, entering a delay setting module for execution, and then entering a weak priority executing module for execution;

if not, entering a direct processing module for execution, wherein the direct processing module is used for directly carrying out service processing on the first data according to the queuing sequence and ending the processing flow of the first data; and before the next data is processed, the method jumps to the threshold value judging module for execution.

An electronic device, comprising: at least one processor and memory; wherein the memory stores computer-executable instructions; executing computer-executable instructions stored in the memory at the at least one processor, causing the at least one processor to perform the high concurrency data weak priority processing method as described.

A computer readable storage medium having stored thereon a computer program which, when executed by a processor, controls a device in which the storage medium resides to perform the method of weak priority processing of high concurrency data.

The invention has the beneficial effects that:

the method logic of the invention can lead the program to preferentially process the emergency request processing set by artificial weight under the conditions that high concurrency and high data volume are high and the server can not provide more hardware resources, provides a queue inserting mode for the emergency data, but is not the traditional strong queue inserting condition, the priority of the strong queue inserting can lead the data with lower priority to be unable to be transmitted in a short time, and further leads the real-time performance of other data to be seriously reduced.

Drawings

Fig. 1 shows a flowchart of overall steps of a method for weak priority processing of high concurrency data according to an embodiment of the present application.

FIG. 2 shows a flowchart of the steps for processing data during a delay in an embodiment of the present application.

Fig. 3 shows an overall flowchart of a method for weak priority processing of high concurrency data according to an embodiment of the present application.

Fig. 4 shows a block diagram of a high concurrency data weak priority handling apparatus according to an embodiment of the present application.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the following detailed description of the embodiments of the present invention will be given with reference to the accompanying drawings, but the described embodiments of the present invention are some, but not all embodiments of the present invention.

In one aspect of the embodiments of the present application, a method for processing high concurrency data with weak priority is provided, as shown in fig. 1, including the steps of:

(1) When the quantity of the data queued in the buffer pool exceeds a preset threshold value, setting delay; wherein, the data has preset priority level, which can be preset according to emergency situation, as the priority/emergency degree mark of each data. The buffer pool is arranged to provide a buffer waiting area for data to request waiting in case of high concurrency, and large amount of data can not be consumed. And if the data in the buffer pool exceeds the threshold value, the priority judgment processing is needed, and if the data does not exceed the threshold value, the data does not need to be processed yet.

(2) And when the delay exists and the delay is not finished, processing each data in sequence according to the queuing sequence. That is, after the threshold value is judged to be exceeded at one time, the time delay is started, and the threshold value is not required to be judged again in the time delay time length, and the operation is performed in this way.

The specific processing, as shown in fig. 2, includes the following steps:

and judging the count value of the current counter. Here, the counter is mainly used for counting, and aims to reflect the number of request data being processed by the current service through the value thereof, when one request data is being processed, the count value of the counter is increased by 1, and when one request data processing is finished, the count value is decreased by 1. In the initial state, the count value of the counter is set to 0.

If it is determined that the current count value is equal to or less than 0 (generally, 0 is often the case when the current count value is equal to or less than 0, but the current count value is also classified into the following cases, the current count value is equal to or less than 0, and the following cases are described as 0), the priority consistency determination is performed after the count value of the counter is increased by 1, that is, it is indicated that a new request data start to be added to the processing flow.

If the current count value is greater than 0, it indicates that other request data are currently being processed, then it needs to determine whether the priority level of the current data is better than or equal to the priority level recorded in the current recorder, because the priority level of the request data which is generally being processed is recorded in the recorder, here, in order to see whether the data which is ready to enter into processing should be processed in preference to the data in processing, if not, if so, the current data is re-input into the buffer pool for queuing, if so, then the count value of the counter is increased by 1, and then the consistency of the priority levels is determined.

Judging consistency of priority levels, and comparing whether the priority level of the current data is consistent with the priority level recorded in the recorder or not: if the data are consistent, carrying out service processing on the data; and if the priority levels are inconsistent, modifying the priority levels recorded in the recorder into the priority levels of the current data, and then carrying out service processing on the data. The service processing is to correspondingly process the related service according to the request of each data, and belongs to the algorithm. In the initial state, no record is recorded in the recorder, an inconsistent conclusion is obtained in the first judgment, and then the priority level of the data in the first judgment is recorded.

In this example, when the delay does not exist/end, it is determined whether the amount of data queued in the buffer pool exceeds a preset threshold:

if yes, setting delay, if yes, performing step (1), and then performing step (2).

If the data is not exceeded, the priority processing is not needed temporarily, and the requirements can be met only by processing the data in sequence according to the queuing sequence, so that the processing flow of the first data is ended after the service processing is directly carried out on the first data which is currently queued; then before the next data processing, it is necessary to determine whether the amount of data queued in the buffer pool exceeds a preset threshold, and then execute the process according to the above steps.

As an application of the embodiment of the application in the medical service platform/system, the queuing data is medical related data, for example, includes one or more of outpatient registration data, outpatient charging data and inpatient order data, which have different priorities in processing, and generally, the priority of outpatient registration data is better than the priority of outpatient charging data, and the priority of outpatient charging data is better than the priority of inpatient order data.

To more clearly illustrate the algorithm logic of this example, take outpatient registration data (hereinafter abbreviated GH), outpatient charging data (hereinafter abbreviated SF), and hospitalization order data (hereinafter abbreviated YZ) as examples, a flowchart refers to fig. 3. GH is special emergency data, the priority level is 1, and the highest is achieved; SF is sub-emergency data, the priority level is 2, and the sub-emergency data is sub-high; YZ is non-urgent data, the priority level is 3, and the data is not urgent. Here, the priority level is represented by a numerical value, so that the program is easy to run, and the lower the numerical value is, the more preferable it is.

The service now has 2 processes in the interior, which need to be circulated, the processing time is 1 second process, and 2 seconds is the process is completed. The queuing data in the current buffer pool and the sequence are as follows: GH. SF, YZ, YZ, YZ, YZ, SF, GH, GH, YZ. Algorithm initialization: the count value of the counter is equal to 0, and the recorder records a value of 1.

1. The first GH enters, the count value of the counter is increased by 1, the recorder records a value of 1, and the time steps are 0-1; queuing state: GH (in process), SF, YZ, YZ, YZ, YZ, SF, GH, GH, YZ.

2. And entering the second SF, wherein the count value of the counter is more than 0, judging the priority level of the record, wherein the priority level of the SF is more than 2 and is more than 1, and queuing in the buffer pool again when the entering condition is not met. Service state at this time: count 1, recorder 1, time steps 1-2, queuing status is GH (in process), YZ, YZ, YZ, YZ, SF, GH, GH, YZ, SF (re-queuing) in order.

3. And when the third YZ enters, the count value of the counter is more than 0, the priority level of the counter and the record is judged, the priority level of the YZ is more than 3 and is more than 1, the entering condition is not met, and the buffer pool is used for queuing again. At this time, the first GH completes processing, the count value of the counter is decremented by 1, equal to 0, and the recorder records as 1. Service state: count value 0, recorder 1, time step 2-0, queuing status is GH (processing completed), YZ, YZ, YZ, SF, GH, GH, YZ, SF, YZ (re-queuing) in order.

4. A fourth YZ is entered, the count value of the counter at this time=0, and then the count value is incremented by 1 to perform a coincidence judgment with the recorded priority level, and the priority level of YZ is 3-! The priority level of the modified record is 3, time-step 0-1. Service state: count value 1, recorder 3, time step 0-1, queuing status YZ (in process), YZ, YZ, SF, GH, GH, YZ, SF, YZ.

5. And the fifth YZ is entered, the count value of the counter is more than 0, the priority level of the YZ is 3=recorded value 3, the count value is added with 1 and then is 2, the consistency of the priority level is judged, 3=3, the recorder is not modified, and the time steps are 1-2. Service state: count value 2, recorder 3, time steps 0-1,1-2, queuing status YZ (in process), YZ, SF, GH, GH, YZ, SF, YZ in order.

6. The sixth YZ enters, at this time, the count value of the counter is more than 0, the priority level of the YZ is 3=recorded value 3, the count value is added with 1 and then is 3, the consistency of the priority level is judged, 3=3, the recorder is not modified, the time step is 2-0, the fourth YZ processing is completed, and the count value is subtracted by 1 and is 2. Service state: count value 2, recorder 3, time steps 0-1,1-2,2-0, queuing status is YZ (processing completed), YZ (processing), SF, GH, GH, YZ, SF, YZ in order.

7. The seventh SF is entered, the count value of the counter is more than 0, the priority level of SF is judged to be 2 < 3, the count value is added with 1 to be 3, then the consistency judgment of the priority level is carried out, the priority level is judged to be 2-! The modification recorder records 2, time steps 1-2, and the fifth YZ processing is completed, and the count value is decremented by 1, which is 2. Service state: count value 2, recorder 2, time steps 0-1,1-2,2-0, queuing status is YZ (processing completed), YZ (processing), SF (processing), GH, GH, YZ, SF, YZ in order.

8. The eighth GH enters, at this time, the count value of the counter is more than 0, the priority level of GH is judged to be less than the recorded value of 2, the priority level of GH is 1, the count value is added with 1 and then is 3, and then consistency judgment of the priority level is carried out, 1-! The modification recorder records 1, time step 2-0, and the sixth YZ process is completed, and the count value is decremented by 1, which is 2. Service state: count value 2, recorder 1, time steps 0-1,1-2,2-0, queuing status YZ (processing completed), SF (processing), GH (processing), GH, YZ, SF, YZ in order.

9. The ninth GH enters, at this time, the count value of the counter is greater than 0, the priority level of the GH is judged to be 1=recorded value 1, the priority level of the GH is 3 after the count value is added with 1, then consistency judgment of the priority level is carried out, 1=1, the recorder is not modified for recording, the time step is 0-1, the seventh SF processing is completed, and the count value is reduced by 1 to 2. Service state: count value 2, recorder 1, time steps 0-1,1-2,2-0, queuing status is SF (processing completed), GH (in process), YZ, SF, YZ in order.

10. And when the tenth YZ enters, the count value of the counter is more than 0, the priority level of the counter and the record is judged, the priority level of the YZ is more than 3 and is more than 1, the entering condition is not met, queuing is carried out in the buffer pool again, the time steps are 1-2, the eighth GH processing is completed, and the count value is reduced by 1 and is 1. Service state: count value 1, recorder 1, time steps 1-2,2-0, queuing status is GH (processing completed), GH (in process), SF, YZ (re-queuing) in order.

11. The eleventh SF (namely the second SF re-queued before) enters, at the moment, the count value of the counter is more than 0, the priority level of the record is judged, the priority level of the SF is 2 & gtrecord value 1, the entering condition is not met, the buffer pool is re-queued, the time step is 2-0, the ninth GH processing is completed, and the count value is reduced by 1 to 0. Service state: count value 0, recorder 1, time step 2-0, queuing status is GH (processing completed), YZ, SF (re-queuing) in order.

12. The twelfth YZ (i.e. the third YZ in the previous queuing) enters, at this time, the count value of the counter=0, the counter is added with 1 to 1, and then consistency judgment of priority is carried out, and the priority of YZ is 3, 3-! =1, the modification recorder records 3, time steps 0-1. Service state: count value 1, recorder 3, time step 0-1, queuing state YZ (in process), YZ, SF in order.

13. The thirteenth YZ (i.e., the tenth YZ that was re-queued before) enters, at this time, the count value of the counter is > 0, the priority level of the YZ is 3=recording value 3, the counter is incremented by 1 and 2, then the consistency of the priority levels is judged, the priority level of the YZ is 3, 3=3, the recorder recording is not modified, and the time steps are 1-2. Service state: count value 2, recorder 3, time steps 0-1,1-2, queuing status YZ (in process), SF in order.

14. A fourteenth SF (i.e. the eleventh SF re-queued before) enters, at the moment, the count value of the counter is more than 0, the judgment of the priority level of the record is carried out, the priority level of the SF is 2 < the record value 3, the counter is added with 1 to 3, then the consistency judgment of the priority level is carried out, the priority level is more than 2-! =3, the modification recorder records 2, time steps 2-0, the twelfth YZ processing is completed, and the count value is decremented by 1 to 2. Service state: count value 2, recorder 2, time steps 0-1,1-2,2-0, queuing status is YZ (processing completed), YZ (processing), SF (processing in sequence).

15. YZ and SF in the treatment, after the treatment is finished, the counter is respectively reduced by 1, and the final reduction state is as follows:

service state: a counter: 0; a recorder: 2; time step: the method is free; queuing state: and no.

Data support description of this example:

the precondition is that: 10000 pieces of data with priority of 1, 10000 pieces of data with priority of 2 and 10000 pieces of data with priority of 3 are sent.

Test background: in the multithreading state, 1000000 tens of thousands of times are accumulated for one number, and the thread waits for 10 milliseconds at the 10 th time.

Test results:

priority control results: number 1 averages 19 milliseconds, number 2 averages 19 milliseconds, and number 3 averages 21 milliseconds;

and (5) adding no priority control result: number 1 averages 20 milliseconds, number 2 averages 20 milliseconds, and number 3 averages 20 milliseconds.

According to the method, when priority control is not adopted, all data operation time is 20 milliseconds, no data difference is caused, priority is not caused, emergency data is processed by the same person under the condition of high concurrency, but when the weak priority control scheme of the embodiment of the application is adopted, the data operation time with high priority can be obviously seen to be much less, so that the waiting time of the emergency data with high priority is shortened, the waiting time of the data with low priority is lengthened to a certain extent, the data with high priority is processed preferentially, and the data with low priority can be processed with a little longer waiting time.

In another aspect of the embodiments of the present application, a device for weak priority processing of high concurrency data is provided, as shown in fig. 4, including a delay setting module and a weak priority executing module.

The delay setting module is used for setting delay when the number of the data queued in the buffer pool exceeds a preset threshold value; wherein, the data has a preset priority level.

Preferably, the apparatus of this example further includes a threshold judgment module and a direct processing module.

In yet another aspect of the embodiments of the present application, there is provided an electronic device, including: at least one processor and memory; wherein the memory stores computer-executable instructions; executing computer-executable instructions stored in the memory at the at least one processor causes the at least one processor to perform the high concurrency data weak priority processing method described in the previous embodiments.

In yet another aspect of the embodiments of the present application, a computer readable storage medium has stored thereon a computer program, which when executed by a processor, controls a device on which the storage medium is located to perform the method for weak priority processing of high concurrency data described in the previous embodiments.

The foregoing description of the preferred embodiments of the invention is merely exemplary and is not intended to be exhaustive or limiting of the invention. It will be understood by those skilled in the art that various changes may be made and equivalents may be substituted for elements thereof without departing from the scope of the invention.

Claims

1. A method for weakly prioritizing high concurrency data, comprising:

when the quantity of the data queued in the buffer pool exceeds a preset threshold value, setting delay;

judging the count value of the current counter: if the count value is less than or equal to 0, the count value of the counter is added with 1, and then priority consistency judgment is carried out; if the count value is greater than 0, judging whether the priority level of the current data is superior or equal to the priority level recorded in the current recorder, if not, re-inputting the current data into a buffer pool for queuing, and if so, adding 1 to the count value of the counter for priority level consistency judgment;

2. The method for weak priority processing of high concurrency data according to claim 1, wherein when the delay does not exist/ends, it is determined whether the number of data queued in the buffer pool exceeds a preset threshold:

if the time delay exceeds the preset time delay value, setting time delay;

3. The method of weakly prioritized high-concurrency data processing of claim 1, wherein the queued data includes medical data.

4. The method of claim 3, wherein the medical data comprises one or more of out-patient registration data, out-patient charge data, and in-patient order data, the out-patient registration data having a priority level that is superior to the priority level of out-patient charge data, and the out-patient charge data having a priority level that is superior to the priority level of in-patient order data.

5. A high concurrency data weak priority processing apparatus, comprising:

the delay setting module is used for setting delay when the number of the data queued in the buffer pool exceeds a preset threshold value;

judging the count value of the current counter: if the count value is equal to 0, the count value of the counter is added with 1, and then priority consistency judgment is carried out; if the count value is greater than 0, judging whether the priority level of the current data is superior or equal to the priority level recorded in the current recorder, if not, re-inputting the current data into a buffer pool for queuing, and if so, adding 1 to the count value of the counter for priority level consistency judgment;

6. The apparatus for weak priority processing of high concurrency data according to claim 5, further comprising a threshold value judging module and a direct processing module;

if the data is not processed, the data enters a direct processing module for execution, the direct processing module directly processes the first data according to the queuing sequence, then the processing flow of the first data is ended, and the data jumps to a threshold value judging module for execution before the next data is processed.

7. The high concurrency data low priority processing device of claim 5, wherein the queued data comprises medical data.

8. The high concurrency data low priority processing device of claim 7, wherein the medical data comprises one or more of out-patient registration data, out-patient billing data, in-patient order data, the out-patient registration data having a priority level that is superior to the priority level of out-patient billing data, the out-patient billing data having a priority level that is superior to the priority level of in-patient order data.

9. An electronic device, comprising: at least one processor and memory; wherein the memory stores computer-executable instructions; computer-executable instructions stored in the memory are executed on the at least one processor to cause the at least one processor to perform the high concurrency data weak priority processing method of any one of claims 1-4.

10. A computer readable storage medium having stored thereon a computer program, which when run by a processor controls a device in which the storage medium is located to perform the high concurrency data weak priority handling method according to any one of claims 1-4.