CN116013085B - Data processing system for obtaining target traffic flow - Google Patents
Data processing system for obtaining target traffic flow Download PDFInfo
- Publication number
- CN116013085B CN116013085B CN202310098003.2A CN202310098003A CN116013085B CN 116013085 B CN116013085 B CN 116013085B CN 202310098003 A CN202310098003 A CN 202310098003A CN 116013085 B CN116013085 B CN 116013085B
- Authority
- CN
- China
- Prior art keywords
- data processing
- target
- list
- task
- obtaining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Traffic Control Systems (AREA)
Abstract
The invention provides a data processing system for acquiring target traffic flow, which comprises: a processor and a memory storing a computer program which, when executed by the processor, performs the steps of: the method comprises the steps of receiving a traffic flow obtaining request, obtaining an initial task list sent by a first data platform and sending the initial task list to N first data processing platforms, obtaining a target task set, processing the first task list by adopting a first rule to obtain a first candidate function list, processing the target task list by adopting a second rule to obtain a first target traffic flow list when the time for executing the first rule by the first data processing platform is not less than a first preset threshold value, processing the target task by adopting the first rule to obtain a second candidate function list, and repeatedly executing and obtaining the target traffic flow list sent by the first data processing platform.
Description
Technical Field
The invention relates to the technical field of traffic application, in particular to a data processing system for acquiring target traffic flow.
Background
Along with the development of economy and the continuous promotion of the living standard of people, the motor vehicles on the traffic road are more and more, the phenomenon causes the pressure of the traffic road of the city to be continuously increased, the congestion becomes a very common and puzzled phenomenon in the traveling of people, how to predict the traffic flow of the traffic road becomes a current hot research problem, the congestion condition of the traffic road can be known in advance through accurately predicting the traffic flow, and measures are taken in advance to avoid the situation of accident occurrence caused by the congestion phenomenon.
In the prior art, the method for acquiring the target traffic flow comprises the following steps: the historical vehicle traffic data in the road section to be predicted is extracted, the sample flow data is summarized according to lanes, the flow change floating values of each lane in different time periods are calculated on different platforms respectively, the historical vehicle flow data is counted, and finally the vehicle flow predicted value is obtained.
In summary, the above method for acquiring the target vehicle flow has the following problems: the system can not be subdivided into each lane, training of different models can not be carried out according to different traffic roads, a plurality of tasks can not be executed by the data processing platform, and the model is adjusted without considering abnormal changes, so that the accuracy of the obtained target traffic flow is lower.
Disclosure of Invention
Aiming at the technical problems, the invention adopts the following technical scheme: a data processing system for acquiring a target vehicle flow, the system comprising: a processor and a memory storing a computer program which, when executed by the processor, performs the steps of:
s100, receiving a vehicle flow acquisition request, and acquiring an initial task list sent by a first data platform.
S200, sending the initial task list to N first data processing platforms to obtain a first data processing platform list B= { B 1 ，B 2 ，……，B j ，……，B N Target task set B ' = { B ' corresponding to B } and B ' 1 ，B' 2 ，……，B' j ，……，B' N }，B j For the j-th first data processing platform, B' j For the target task list corresponding to the j-th first data processing platform, j= … … N, where N is the number of first data processing platforms.
S300，B j For B' j Processing by adopting a first rule to obtain B j Transmitted first candidate function list RB j ＝{RB' j1 ，RB' j2 ，……，RB' jy ，……，RB' jq }，RB' jy Is B j The y first candidate function is transmitted, y= … … q, q being the number of first candidate functions.
S400, when T 1 j ≥T 0 Time B j For RB' j The corresponding target task is processed by adopting a second rule to obtain B j Sent first target traffic list QB j ＝{QB' j1 ，QB' j2 ，……，QB' jd ，……，QB' ja }，RB' jd Is B j The d first target traffic flow, d= … … a, a is the number of the first target traffic flows, wherein a is less than or equal to q, T 1 j Is B j For B' j Time of processing with first rule, T 0 Is a first preset time threshold.
S500, when T 2 j When not less than T', B j For B' j Processing with the first rule to obtain B j A second list of candidate functions to be transmitted, wherein T 1 j Is B j For B' j And adopting the time processed by the second rule, wherein T' is a second preset time threshold.
S600, repeatedly executing S400-S500 to obtain B j Transmitted B' j Corresponding target traffic list EB j ＝{EB j1 ，EB j2 ，……，EB ju ，……，EB jf }，EB ju Is B' j The target traffic flow corresponding to the u-th target task.
The invention provides a data processing system for acquiring target traffic flow, which comprises: a processor and a memory storing a computer program which, when executed by the processor, performs the steps of: the method comprises the steps of receiving a traffic flow acquisition request, acquiring an initial task list sent by a first data processing platform, sending the initial task list to N first data processing platforms, acquiring a first data processing platform list and a target task set corresponding to the first data processing platform list, wherein the target task set comprises a plurality of target task lists, the first data processing platform processes the target task list by adopting a first rule, acquiring a first candidate function list sent by the first data processing platform, when the time of the first data processing platform for processing the target task list by adopting the first rule is not smaller than a first preset threshold, the first data processing platform processes the target task list by adopting a second rule to acquire the first target traffic flow list sent by the first data processing platform, wherein the number of the first target traffic flows in the first target traffic flow list is not larger than the number of the first candidate functions in the first candidate function list, when the time of the first data processing platform for processing the initial task by adopting the second rule exceeds a second preset time threshold, the first data processing platform processes the target task by adopting the first rule, so that the first candidate functions can be acquired by adopting the first rule, the first data processing platform can acquire the target traffic flow lists according to the different traffic flow sets, the traffic flow can be processed by the first data processing platform, and the traffic flow can be better than the traffic flow of the traffic models, and the traffic models can be better acquired by the traffic models, and the traffic models can be better processed according to the traffic models, and the traffic models can be better acquired by the traffic models, and can not be better processed by executing the traffic models, and the traffic models according to the traffic models.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flowchart of an executing computer program of a data processing system for obtaining a target traffic flow according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to fall within the scope of the invention.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or server that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example 1
The present embodiment provides a data processing system for acquiring a target traffic flow, the system including: a processor and a memory storing a computer program which, when executed by the processor, performs the steps of, as shown in fig. 1:
s100, receiving a vehicle flow acquisition request, and acquiring an initial task list sent by a first data platform.
Specifically, the traffic flow obtaining request is a request for obtaining traffic flow sent by the first data platform.
Specifically, the first data platform is a data platform corresponding to an acquirer of an initial task, wherein the data platform is a platform for receiving data and storing the data.
Further, the initial task list comprises a plurality of initial tasks, wherein the initial tasks are used for predicting the traffic flow of a target lane, and the target lane is a lane to be detected provided by a user.
S200, sending the initial task list to N first data processing platforms to obtain a first data processing platform list B= { B 1 ，B 2 ，……，B j ，……，B N Target task set B ' = { B ' corresponding to B } and B ' 1 ，B' 2 ，……，B' j ，……，B' N }，B j For the j-th first data processing platform, B' j For the target task list corresponding to the j-th first data processing platform, j= … … N, where N is the number of first data processing platforms.
Specifically, the first data processing platform is a data processing platform for receiving an initial task and generating candidate functions according to the initial task.
Specifically, the step S200 further includes the following steps:
s201, acquiring an initial task list A= { A 1 ，A 2 ，……，A i ，……，A n Initial time list A corresponding to A 0 ＝{A 0 1 ，A 0 2 ，……，A 0 i ，……，A 0 n }，A i For the ith initial task, A i Is A i Corresponding initial time, i= … … n, n being the number of initial tasks.
Specifically, the initial time is the time required for completing the initial task.
S203, acquiring a first data processing platform list B= { B 1 ，B 2 ，……，B j ，……，B N Candidate time list B corresponding to B 0 ＝{B 0 1 ，B 0 2 ，……，B 0 j ，……，B 0 N }，B j For the j-th first data processing platform, B 0 j Is B j Corresponding candidate times.
Specifically, the candidate time is the maximum time that the first data processing platform can carry for executing a task, for example, the candidate time of a certain first data processing platform is 16 seconds, which can be understood as: a first data processing platform can complete tasks for 16 seconds and less than 16 seconds at a time, for example, a task for 15 seconds can be completed at a time, and a task for 8 seconds and a task for 6 seconds can be completed at a time.
Further, the one-time task may include a number of tasks.
Specifically, B j Is Null; it can be understood that: the initial first data processing platform does not include any tasks therein.
S205, when N is less than or equal to N, acquiring a target task set B ' = { B ' corresponding to B ' 1 ，B' 2 ，……，B' j ，……，B' N }，B' j ＝A j And A is 0 j ≤B 0 j Wherein A is j For the j-th initial task, A 0 j Is A j Corresponding initial time.
S207, when n >When N, a target task set B ' = { B ' corresponding to B is obtained ' 1 ，B' 2 ，……，B' j ，……，B' N }，B' j Is B j A corresponding target task list.
Specifically, in S207, the following steps are further included:
s2071, whenIn this case, the target task set B ' = { B ' corresponding to B is acquired ' 1 ，B' 2 ，……，B' j ，……，B' N }，B' j ＝{{B' j1 ，B' j2 ，……，B' jr ，……，B' js(j) }, wherein->TB' jr Is B' jr Corresponding initial time, B' jr Is B j The method comprises the steps that an r-th target task in a corresponding target task list is an initial task obtained from an initial task list; it can be understood that: when the time required for completing all the initial tasks is not more than the candidate time of all the first data processing platforms, each first data processing platform performs task preemption according to the candidate time so that the time required by the acquired tasks does not exceed the candidate time.
Specifically, those skilled in the art know that any method of preempting tasks according to platform resource allocation in the prior art falls into the protection scope of the present invention, and is not described herein.
S2073, whenIn this case, the first target task set fb= { FB corresponding to B is acquired 1 ，FB 2 ，……，FB j ，……，FB N }，FB j Is B j And a corresponding first target task list.
Specifically, the acquiring manner of the first target task set may refer to the acquiring manner of the target task set in S2071.
Further, the first target task list includes a plurality of first target tasks, and the first target tasks are initial tasks obtained from an initial task list.
S2075, according to FB, obtaining a candidate task list C= { C 1 ，C 2 ，……，C e ，……，C t }，C e For the e candidate task, e= … … t, where t is the number of candidate tasks, where the candidate task is an initial task after deleting the first target task list from the initial task list.
S2077, according to B and C, obtaining a second target task set DB = { DB corresponding to B 1 ，DB 2 ，……，DB j ，……，DB N }，DB j Is B j A corresponding second target task list, wherein in S2077, further comprises the steps of:
s20771, when C 0 e ≤B 0 jg -T 0 At the time, C e Insert into DB j And B is j Using the first execution mode to DB j Processing is performed, wherein B 0 jg Is FB j Initial time corresponding to g first target task, C 0 e Is C e Corresponding initial time, T 0 A preset time threshold value;
specifically, the first execution mode is a polling execution mode; it can be understood that: when the running time of a certain first target task in the jth first data processing platform is 15 seconds, and the running time of a certain candidate task in the candidate task list is 3 seconds, inserting the candidate task into the first target task being executed, executing the first target task for 1 second, executing the candidate task for 1 second, executing the first target task for 1 second, and executing the candidate task for 1 second, … ….
Further, those skilled in the art know that any polling implementation manner in the prior art falls within the protection scope of the present invention, and is not described herein;
specifically, those skilled in the art know that the selection of the preset time threshold value can be performed according to the actual requirement, which falls into the protection scope of the present invention, and will not be described herein.
When the sum of the initial time of the initial tasks exceeds the candidate time corresponding to the first data processing platform, the first target task is acquired from the initial task list so that the first data processing platform can work in a full load state, and when the initial time corresponding to the candidate task is shorter and the time required by a certain first data processing platform to execute the task is longer, the candidate task is inserted into the first data processing platform to be executed alternately with the task being executed in the first data processing frequency table, so that the predicted tasks can be balanced, effective execution of all the initial tasks is ensured, and the operation efficiency of the data processing platform is improved.
S20773, when C 0 e ＞B 0 jg -T 0 And when the candidate task in the C is inserted into the DB according to the candidate principle, and the B adopts a second execution mode to process the DB.
Specifically, the candidate principle is that task selection is performed according to a first candidate time when the first data processing platform finishes processing a part of the corresponding first target task.
Further, the first candidate time is a time obtained by subtracting a time for completing the first target task in real time from the candidate time.
Specifically, the second execution mode is an execution mode executed according to the priority order of the tasks, which can be understood as: and processing the first target task, and sequentially processing the second target task after the first data processing platform processes the first target task.
According to the method, the time of the initial task is compared with the time corresponding to the first data processing platform, different processing modes are carried out according to different conditions, the execution balance of the task can be guaranteed, the effective execution of the task is guaranteed by fully utilizing resources, the operation efficiency of the data processing platform is improved, and the accuracy of the acquired target traffic flow is higher.
S2079, according to FB and DB, obtaining a target task set B ' = { B ' corresponding to B ' 1 ，B' 2 ，……，B' j ，……，B' N And (B) wherein' j ＝{FB j ，DB j }。
S300，B j For B' j Processing by adopting a first rule to obtain B j Transmitted first candidate function list RB j ＝{RB' j1 ，RB' j2 ，……，RB' jy ，……，RB' jq }，RB' jy Is B j The y first candidate function is transmitted, y= … … q, q being the number of first candidate functions.
Specifically, the first rule is a rule for acquiring a function.
Specifically, the system also comprises a sample traffic road information database.
Specifically, the sample traffic road information database comprises a sample lane coding list and a sample lane information set corresponding to the sample lane coding list.
Further, the sample lane coding list includes a plurality of sample lane codes, where the sample lane codes are codes corresponding to the sample lanes, and those skilled in the art know that any method for coding lanes in the prior art falls within the protection scope of the present invention, and is not described herein again; for example, the method of encoding the lane is: acquiring an innermost lane of the city K, and encoding the innermost lane of the city K as K 1 The lanes are coded as K outwards in turn 2 、K 3 、……、K w 。
Further, the sample lanes include lanes corresponding to each city.
Specifically, the sample lane information set includes a sample traffic flow for each sample lane over a sample period of time.
Further, those skilled in the art know that the selection of the sample time period can be performed according to the actual requirement, which falls within the protection scope of the present invention, and will not be described herein.
Further, the sample traffic flow is obtained by using a traffic flow detection method on the sample traffic image.
Further, the sample traffic image is a traffic image obtained by decoding a sample traffic video, and the sample traffic image is a video shot by a traffic intersection camera in real time; it can be understood that: two traffic intersections are arranged at two ends of one lane, and the sample traffic image is obtained by decoding videos shot by cameras of the two traffic intersections at two ends.
Further, those skilled in the art know that any method for decoding a video to obtain an image in the prior art falls within the protection scope of the present invention, and is not described herein.
Further, those skilled in the art will know that any vehicle flow detection method in the prior art falls within the protection scope of the present invention, and will not be described herein.
Specifically, the system further comprises a sample model database, wherein the sample model database comprises sample models divided according to a plurality of dimensions, the dimensions comprise algorithm types and super parameters, and for example, the sample models divided according to the algorithm types comprise: random forest, logistic regression, etc., and the sample model divided according to the super parameters comprises basic historical statistical characteristics, real-time feedback characteristics, etc.
Specifically, the step S300 includes the following steps:
s301 from B' j Obtain PB' j ＝{B' j1 ，B' j2 ，……，B' jy ，……，B' jq }，B' jy Is B' j Is the y-th target task.
Specifically, deltaB' j1 ＞ΔB' j2 ＞……＞ΔB' jy ＞……＞ΔB' jq Wherein ΔB' jy Is B' jy Is a priority of (a).
Further, the priority degree is the sequence of the target tasks, which can be understood as: the earlier the time of the obtained target task is, the greater the priority of the target task is.
S303, obtaining B' jy Corresponding target lane coding EB' jy 。
S305, acquiring a sample lane coding list according to the sample lane traffic information databaseE＝{E 1 ，E 2 ，……，E μ ，……，E ζ }，E μ Mu = 1 … … ζ for the mu-th sample lane code, ζ being the number of sample lane codes.
S307, when EB' jy And E is connected with μ When the two are consistent, obtain E μ Corresponding sample lane information set FE μ 。
S309, according to FE μ Invoking a sample model in a sample model database to obtain RB' jy 。
Specifically, the sample model in the calling sample model database adopts a random mode.
Specifically, those skilled in the art know that any method of generating a function according to data and a model in the prior art falls within the protection scope of the present invention, and is not described herein.
Above-mentioned, compare the target lane that the target task corresponds with the sample lane code list in the sample lane traffic information database, call the sample model in the sample model database after the matching is accomplished, subdivide to each lane, can carry out the training of different models according to the difference of traffic road for the degree of accuracy of the target traffic flow who obtains is higher.
S400, when T 1 j ≥T 0 Time B j For RB' j The corresponding target task is processed by adopting a second rule to obtain B j Sent first target traffic list QB j ＝{QB' j1 ，QB' j2 ，……，QB' jd ，……，QB' ja }，RB' jd Is B j The d first target traffic flow, d= … … a, a is the number of the first target traffic flows, wherein a is less than or equal to q, T 1 j Is B j For B' j Time of processing with first rule, T 0 Is a first preset time threshold.
Specifically, the second rule is a rule for acquiring the traffic flow.
Specifically T 0 The value of (2) is 30-60 minutes.
Further, the methodThose skilled in the art will recognize that T can be performed according to actual requirements 0 All falling within the protection scope of the present invention and will not be described herein.
Specifically, in S400, the following steps are further included:
s401 from B' j B 'in (B)' j1 Begin to obtain B 'in turn' jy 。
S403, from RB j Obtain B' jy Corresponding candidate function list D jy ＝{D 1 jy ，D 2 jy ，……，D θ jy ，……，D β jy }，D θ jy Is B' jy Corresponding θ candidate function, θ= … … β, β is B' jy The number of corresponding candidate functions.
Specifically, the candidate function is a function for acquiring the target vehicle flow, for example, a candidate function such as a binary cubic function.
S405, when β=1, use D 1 jy For B' jy Processing to obtain B' jy Corresponding target traffic flow.
Specifically, those skilled in the art know that any method for predicting a function known in the prior art falls within the protection scope of the present invention, and is not described herein.
S407, when beta > 1, use D jy For B' jy Processing to obtain B' jy Corresponding candidate traffic flow list L' jy ＝{L 1 jy ，L 2 jy ，……，L θ jy ，……，L β jy }, wherein L θ jy To utilize D θ jy For B' jy And processing the obtained candidate traffic flow.
Specifically, when beta is more than 1, it is indicated that a history function is stored in the first data platform, an abnormal event exists in a target lane corresponding to the obtained initial task, and a sample model is continuously obtained to update the candidate function.
Further, the history function is a function obtained by processing the target lane before.
Further, the abnormal event is an event affecting the traffic flow existing in the road, such as an abnormal event of bad weather, road repair, traffic accident, etc.
The method is not limited to a function when the initial task is processed to obtain the target traffic flow, and the candidate function is updated by selecting the sample model according to the change of the target lane, so that the accuracy of the obtained target traffic flow is improved.
S409, according to L' jy Obtaining B' jy Corresponding target traffic flow HB' jy Wherein HB' jy Meets the following conditions:
wherein lambda is θ The priority is preset for the theta.
Specifically, 0 < lambda 1 ＜……＜λ θ ＜……＜λ β ＜1。
Further, those skilled in the art know that the selection of the preset priority level can be performed according to the actual requirement, which falls within the protection scope of the present invention, and will not be described herein.
S500, when T 2 j When not less than T', B j For B' j Processing with the first rule to obtain B j A second list of candidate functions to be transmitted, wherein T 1 j Is B j For B' j And adopting the time processed by the second rule, wherein T' is a second preset time threshold.
Specifically, the method for obtaining the second candidate function list may refer to the method for obtaining the first candidate function list in S300.
Specifically, the value range of T' is 10 minutes to 20 minutes.
Further, those skilled in the art know that T' may be selected according to actual requirements, which all fall within the protection scope of the present invention, and will not be described herein.
S600, repeatedly executing S400-S500 to obtain B j Transmitted B' j Corresponding target traffic list EB j ＝{EB j1 ，EB j2 ，……，EB ju ，……，EB jf }，EB ju Is B' j The target traffic flow corresponding to the u-th target task.
Specifically, EB j The acquisition method of (a) may refer to the acquisition method of the first target traffic list in S400.
The first data processing platform processes two tasks, one of the two tasks generates a function according to the task and sample data, and the other of the two tasks processes the initial task according to the function to obtain the target traffic flow, and the data processing platform can execute a plurality of tasks to adjust the model by taking abnormal changes into consideration, so that the accuracy of the obtained target traffic flow is higher.
The data processing system for obtaining a target traffic flow provided in this embodiment includes: a processor and a memory storing a computer program which, when executed by the processor, performs the steps of: the method comprises the steps of receiving a traffic flow acquisition request, acquiring an initial task list sent by a first data processing platform, sending the initial task list to N first data processing platforms, acquiring a first data processing platform list and a target task set corresponding to the first data processing platform list, wherein the target task set comprises a plurality of target task lists, the first data processing platform processes the target task list by adopting a first rule, acquiring a first candidate function list sent by the first data processing platform, when the time of the first data processing platform for processing the target task list by adopting the first rule is not smaller than a first preset threshold, the first data processing platform processes the target task list by adopting a second rule to acquire the first target traffic flow list sent by the first data processing platform, wherein the number of the first target traffic flows in the first target traffic flow list is not larger than the number of the first candidate functions in the first candidate function list, when the time of the first data processing platform for processing the initial task by adopting the second rule exceeds a second preset time threshold, the first data processing platform processes the target task by adopting the first rule, so that the first candidate functions can be acquired by adopting the first rule, the first data processing platform can acquire the target traffic flow lists according to the different traffic flow sets, the traffic flow can be processed by the first data processing platform, and the traffic flow can be better than the traffic flow of the traffic models, and the traffic models can be better acquired by the traffic models, and the traffic models can be better processed according to the traffic models, and the traffic models can be better acquired by the traffic models, and can not be better processed by executing the traffic models, and the traffic models according to the traffic models.
Example two
The present embodiment provides a data processing system for acquiring a target traffic flow, the system including: a sample traffic road information database, a processor and a memory storing a computer program which, when executed by the processor, performs the steps of:
s100, receiving a vehicle flow acquisition request, and acquiring an initial task list sent by a first data platform.
Specifically, the traffic flow obtaining request is a request for obtaining traffic flow sent by the first data platform.
Specifically, the first data platform is a data platform corresponding to an acquirer of an initial task, wherein the data platform is a platform for receiving data and storing the data.
Further, the initial task list comprises a plurality of initial tasks, wherein the initial tasks are used for predicting the traffic flow of a target lane, and the target lane is a lane to be detected provided by a user.
S200, sending the initial task list to N first data processing platforms, and obtaining a target task set B ' = { B ' corresponding to the first data processing platforms ' 1 ，B' 2 ，……，B' j ，……，B' N }，B' j For the target task list corresponding to the j-th first data processing platform, j= … … N, where N is the number of first data processing platforms.
Specifically, the first data processing platform is a data processing platform for receiving an initial task and generating candidate functions according to the initial task.
Specifically, the method of obtaining B 'in S200 may refer to the method of obtaining B' in S200 in the first embodiment.
S300, according to B', obtaining a candidate function set D= { D sent by the first data processing platform 1 ，D 2 ，……，D j ，……，D N And send D to the second data platform, wherein D j And a candidate function list sent for the j-th first data processing platform.
Specifically, the second data platform is a data platform for receiving candidate functions and storing the candidate functions.
Further, in S300, the method further includes the following steps:
s301, acquiring an xth target task B 'in a jth first data processing platform according to B' jx 。
S303, according to the sample traffic road information database and B' jx Obtaining B' jx Corresponding candidate function D jx And will D jx Inserted into D j Is a kind of medium.
Specifically, the sample traffic road information database is consistent with the sample traffic road information database of the embodiment.
Specifically, the system further comprises a sample model database, wherein the sample model database comprises sample models divided according to a plurality of dimensions, the dimensions comprise algorithm types and super parameters, and for example, the sample models divided according to the algorithm types comprise: random forest, logistic regression, etc., and the sample model divided according to the super parameters comprises basic historical statistical characteristics, real-time feedback characteristics, etc.
Specifically, D is obtained in S303 by the following steps jx ：
S3031, obtain B' jx Corresponding target lane coding EB' jx 。
S3033, obtaining a sample lane coding list E= { E according to the sample lane traffic information database 1 ，E 2 ，……，E μ ，……，E ζ }，E μ Mu = 1 … … ζ for the mu-th sample lane code, ζ being the number of sample lane codes.
S3035, when EB' jx And E is connected with μ When the two are consistent, obtain E μ Corresponding sample lane information set FE μ 。
S3037, according to FE μ Invoking the sample model in the sample model database to obtain D jx 。
Specifically, the sample model in the calling sample model database adopts a random mode.
Specifically, those skilled in the art know that any method of generating a function according to data and a model in the prior art falls within the protection scope of the present invention, and is not described herein.
Above-mentioned, compare the target lane that the target task corresponds with the sample lane code list in the sample lane traffic information database, call the sample model in the sample model database after the matching is accomplished, subdivide to each lane, can carry out the training of different models according to the difference of traffic road for the degree of accuracy of the target traffic flow who obtains is higher.
S305, D to be generated j And sending the data to the second data platform.
According to the method, the initial task sent by the first data platform is utilized to combine the initial task with the data in the sample lane traffic information database, the model in the sample model database is called to generate the candidate function set, the data and the task are separated, the workload of the data processing platform is reduced, and the operation efficiency of the data processing platform is improved.
S400, the initial task list is sent to M second data processing platforms, and the designated task set corresponding to the second data processing platforms is obtained.
Specifically, the designated task set comprises designated task lists corresponding to each second data processing platform, wherein each designated task list comprises a plurality of designated tasks, and the designated tasks are initial tasks obtained from an initial task list.
Specifically, the second data processing platform is a data processing platform for receiving an initial task and acquiring a target traffic flow corresponding to the initial task according to the initial task.
Specifically, the acquiring manner of the designated task set corresponding to the second data processing platform may refer to the acquiring manner of the target task set corresponding to the first data processing platform in S200.
S500, calling D to acquire a target vehicle flow list corresponding to the initial task list sent by the second data processing platform according to the designated task set.
Specifically, the method for obtaining the target traffic flow in S500 may refer to the method for obtaining the first target traffic flow in S400 in the first embodiment.
According to the method, the process of generating the function by using the sample information and the process of predicting are executed in different data processing platforms, so that each data processing platform executes an independent task, the workload of the data processing platform is reduced, and the operation efficiency of the data processing platform is improved.
The data processing system for obtaining a target traffic flow provided in this embodiment includes: a sample traffic road information database, a processor and a memory storing a computer program which, when executed by the processor, performs the steps of: the method comprises the steps of receiving a traffic flow obtaining request, obtaining an initial task list sent by a first data platform, sending the initial task list to N first data processing platforms, obtaining a target task set corresponding to the first data processing platform, obtaining a candidate function set sent by the first data processing platform according to the target task set corresponding to the first data processing platform, sending the candidate function set to a second data processing platform, sending the initial task list to M second data processing platforms, obtaining a designated task set corresponding to the second data processing platform, and calling the candidate function set according to the designated task set to obtain a target traffic flow list corresponding to the initial task list sent by the second data processing platform.
While certain specific embodiments of the invention have been described in detail by way of example, it will be appreciated by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the invention. Those skilled in the art will also appreciate that many modifications may be made to the embodiments without departing from the scope and spirit of the invention. The scope of the invention is defined by the appended claims.
Claims (8)
1. A data processing system for acquiring a target vehicle flow, the system comprising: a processor and a memory storing a computer program which, when executed by the processor, performs the steps of:
s100, receiving a traffic flow acquisition request, and acquiring an initial task list sent by a first data platform;
s200, sending the initial task list to N first data processing platforms to obtain a first data processing platform list B= { B 1 ，B 2 ，……，B j ，……，B N Target task set B ' = { B ' corresponding to B } and B ' 1 ，B' 2 ，……，B' j ，……，B' N }，B j For the j-th first data processing platform, B' j For a target task list corresponding to the j-th first data processing platform, j= … … N, where N is the number of first data processing platforms;
S300，B j for B' j Processing by adopting a first rule to obtain B j Transmitted first candidate function list RB j ＝{RB' j1 ，RB' j2 ，……，RB' jy ，……，RB' jq }，RB' jy Is B j The y first candidate function is sent, y= … … q, and q is the number of the first candidate functions; wherein, in S300, the method further comprises the following steps:
s301 from B' j Obtain PB' j ＝{B' j1 ，B' j2 ，……，B' jy ，……，B' jq }，B' jy Is B' j A y-th target task;
s303, obtaining B' jy Corresponding target lane coding EB' jy ；
S305, according to the number of the traffic information of the sample laneThe database acquires a sample lane coding list E= { E 1 ，E 2 ，……，E μ ，……，E ζ }，E μ For the mu-th sample lane code, mu= … … ζ, ζ is the number of sample lane codes;
s307, when EB' jy And E is connected with μ When the two are consistent, obtain E μ Corresponding sample lane information set FE μ ；
S309, according to FE μ Invoking a sample model in a sample model database to obtain RB' jy ；
S400, when T 1 j ≥T 0 Time B j For RB' j The corresponding target task is processed by adopting a second rule to obtain B j Sent first target traffic list QB j ＝{QB' j1 ，QB' j2 ，……，QB' jd ，……，QB' ja }，RB' jd Is B j The d first target traffic flow, d= … … a, a is the number of the first target traffic flows, wherein a is less than or equal to q, T 1 j Is B j For B' j Time of processing with first rule, T 0 A first preset time threshold; wherein, in S400, the following steps are further included:
s401 from B' j B 'in (B)' j1 Begin to obtain B 'in turn' jy ；
S403, from RB j Obtain B' jy Corresponding candidate function list D jy ＝{D 1 jy ，D 2 jy ，……，D θ jy ，……，D β jy }，D θ jy Is B' jy Corresponding θ candidate function, θ= … … β, β is B' jy The number of corresponding candidate functions;
s405, when β=1, use D 1 jy For B' jy Processing to obtain B' jy A corresponding target traffic flow;
s407, when beta > 1, use D jy For B' jy Processing to obtain B' jy Corresponding candidate traffic flow listL' jy ＝{L 1 jy ，L 2 jy ，……，L θ jy ，……，L β jy }, wherein L θ jy To utilize D θ jy For B' jy Processing the obtained candidate traffic flow;
s409, according to L' jy Obtaining B' jy Corresponding target traffic flow HB' jy Wherein HB' jy Meets the following conditions:
wherein lambda is θ Presetting a priority for the theta;
s500, when T 2 j When not less than T', B j For B' j Processing with the first rule to obtain B j A second list of candidate functions to be transmitted, wherein T 2 j Is B j For B' j Adopting a second rule to process time, wherein T' is a second preset time threshold;
s600, repeatedly executing S400-S500 to obtain B j Transmitted B' j Corresponding target traffic list EB j ＝{EB j1 ，EB j2 ，……，EB ju ，……，EB jf }，EB ju Is B' j The target traffic flow corresponding to the u-th target task.
2. The data processing system for obtaining a target traffic volume according to claim 1, wherein the first data platform is a data platform corresponding to an obtaining party of an initial task.
3. The data processing system for obtaining a target vehicle flow of claim 1, wherein the first data processing platform is a data processing platform for receiving an initial task and generating a candidate function based on the initial task.
4. The data processing system for obtaining a target vehicle flow according to claim 1, further comprising a sample traffic road information database.
5. The data processing system for obtaining a target vehicle flow according to claim 1, wherein Δb' j1 ＞ΔB' j2 ＞……＞ΔB' jy ＞……＞ΔB' jq Wherein ΔB' jy Is B' jy Is a priority of (a).
6. A data processing system for obtaining a target vehicle flow according to claim 1, wherein T 0 The value of (2) is 30-60 minutes.
7. The data processing system for obtaining a target traffic flow according to claim 1, wherein when β > 1, it is indicated that a history function is stored in the first data platform, and an abnormal event exists in the target lane corresponding to the obtained initial task.
8. A data processing system for obtaining a target vehicle flow according to claim 1, wherein 0 < λ 1 ＜……＜λ θ ＜……＜λ β ＜1。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310098003.2A CN116013085B (en) | 2023-02-03 | 2023-02-03 | Data processing system for obtaining target traffic flow |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310098003.2A CN116013085B (en) | 2023-02-03 | 2023-02-03 | Data processing system for obtaining target traffic flow |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116013085A CN116013085A (en) | 2023-04-25 |
| CN116013085B true CN116013085B (en) | 2023-09-22 |
Family
ID=86021174
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310098003.2A Active CN116013085B (en) | 2023-02-03 | 2023-02-03 | Data processing system for obtaining target traffic flow |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116013085B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108986455A (en) * | 2018-06-29 | 2018-12-11 | 淮阴工学院 | The preferential lane the HOV dynamic management-control method of share-car under a kind of car networking environment |
| CN111311005A (en) * | 2020-02-21 | 2020-06-19 | 江苏苏宁物流有限公司 | Distribution path planning method, distribution path planning device, distribution path planning medium and computer equipment |
| WO2021142917A1 (en) * | 2020-01-15 | 2021-07-22 | 深圳大学 | Multi-depot vehicle routing method, apparatus, computer device and storage medium |
| CN115150782A (en) * | 2022-06-29 | 2022-10-04 | 中国第一汽车股份有限公司 | Service quality monitoring method and device for Internet of vehicles cloud service, monitoring platform, server and storage medium |
-
2023
- 2023-02-03 CN CN202310098003.2A patent/CN116013085B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108986455A (en) * | 2018-06-29 | 2018-12-11 | 淮阴工学院 | The preferential lane the HOV dynamic management-control method of share-car under a kind of car networking environment |
| WO2021142917A1 (en) * | 2020-01-15 | 2021-07-22 | 深圳大学 | Multi-depot vehicle routing method, apparatus, computer device and storage medium |
| CN111311005A (en) * | 2020-02-21 | 2020-06-19 | 江苏苏宁物流有限公司 | Distribution path planning method, distribution path planning device, distribution path planning medium and computer equipment |
| CN115150782A (en) * | 2022-06-29 | 2022-10-04 | 中国第一汽车股份有限公司 | Service quality monitoring method and device for Internet of vehicles cloud service, monitoring platform, server and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116013085A (en) | 2023-04-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110334885B (en) | Task data processing method and device | |
| CN113326126A (en) | Task processing method, task scheduling device and computer equipment | |
| CN111383005B (en) | Digital currency flow direction tracking method and device | |
| CN110287250B (en) | User grade quantification method and device | |
| US9953264B2 (en) | Smart process management | |
| CN116013085B (en) | Data processing system for obtaining target traffic flow | |
| Park et al. | Intuitionistic fuzzy optimized weighted geometric Bonferroni means and their applications in group decision making | |
| CN111444930B (en) | Method and device for determining prediction effect of two-classification model | |
| CN116071924B (en) | Data processing system for acquiring target traffic flow based on task allocation | |
| CN115798216B (en) | Data processing system for obtaining target traffic flow | |
| CN110516872B (en) | Information processing method and device, storage medium and electronic equipment | |
| US6813390B2 (en) | Scalable expandable system and method for optimizing a random system of algorithms for image quality | |
| CN111626098A (en) | Method, device, equipment and medium for updating parameter values of model | |
| CN109145207B (en) | Information personalized recommendation method and device based on classification index prediction | |
| CN110880117A (en) | False service identification method, device, equipment and storage medium | |
| CN110930106A (en) | Information processing method, device and system of online interview system | |
| CN112800037B (en) | Optimization method and device for engineering cost data processing | |
| CN115016950A (en) | Data analysis method and system based on multithreading model | |
| CN115034686A (en) | Job behavior quantification method and device, computer equipment and storage medium | |
| CN111611397A (en) | Information matching method and device, computer equipment and storage medium | |
| CN113065791A (en) | Navigation density determination method and device, computer equipment and storage medium | |
| CN112272358A (en) | Weather early warning method and device and electronic equipment | |
| CN112329968A (en) | Resource allocation method, device and storage medium | |
| CN115797849B (en) | Data processing system for determining abnormal behavior based on image | |
| CN116071770B (en) | Method, device, equipment and medium for general identification of form |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |