WO2022227105A1

WO2022227105A1 - Cooperative control system and method based on device-cloud fusion

Info

Publication number: WO2022227105A1
Application number: PCT/CN2021/092325
Authority: WO
Inventors: 陈升东; 秦佩; 袁峰
Original assignee: 广州中国科学院软件应用技术研究所
Priority date: 2021-04-30
Filing date: 2021-05-08
Publication date: 2022-11-03
Also published as: CN113327442B; CN113327442A

Abstract

A cooperative control system and method based on device-cloud fusion. The solution comprises a cloud cooperative control platform (101), an edge sensing analysis system (102) and a mobile terminal control system (103), wherein the mobile terminal control system (103) is installed on a technical facility of an intelligent connected vehicle or a road, and is used for collecting information and executing a cooperative control instruction; the edge sensing analysis system (102) is deployed on both sides of the road or a 5G service base station, and is used for collecting information and fusing information; and the cloud cooperative control platform (101) is deployed on a cloud platform, and is used for data management, service communication, and the generation of the cooperative control instruction. By means of the solution, online operation, model correction and real-time scheduling control are performed on data sensing calculation in a hybrid running process of an intelligent vehicle and a human-driven vehicle, so as to ensure that there is no divide in the information interaction between vehicles in two different driving modes, thereby improving the self-driving management efficiency.

Description

A collaborative control system and method based on terminal-cloud integration

technical field

The present invention relates to the technical field of assisted driving, and more particularly, to a collaborative control system and method based on terminal-cloud fusion.

Background technique

With the development of automotive assisted driving technology, human-machine collaborative control has been more applied to the field of automobile control. Human-machine collaborative driving has been realized on a large number of L2 and L3 intelligent vehicles. With the L4 level of intelligent driving vehicles With the development of the technology, more and more vehicles have the ability to complete driving autonomously.

However, the existing technologies are mainly focused on assisted driving. But for fully autonomous driving from the L4 and L5 levels, the vehicle needs to cope with a more complex road environment, plus the limitations of the autonomous vehicle's own perception. However, there is currently no reasonable autonomous driving that incorporates road information.

With the development of intelligent networked vehicles, it is bound to bring challenges to the existing road management model. The original urban traffic management system needs to deal with the data perception calculation and collaborative scheduling in the mixed operation of intelligent vehicles and human-driven vehicles. problem, thereby ensuring that there is no information exchange gap between vehicles with two different driving modes.

SUMMARY OF THE INVENTION

In view of the above problems, the present invention proposes a collaborative control system and method based on terminal-cloud fusion, which performs online computation, model correction, and real-time scheduling control for data-aware computing during the mixed operation of intelligent vehicles and human-driven vehicles. Ensure that there is no information exchange gap between vehicles with two different driving modes, and improve the efficiency of driverless management.

According to the first aspect of the embodiments of the present invention, a collaborative control system based on terminal-cloud integration is provided.

The described collaborative control system based on terminal-cloud integration specifically includes:

Cloud collaborative control platform, edge perception analysis system and mobile terminal control system; wherein, the mobile terminal control system is installed on the technical facilities of intelligent networked vehicles or roads to collect information and execute coordinated control instructions; the edge perception The analysis system is deployed on both sides of the road or 5G service base stations for information collection and information fusion, and the cloud collaborative control platform is deployed on the cloud platform for data management, business communication and generation of coordinated control instructions.

In one or more embodiments, preferably, the cloud collaborative control platform includes an algorithm model library, an algorithm training engine, a model distributor, and a collaborative scheduling/control engine;

The algorithm training engine is used to obtain the first perception data uploaded by the edge perception analysis system, perform algorithm training, and generate an operation model according to the minimum objective function value;

The algorithm model library is used to obtain the operation model generated by the algorithm training engine;

the model distributor, configured to transfer the operation model in the algorithm model library to the edge-aware analysis system;

The collaborative scheduling/control engine is used to perform state evaluation in real time according to the perception data, and give control instructions to the edge perception analysis system;

The collaborative scheduling/control engine is configured with an optimal computing scheduling decision algorithm;

A scheduling space S algorithm is configured in the model distributor.

In one or more embodiments, preferably, the cloud collaborative control platform further includes a service governance and open interface management module, a capability container management module, and an infrastructure and operating environment platform;

The service governance and open interface management module includes a service interface sub-module, an operation management sub-module, a distribution scheduling sub-module and a security management sub-module, and is used for collaborative management of multi-source heterogeneous data with the mobile terminal;

The capability container management module includes a data service sub-module, an intelligent algorithm and application sub-module, a micro-service architecture sub-module, a multi-source heterogeneous device management sub-module, and a terminal-cloud collaboration sub-module, which jointly complete the cross-business application services and dynamic information management. collaboration;

The infrastructure and operating environment platform are used for the performance support of storage, computing and data processing for the entire cloud collaborative control platform.

In one or more embodiments, preferably, the mobile terminal control system specifically includes: a local perception module, a local control module, a local upload module, a collaborative perception module, and a collaborative control module;

The local perception module collects data through sensors connected to the mobile terminal, and saves it as protocol data;

The local control module is configured to receive the control instruction information sent by the edge perception analysis system, and perform coordinated control according to the control instruction information;

The local uploading module is configured to store the data obtained by the local perception module as perception data in a fixed format and send it to the collaborative perception module and the edge perception analysis system;

The collaborative sensing module is used to determine the confidence level of the sensing data according to different sensing data types;

The cooperative control module is configured to obtain the control instruction issued by the edge perception analysis system.

In one or more embodiments, preferably, the edge perception analysis system specifically includes: a perception analysis module and a cooperative control module;

Wherein, the perception analysis module includes a cloud data transceiver, a deep learning engine, a road-end data collector, and a terminal data receiver;

Wherein, the collaborative control module includes a cloud control receiver, a decision controller, and a terminal control transmitter.

In one or more embodiments, preferably, the optimal computing scheduling decision algorithm specifically includes:

Get data input scale and calculation schedule set;

Use the model optimizer to generate historical calculation data, and use the cost estimation model to calculate the loss output of the regression loss function;

Obtain the model coefficient corresponding to the lowest loss output;

sending the model coefficients to the edge computing model to generate a corresponding target model;

The objective function is sent to the edge-aware analysis system.

In one or more embodiments, preferably, the scheduling space S algorithm specifically includes:

Setting input data, the input data includes the intermediate expression of the calculation graph and the description of the edge intelligent computing terminal;

Setting output data, the output data is a scheduling matching space;

initializing the scheduling matching space;

Perform operator fusion and replacement on the intermediate representation of the computation graph according to the description of the edge intelligent computing terminal to generate a computation graph representation;

Sort the hardware acceleration operators by size according to the intermediate expression amount of the computation graph, and generate a scheduling configuration set;

Obtain the CPU to perform constraint analysis on the hardware acceleration operator, and generate a scheduling set that does not meet the restriction;

The schedule sets that do not meet the constraints are tripled from the schedule configuration set and stored in a schedule matching space.

According to the second aspect of the embodiments of the present invention, a collaborative control method based on terminal-cloud integration is provided.

In one or more embodiments, the described collaborative control method based on terminal-cloud integration includes:

The mobile terminal control system collects information and coordinates the execution of control instructions through mobile phones, drones, cars, traffic lights, and cameras;

The edge-aware analysis system is deployed by collecting information, and performing information fusion on the obtained collected data;

The cloud collaborative control platform performs data management, business communication and coordinated control instruction generation, and performs online computing model training according to the data sent by the edge perception analysis system.

According to a third aspect of the embodiments of the present invention, there is provided a computer-readable storage medium storing computer program instructions thereon, the computer program instructions implementing the method according to the first aspect of the embodiments of the present invention when executed by a processor.

According to a fourth aspect of an embodiment of the present invention, an electronic device is provided, including a memory and a processor, the memory being used to store one or more computer program instructions, wherein the one or more computer program instructions are processed by the processing The controller executes to implement the steps described in the first aspect of the embodiment of the present invention.

The technical solutions provided by the embodiments of the present invention may include the following beneficial effects:

1) In the embodiment of the present invention, by adding an edge perception analysis subsystem, the difficulty of realizing L4-level intelligence is reduced on the premise that the intelligence of the vehicle is not improved too much.

2) In the embodiment of the present invention, the problem of blind spots existing in the perception of a single vehicle is made up through the collaborative control of multiple edge perceptions and the cloud;

3) In the embodiment of the present invention, various types of sensors are used to judge the operating state, and combined with the judgment results, efficient traffic management in a mixed operation scenario of manned and unmanned operation can be realized.

Other features and advantages of the present invention will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention may be realized and attained by the structure particularly pointed out in the written description, claims, and drawings.

The technical solutions of the present invention will be further described in detail below through the accompanying drawings and embodiments.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings used in the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those skilled in the art, other drawings can also be obtained from these drawings without creative effort.

FIG. 1 is a structural diagram of a collaborative control system based on terminal-cloud fusion according to an embodiment of the present invention.

FIG. 2 is a structural diagram of a cloud collaborative control platform in a terminal-cloud integration-based collaborative control system according to an embodiment of the present invention.

Fig. 3 is a schematic diagram of the connection relationship between the cloud side and the terminal side in a collaborative control system based on terminal cloud fusion according to an embodiment of the present invention.

FIG. 4 is a structural diagram of a mobile terminal control system in a collaborative control system based on terminal-cloud integration according to an embodiment of the present invention.

FIG. 5 is a schematic diagram of a perception confidence level table in a collaborative control system based on device-cloud fusion according to an embodiment of the present invention.

FIG. 6 is a structural diagram of an edge-aware analysis system in a collaborative control system based on terminal-cloud fusion according to an embodiment of the present invention.

FIG. 7 is a schematic structural diagram of a collaborative control system based on terminal-cloud fusion according to an embodiment of the present invention.

FIG. 8 is a flowchart of an optimal computing scheduling decision algorithm in a collaborative control system based on terminal-cloud fusion according to an embodiment of the present invention.

FIG. 9 is a flowchart of a scheduling space S algorithm in a collaborative control system based on terminal-cloud fusion according to an embodiment of the present invention.

FIG. 10 is a flowchart of a collaborative control method based on terminal-cloud integration according to an embodiment of the present invention.

FIG. 11 is a structural diagram of an electronic device in an embodiment of the present invention.

Detailed ways

In some of the processes described in the description and claims of the present invention and the above-mentioned drawings, various operations are included in a specific order, but it should be clearly understood that these operations may not be in accordance with the order in which they appear herein. For execution or parallel execution, the sequence numbers of the operations, such as 101, 102, etc., are only used to distinguish different operations, and the sequence numbers themselves do not represent any execution order. Additionally, these flows may include more or fewer operations, and these operations may be performed sequentially or in parallel. It should be noted that the descriptions such as "first" and "second" in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequence, nor do they limit "first" and "second" are different types.

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of the present invention.

With the development of intelligent networked vehicles, it is bound to bring challenges to the existing road management model. The original urban traffic management system needs to deal with the data perception calculation and collaborative scheduling in the mixed operation of intelligent vehicles and human-driven vehicles. problem, thereby ensuring that there is no information exchange gap between vehicles with two different driving modes. Specifically, the two different driving modes are human driving and automatic driving.

In the embodiments of the present invention, a collaborative control system and method based on terminal-cloud integration are provided. This solution ensures that there is no information exchange gap between vehicles with two different driving modes, and improves the Efficiency of driverless management.

As shown in Figure 1, the described collaborative control system based on terminal-cloud integration specifically includes:

The cloud collaborative control platform 101, the edge perception analysis system 102, and the mobile terminal control system 103; wherein, the mobile terminal control system is installed on the technical facilities of the intelligent networked vehicle or road, and is used for collecting information and executing coordinated control instructions; The edge perception analysis system 102 is deployed on both sides of the road or 5G service base stations for information collection and information fusion, and the cloud collaborative control platform 101 is deployed on the cloud platform for data management, business communication and coordinated control instruction generation.

In the embodiment of the present invention, since it is extremely difficult to realize automatic driving through the original intelligent networked vehicle itself, this solution can effectively solve the problem of scene functions that cannot be realized by single-vehicle intelligence. The flow of data and control instructions can effectively clarify the perceptual data obtained at different levels, and then use the corresponding perceptual data for control to realize the iterative update of the algorithm in the terminal, edge, and cloud environment.

As shown in FIG. 2, in one or more embodiments, preferably, the cloud collaborative control platform 101 includes an algorithm model library 201, an algorithm training engine 202, a model distributor 203, and a collaborative scheduling/control engine 204;

The algorithm training engine 202 is configured to obtain the first perception data uploaded by the edge perception analysis system, perform algorithm training, and generate an operation model according to the minimum objective function value;

The algorithm model library 201 is used to obtain the operation model generated by the algorithm training engine;

the model distributor 203, configured to transfer the operation model in the algorithm model library 201 to the edge-aware analysis system 102;

The collaborative scheduling/control engine 204 is configured to perform state evaluation in real time according to the perception data, and give control instructions to the edge perception analysis system 102;

The collaborative scheduling/control engine 204 is configured with an optimal calculation scheduling decision algorithm;

A scheduling space S algorithm is configured in the model distributor 203 .

In the embodiment of the present invention, a specific system structure of the cloud collaborative control platform is provided, and data acquisition, model training and control instruction generation are performed through the structure, thereby realizing real-time online control of the entire system.

In one or more embodiments, preferably, the cloud collaborative control platform 101 further includes a service governance and open interface management module, a capability container management module, and an infrastructure and operating environment platform;

The capability container management module includes a data service sub-module, an intelligent algorithm and application sub-module, a micro-service architecture sub-module, a multi-source heterogeneous device management sub-module, and a terminal-cloud collaboration sub-module to jointly complete cross-business application services and dynamic information management. collaboration;

The infrastructure and operating environment platform is used to perform storage, computing and data processing performance support for the entire cloud collaborative control platform.

In this embodiment of the present invention, functional configurations other than collaborative control and model analysis are provided. The specific functional configurations include two types. The first type is for data processing, and the second type is for information and service collaboration. , complete the data interaction with the mobile terminal through the above two methods.

In one or more embodiments, preferably, the mobile terminal control system 103 specifically includes: a local perception module 401, a local control module 402, a local upload module 403, a collaborative perception module 404, and a collaborative control module 405;

The local perception module 401 collects data through sensors connected to the mobile terminal, and saves it as protocol data;

The local control module 402 is configured to receive the control instruction information sent by the edge perception analysis system 102, and perform coordinated control according to the control instruction information;

The local uploading module 403 is configured to store the data obtained by the local perception module 401 as perception data in a fixed format and send it to the collaborative perception module 404 and the edge perception analysis system 102;

The collaborative sensing module 404 is used to determine the confidence of the sensing data according to different sensing data types;

The confidence level is specifically a perceptual data confidence level table, the structure of which is shown in FIG. 5 , and the perceptual data confidence level table may include confidence levels of different types of data from different data sources.

The collaborative control module 405 is configured to obtain the control instruction issued by the edge perception analysis system.

In the embodiment of the present invention, the mobile terminal control system is mainly a set of software + hardware equipment, running on various types of intelligent mobile terminal equipment, and the intelligent mobile terminal equipment may include intelligent network connection or unmanned driving. car. The module is mainly composed of a local perception module, a local control module, a data upload module, a collaborative perception module and a collaborative control module. The local sensing module collects data from various types of sensor data accessed by the mobile terminal through the terminal data collector or receives the edge sensing data received by the collaborative sensing module, that is, determines the confidence level of the sensing data according to different sensing data types, so as to determine that the data is being analyzed. The calculation weight in the engine is sent to the local analysis engine for analysis. If the data analysis and calculation can be completed locally, the result of the analysis will be sent to the local control module, and the sensing data will be sent to the edge sensing collaboration through the data upload module as needed. system. The local control module receives the analysis results from the local perception module and the control commands from the collaborative control module, and inputs the data into the local decision controller. The decision controller analyzes and calculates the weights of different data sources, generates execution control commands, and sends them to The terminal instructs the executor to execute.

As shown in FIG. 6, in one or more embodiments, preferably, the edge perception analysis system 102 specifically includes: a perception analysis module 601 and a cooperative control module 602;

Wherein, the perception analysis module includes a cloud data transceiver 603, a deep learning engine 604, a roadside data collector 605, and a terminal data receiver 606;

The collaborative control module includes a cloud control receiver 607 , a decision controller 608 , and a terminal control issuer 609 .

In the embodiment of the present invention, the edge sensing collaborative system is composed of a sensing analysis module and a collaborative control module. The perception analysis module includes a terminal data receiver, a road-end data collector, a deep learning engine, and a cloud data reporter. The terminal data receiver receives data from the terminal that needs to be analyzed by the edge system, and the road-end data collector The sensor data of the terminal is collected, and the terminal data or road terminal data is transmitted to the deep learning engine for calculation according to the actual needs. At the same time, the calculated results or the original perception data can also be sent to the cloud platform through the cloud data transceiver for algorithm model training. The new model trained in the cloud can also be sent to the deep learning engine of the edge perception collaboration system through the cloud data transceiver. . The collaborative control module mainly receives the output results of the perception analysis module and the control commands of the cloud control platform, and inputs them into the decision controller. The decision controller generates the actual control commands, and sends the control commands to the mobile terminal for execution through the terminal control issuer. .

FIG. 7 is a schematic structural diagram of a collaborative control system based on terminal-cloud fusion according to an embodiment of the present invention. As shown in FIG. 7 , in the embodiment of the present invention, the multi-layer perception data and algorithm analysis application definitions between the mobile terminal, the edge system and the cloud platform are determined, and the flow process of data and control instructions before each level is clarified, In this way, the coordination of perception data and control instructions on the terminal side, edge side and cloud side can be realized, and by setting the perception data confidence table, the reliability of different data sources relative to the system at this layer can be well resolved when applied in different perception layers. question. At the same time, the patented method clarifies how to realize the continuous iterative update of the algorithm in the environment of end, edge and cloud. The above configuration method is beneficial to continuously optimize the reliability and accuracy of the system according to the scene data in the subsequent practical application process.

As shown in FIG. 8, in one or more embodiments, preferably, the optimal computing scheduling decision algorithm specifically includes:

S801. Obtain a data input scale and a calculation scheduling set;

S802. Use the model optimizer to generate historical calculation data, and use the cost estimation model to calculate the loss output of the regression loss function;

S803, obtaining the model coefficient corresponding to the lowest loss output;

S804, sending the model coefficients to the edge computing model to generate a corresponding target model;

S805. Send the objective function to the edge-aware analysis system.

In the embodiment of the present invention, the model optimizer is the central module. In each iteration, the model optimizer selects a batch of model coefficients with the best performance according to the cost estimation model to run on the edge intelligent computing terminal, and the collected data is used to update Historical data and cost estimation models. The code of the heterogeneous computing backend generated by the cost estimation model is encoded as an embedding vector together with the smart chip model, and then a linear layer is used to predict the final cost value for the embedding vector. The objective function usually chooses the regression loss function.

As shown in FIG. 9, in one or more embodiments, preferably, the scheduling space S algorithm specifically includes:

S901. Set input data, where the input data includes an intermediate expression of a computational graph and a description of an edge intelligent computing terminal;

S902, setting output data, where the output data is a scheduling matching space;

S903, initialize the scheduling matching space;

S904. Perform operator fusion and replacement on the intermediate expression quantity of the calculation graph according to the description of the edge intelligent computing terminal to generate a calculation graph expression;

S905. Sort the hardware acceleration operators by size according to the intermediate expression amount of the calculation graph, and generate a scheduling configuration set;

S906, obtain the CPU to perform constraint analysis on the hardware acceleration operator, and generate a scheduling set that does not meet the restriction;

S907 , triplet the scheduling set that does not meet the restriction from the scheduling configuration set, and store it in the scheduling matching space.

In the embodiment of the present invention, the model distributor adopts the optimal computing scheduling decision to perform optimal computing scheduling on the computing graph after a given intelligent hardware computing device, and the heterogeneous computing backend completes the inference of the respective computing subgraphs in parallel. The global calculation delay is minimized.

As shown in FIG. 10 , in one or more embodiments, the described method for collaborative control based on terminal-cloud fusion includes:

S1001. The mobile terminal control system performs information collection and coordinated control instruction execution through mobile phones, drones, automobiles, traffic lights, and cameras;

S1002, the edge-aware analysis system is deployed by collecting information, and performing information fusion on the obtained collected data;

S1003: The cloud collaborative control platform performs data management, business communication and generation of coordinated control instructions, and performs online computing model training according to data sent by the edge perception analysis system.

In the embodiment of the present invention, data is collected through multiple types of devices in the mobile terminal, and different smart devices are used to execute different coordinated control instructions; the collected information is deployed and distributed in the edge perception analysis system, so as to realize the acquisition of information. Fusion and real-time data interaction, online running model training is carried out on the cloud platform through real-time acquisition data, and online operation is carried out through real-time acquisition of sensor data to generate cloud control instructions, to the lower edge perception collaborative system and It is distributed in the mobile terminal to realize the data coordination in the whole system.

According to a third aspect of the embodiments of the present invention, there is provided a computer-readable storage medium on which computer program instructions are stored, and when executed by a processor, the computer program instructions implement any one of the first aspect of the embodiments of the present invention. method described.

According to a fourth aspect of the embodiments of the present invention, an electronic device is provided. FIG. 11 is a structural diagram of an electronic device in an embodiment of the present invention. The electronic device shown in FIG. 11 is a general terminal-cloud collaborative control device, which includes a general computer hardware structure, which at least includes a processor 1101 and a memory 1102 . The processor 1101 and the memory 1102 are connected by a bus 1103 . The memory 1102 is adapted to store instructions or programs executable by the processor 1101 . The processor 1101 may be an independent microprocessor, or may be a set of one or more microprocessors. Thus, the processor 1101 executes the instructions stored in the memory 1102, thereby executing the above-mentioned method flow of the embodiments of the present invention to process data and control other devices. The bus 1103 connects the above-mentioned various components together, while connecting the above-mentioned components to the display controller 1104 and the display device and the input/output (I/O) device 1105 . The input/output (I/O) device 1105 may be a mouse, a keyboard, a modem, a network interface, a touch input device, a somatosensory input device, a printer, and other devices known in the art. Typically, input/output devices 1105 are connected to the system through input/output (I/O) controllers 1106 .

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media having computer-usable program code embodied therein, including but not limited to disk storage, optical storage, and the like.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each process and/or block in the flowchart illustrations and/or block diagrams, and combinations of processes and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit and scope of the invention. Thus, provided that these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

A collaborative control system based on terminal-cloud integration, characterized in that the system includes a cloud collaborative control platform, an edge perception analysis system and a mobile terminal control system; wherein the mobile terminal control system is installed in an intelligent networked vehicle or a road In terms of technical facilities, it is used to collect information and execute coordinated control instructions; the edge perception analysis system is deployed on both sides of the road or 5G service base stations for information collection and information fusion, and the cloud collaborative control platform is deployed on the cloud platform, For data management, business communication and coordination control command generation.
The collaborative control system based on terminal-cloud integration according to claim 1, wherein the cloud collaborative control platform comprises an algorithm model library, an algorithm training engine, a model distributor, and a collaborative scheduling/control engine;

The algorithm training engine is used to obtain the first perception data uploaded by the edge perception analysis system, perform algorithm training, and generate an operation model according to the minimum objective function value;

The algorithm model library is used to obtain the operation model generated by the algorithm training engine;

the model distributor, configured to transfer the operation model in the algorithm model library to the edge-aware analysis system;

The collaborative scheduling/control engine is used to perform state evaluation in real time according to the perception data, and give control instructions to the edge perception analysis system;

The collaborative scheduling/control engine is configured with an optimal computing scheduling decision algorithm;

A scheduling space S algorithm is configured in the model distributor.
The collaborative control system based on terminal-cloud integration according to claim 1, wherein the cloud collaborative control platform further comprises a service governance and open interface management module, a capability container management module, and an infrastructure and operating environment platform;

The service governance and open interface management module includes a service interface sub-module, an operation management sub-module, a distribution scheduling sub-module and a security management sub-module, and is used for collaborative management of multi-source heterogeneous data with the mobile terminal;

The capability container management module includes a data service sub-module, an intelligent algorithm and application sub-module, a micro-service architecture sub-module, a multi-source heterogeneous device management sub-module, and a terminal-cloud collaboration sub-module, which jointly complete the cross-business application services and dynamic information management. collaboration;

The infrastructure and operating environment platform is used to perform storage, computing and data processing performance support for the entire cloud collaborative control platform.
The collaborative control system based on terminal-cloud integration according to claim 1, wherein the mobile terminal control system specifically includes: a local perception module, a local control module, a local upload module, a collaborative perception module, and a collaborative control module. module;

The local perception module collects data through sensors connected to the mobile terminal, and saves it as protocol data;

The local control module is configured to receive the control instruction information sent by the edge perception analysis system, and perform coordinated control according to the control instruction information;

The local uploading module is configured to store the data obtained by the local perception module as perception data in a fixed format and send it to the collaborative perception module and the edge perception analysis system;

The collaborative sensing module is used to determine the confidence level of the sensing data according to different sensing data types;

The cooperative control module is configured to obtain the control instruction issued by the edge perception analysis system.
The collaborative control system based on terminal-cloud fusion according to claim 1, wherein the edge perception analysis system specifically comprises: a perception analysis module and a collaborative control module;

Wherein, the perception analysis module includes a cloud data transceiver, a deep learning engine, a road-end data collector, and a terminal data receiver;

Wherein, the collaborative control module includes a cloud control receiver, a decision controller, and a terminal control transmitter.
The collaborative control system based on terminal-cloud fusion according to claim 2, wherein the optimal calculation scheduling decision algorithm specifically includes:

Get data input scale and calculation schedule set;

Use the model optimizer to generate historical calculation data, and use the cost estimation model to calculate the loss output of the regression loss function;

Obtain the model coefficient corresponding to the lowest loss output;

sending the model coefficients to the edge computing model to generate a corresponding target model;

The objective function is sent to the edge-aware analysis system.
The collaborative control system based on terminal-cloud fusion according to claim 2, wherein the scheduling space S algorithm specifically includes:

Setting input data, the input data includes the intermediate expression of the calculation graph and the description of the edge intelligent computing terminal;

Setting output data, the output data is a scheduling matching space;

initializing the scheduling matching space;

Perform operator fusion and replacement on the intermediate representation of the computation graph according to the description of the edge intelligent computing terminal to generate a computation graph representation;

Sort the hardware acceleration operators by size according to the intermediate expression amount of the computation graph, and generate a scheduling configuration set;

Obtain the CPU to perform constraint analysis on the hardware acceleration operator, and generate a scheduling set that does not meet the restriction;

The schedule sets that do not meet the constraints are tripled from the schedule configuration set and stored in a schedule matching space.
A collaborative control method based on terminal-cloud fusion, characterized in that the method comprises:

The mobile terminal control system collects information and coordinates the execution of control instructions through mobile phones, drones, cars, traffic lights, and cameras;

The edge-aware analysis system is deployed by collecting information, and performing information fusion on the obtained collected data;

The cloud collaborative control platform performs data management, business communication and coordinated control instruction generation, and performs online computing model training according to the data sent by the edge perception analysis system.
A computer-readable storage medium on which computer program instructions are stored, wherein the computer program instructions implement the method according to any one of claims 8 when executed by a processor.
An electronic device, comprising a memory and a processor, wherein the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the claim The steps of any one of claim 8.