WO2023207624A1

WO2023207624A1 - Data processing method, device, medium, and roadside collaborative device and system

Info

Publication number: WO2023207624A1
Application number: PCT/CN2023/088250
Authority: WO
Inventors: 刘彦斌; 高玉涛
Original assignee: 阿里云计算有限公司
Priority date: 2022-04-26
Filing date: 2023-04-14
Publication date: 2023-11-02
Also published as: CN115063969A

Abstract

Provided in the embodiments of the present application are a data processing method, a device, a medium, and a roadside collaborative device and system. The method comprises: controlling a plurality of roadside apparatuses to execute roadside data acquisition operations; performing data preprocessing on roadside data provided by the plurality of roadside apparatuses, so as to obtain collaborative data; and, on the basis of the collaborative data, performing edge computing processing, so as to execute a collaborative task on the basis of an edge computing processing result. In the present application, the plurality of roadside apparatuses and an edge computing module are integrated into a whole, and a large amount of roadside data acquired by the plurality of roadside apparatuses can also be directly transmitted to the edge computing module, so that the edge computing module performs edge computing processing on the large amount of roadside data. Therefore, procedures of encoding, decoding, network transmission, etc. are not needed any more, thus saving part of time for data processing and data transmission, effectively improving data processing efficiency, and accordingly effectively increasing the response speed of the roadside collaborative device.

Description

Data processing methods, equipment, media and roadside collaborative equipment and systems

This application requires the priority of the Chinese patent application submitted to the China Patent Office on April 26, 2022, with the application number 202210451931.8 and the application name "Data processing method, equipment, medium and roadside collaborative equipment and system", and its entire content incorporated herein by reference.

Technical field

This application relates to the field of computer technology, and in particular to data processing methods, equipment, media, and roadside collaborative equipment and systems.

Background technique

With the development of vehicle-road collaborative technology, there are also more requirements for the richness of data collection and the timely effect of data processing by roadside collaborative equipment.

In the existing technology, in order to enable roadside collaborative equipment to obtain rich and diverse data information, various roadside devices are usually configured, such as cameras for video image collection, high-resolution vehicle sensing LiDAR with rate information and so on. In practical applications, roadside devices usually use devices with certain complete functions. These roadside devices collect a large amount of roadside data to be processed. Generally, roadside devices do not have the ability to process large amounts of data, so these roadside devices are usually The side data is sent to the cloud server for data processing. However, the roadside device needs to encode and transmit the roadside data over the network before it can be received by the cloud server. The cloud server also needs to decode the roadside data before it can further process the roadside data. Each of the above links takes a certain amount of time, data processing is slow, and the real-time effect is poor. Therefore, a solution that can improve data processing efficiency is needed.

Contents of the invention

In order to solve or improve the problems existing in the existing technology, each embodiment of the present application provides data processing methods, equipment, media, and roadside collaborative equipment and systems.

In a first aspect, in an embodiment of the present application, a data processing method is provided. The method includes:

Control multiple roadside devices to perform roadside data collection operations;

Perform data preprocessing on the roadside data provided by the multiple roadside devices to obtain collaborative data;

Edge computing processing is performed based on the collaborative data to perform collaborative tasks based on edge computing processing results.

In a second aspect, in an embodiment of the present application, a roadside collaboration device is provided. The roadside collaboration device includes integrated: an edge computing module, and the edge computing module Modular connection of multiple roadside devices;

The edge computing module is used to control multiple roadside devices to perform collection operations, preprocess data provided to the roadside through multiple roadside devices to obtain collaborative data, and perform edge computing processing on the collaborative data to facilitate edge computing based on edge computing. Process the results and perform collaborative tasks;

The plurality of roadside devices are used to collect roadside data according to the collection instructions issued by the edge computing module, and send the roadside data to the edge computing module.

In a third aspect, in one embodiment of the present application, a roadside collaboration system is provided, and the system includes:

At least one roadside collaboration device and cloud server; wherein,

The roadside collaboration equipment includes integrated: an edge computing module, and multiple roadside devices connected to the edge computing module; used to control multiple roadside devices to perform roadside data collection operations; to the Perform data preprocessing on roadside data provided by multiple roadside devices to obtain collaborative data; perform edge computing processing based on the collaborative data to perform collaborative tasks based on the edge computing processing results;

The cloud server is configured to receive edge computing processing results provided by the roadside collaboration device, so as to collaborate on tasks based on the edge computing processing results.

In a fourth aspect, in an embodiment of the present application, an electronic device is provided, including a memory and a processor; wherein,

The memory is used to store programs;

The processor is coupled to the memory and is used to execute the program stored in the memory to implement a data processing method described in the first aspect.

In the fifth aspect, in an embodiment of the present application, a non-transitory machine-readable storage medium is provided, the non-transitory machine-readable storage medium stores executable code, and when the executable code is When the processor of the electronic device executes, the processor is caused to execute a data processing method as described in the first aspect.

In the technical solutions provided by the embodiments of this application, in order to obtain various roadside related information more comprehensively, a variety of roadside devices (such as cameras, radars, etc.) are usually used to collect relevant roadside data. In this solution, an edge computing module with large computing power and multiple roadside devices are integrated into an integrated structure, and the edge computing module can control the working status of multiple roadside devices. A large amount of roadside data collected by multiple roadside devices can also be directly transferred to the edge computing module, and then the edge computing module performs edge computing processing of a large amount of roadside data, without the need to go through encoding, decoding, network transmission, etc. It saves some data processing and data transmission time, can effectively improve data processing efficiency, and thus can effectively improve the response speed of roadside collaborative equipment.

Description of drawings

In order to more clearly explain the embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, The drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without exerting creative efforts.

Figure 1 is a schematic structural diagram illustrating a conventional roadside system according to an embodiment of the present application;

Figure 2 is a schematic structural diagram of a roadside collaboration device provided by an embodiment of the present application;

Figure 3 is a schematic diagram of video data processing illustrating an embodiment of the present application;

Figure 4 is a schematic flow chart of the data processing method provided by the embodiment of the present application;

Figure 5 is a schematic structural diagram of a latch timing provided by an embodiment of the present application;

Figure 6 is a schematic diagram of the triggering and image alignment process illustrating an embodiment of the present application;

Figure 7 is a schematic structural diagram of a data processing device provided by an embodiment of the present application;

FIG. 8 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

Detailed ways

In order to enable those in the technical field to better understand the solution of the present application, the technical solution in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application.

Some of the processes described in the specification, claims, and above-mentioned drawings of this application include multiple operations that appear in a specific order. These operations may not be performed in the order in which they appear in this document or may be performed in parallel. The sequence numbers of operations, such as 101, 102, etc., are only used to distinguish different operations. The sequence numbers themselves do not represent any execution order. Additionally, these processes may include more or fewer operations, and the operations may be performed sequentially or in parallel. It should be noted that the descriptions such as "first" and "second" in this article are used to distinguish different messages, devices, modules, etc., and do not represent the order, nor do they limit "first" and "second" are different types. In addition, the embodiments described below are only some of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the scope of protection of this application.

With the development of vehicle-road collaboration technology, the types of data information collected are becoming more and more diverse. For subsequent processing of computing chips, the computing power required is also increasing. Figure 1 is a schematic structural diagram illustrating a conventional roadside system according to an embodiment of the present application. As shown in Figure 1, in conventional roadside equipment, at least one of a camera, millimeter wave radar, lidar, infrared sensor, etc. is selected as a roadside device as needed. Using these roadside devices, a large amount of roadside data can be obtained. Take the roadside device as a camera as an example for illustration. Use the camera to continuously capture videos and collect images at a preset frequency. If multi-view image collection is required, cameras will be placed on different roadside poles or at different positions of the roadside poles to collect image information at the same time, and then the collected image information will be encoded, transmitted, and decoded. etc. The data processing process is cumbersome and time-consuming, making it difficult to meet the demand for rapid response to a large amount of roadside data in roadside collaborative application scenarios. Therefore, there is a need for a solution that can meet the needs of fast and efficient processing of roadside data in roadside collaboration scenarios.

In the technical solution of this application, the specific working process will be explained in the following examples.

Figure 2 is a schematic structural diagram of a roadside collaboration device provided by an embodiment of the present application. From Figure 2 It can be seen that the roadside collaboration equipment includes integrated: an edge computing module 1 and a plurality of roadside devices 2 connected to the edge computing module 1 . The edge computing module 1 is used to control multiple roadside devices 2 to perform collection operations, preprocess data provided to the roadside through the multiple roadside devices 2 to obtain collaborative data, and perform edge computing processing on the collaborative data, so as to Execute collaborative tasks based on edge computing processing results. A plurality of roadside devices 2 are configured to collect roadside data according to collection instructions issued by the edge computing module 1 and send the roadside data to the edge computing module 1 .

The roadside collaborative equipment includes a housing 3. The edge computing module 1 is integrated in the housing 3. The multiple roadside devices 2 are integrated on the housing 3 and located in the housing 1. different positions, so that the plurality of roadside devices 2 have different perception angles and/or different perception capabilities.

The edge computing module mentioned here can be understood as an edge computing chip and memory with large computing power, which can bear the computing and processing capabilities of a large amount of roadside data. In addition, the edge computing module can also directly control roadside equipment (such as traffic lights) based on the edge computing results, or generate a control strategy based on the edge computing results. It can also send the edge computing results or control strategies to nearby vehicles, and also It can be sent to the cloud server so that the cloud server can perform corresponding collaborative tasks based on the received edge computing results or corresponding control policies.

As can be seen from Figure 2, the edge computing module 1 is connected to multiple roadside devices 2 through data transmission ports (such as mipi interfaces). It should be noted that the roadside device 2 may be a radar, a camera, an infrared sensor, etc. For ease of understanding, the roadside device is a camera as an example for description.

For example, FIG. 3 is a schematic diagram of video data processing illustrating an embodiment of the present application. Multiple cameras are arranged at different positions of the housing 3 and face in different directions. Multiple cameras with different sensing capabilities can also be arranged in the same direction at the same time. For example, a wide-angle camera and a large depth-of-field camera can be arranged, or an RGB camera and a depth camera can be arranged to collect data in the same direction. Different image data are obtained to meet the subsequent edge computing processing needs based on the collected image data.

As can be seen from Figure 2, multiple cameras are directly connected to the edge computing module and integrated into the roadside collaboration device shown in Figure 3, becoming an edge computing module with large computing power integrated with multiple roadside devices. structure. When there is a need for image collection, the edge computing module issues corresponding collection instructions (for example, exposure instructions) to control multiple cameras for image collection. Since the work of multiple cameras is controlled by the same edge computing module, multiple cameras have a unified collection frequency. This means that multiple cameras collect images at the same time. In the subsequent image alignment and fusion processing, it is no longer necessary to perform tedious operations such as filtering noise images from different cameras and finding matching timestamps before performing alignment. Simplifying the image data processing process can effectively improve data processing efficiency.

In addition, since the camera is directly controlled by the edge computing module, and the image data collected by the camera is also directly processed by the edge computing module, there is no need to encode or decode the image data, nor does it need to encode the encoded image. Data is transmitted from the camera to the edge computing module. Saves encoding, decoding and transmission time, especially in roadside collaborative application scenarios When rapid response is required and large amounts of image data are processed frequently, it can significantly improve data processing speed and efficiency.

In practical applications, the same roadside collaborative equipment integrates multiple roadside devices at the same time. These roadside devices are used to collect images from different perspectives in different directions, so that current road condition information can be obtained more comprehensively. These multiple cameras may have exactly the same sensing capabilities, or they may have different sensing capabilities. The perceptual capabilities mentioned here can be understood as the parameters of the camera, such as wide-angle size, depth of field size, focal length range, etc. Different cameras have different image collection capabilities and are suitable for different collection scenarios.

Therefore, when setting up multiple cameras, multiple cameras with the same sensing capabilities can be arranged at different viewing angles, and multiple cameras with different sensing capabilities can be arranged at the same viewing angle. This enables the acquisition of large-scale, multi-type image data. These image data will directly send edge computing information without the need for cumbersome operations such as encoding and decoding, which facilitates accurate image recognition by subsequent edge computing modules. Although there are many cameras in number and type, because these cameras are all uniformly controlled by the edge computing module, there is no need to perform cumbersome operations such as image data alignment, which can effectively improve the processing efficiency of large amounts of image data. It should be noted that the layout and positional relationship of multiple cameras in the roadside collaboration equipment mentioned here is only an example and does not constitute a limitation on the technical solution of this application. Users can make corresponding adjustments according to their actual application needs.

In one or more embodiments of the present application, the device further includes: a radar device. The edge computing module is used to generate collection instructions according to the preset collection frequency or the acquired trigger signal, so as to perform the roadside data collection operation through the radar device; based on the radar data collected through the radar device and the The image data is aligned and fused and edge computing is performed.

In practical applications, other roadside devices can be integrated into the roadside collaborative equipment as needed, such as radar roadside devices (for example, laser radar, ultrasonic radar, millimeter wave radar, etc.), infrared roadside devices, etc. These other integrated roadside devices are also controlled by the edge computing module. When the edge computing module sends a trigger command, these roadside devices will perform sensing operations and collect corresponding roadside data (such as radar data). The collected roadside data will be directly provided to the edge computing module together with the image data, and then the edge computing device will perform corresponding alignment and fusion processing and perform edge computing.

It should be noted that there are many different types of roadside devices (such as cameras, radars, etc.) integrated into the roadside collaborative equipment. These different types of roadside devices perform collection operations uniformly under the control of the edge computing module, thereby ensuring that all All kinds of roadside data collected are roadside data with the same timestamp. When performing a variety of roadside data fusion processing, since the roadside data here are all based on the data collected by the same collection instruction, these roadside data have good time consistency, and there is no need to analyze these collected data. Cumbersome operations such as denoising roadside data and finding corresponding timestamps for roadside data alignment. While meeting the needs of diversified roadside data collection, it can effectively improve the data processing efficiency of large and diverse roadside data. In addition, since roadside data does not require network transmission, local edge computing processing can reduce network bandwidth usage. It can also improve the safety effect of roadside data processing.

Figure 4 is a schematic flowchart of the data processing method provided by the embodiment of the present application. The execution subject of the data processing method may be a roadside collaborative device. As shown in Figure 4, the data processing method includes the following steps:

401: Control multiple roadside devices to perform roadside data collection operations.

402: Perform data preprocessing on the roadside data provided by the multiple roadside devices to obtain collaborative data.

403: Execute edge computing processing based on the collaborative data, so as to execute collaborative tasks based on the edge computing processing results.

As can be seen from the foregoing embodiments, after multiple roadside devices and edge computing modules are integrated into one device, the edge computing module can directly control the collection operations of multiple roadside devices. For example, taking the roadside device as a camera, the edge computing module can send collection instructions to multiple roadside cameras, and then the multiple roadside cameras perform roadside data collection based on the collection instructions. Different from the existing technology, each camera independently performs image collection operations according to its own set shooting parameters (such as image collection frequency). In the embodiment of the present application, each camera is controlled by the edge computing module, so that all cameras Execute the collection operation at the same time to obtain roadside data with the same timestamp. Using the above solution, there is no need to consider whether each roadside device has the same standard time, or whether it has the same image acquisition frequency, etc., and there is no need to perform cumbersome encoding, decoding, and denoising of image data (the denoising mentioned here). It can be understood that in the traditional roadside collaborative system, each camera works independently and has its own collection frequency and cycle. This means that the time consistency of the images obtained by each camera is not good, and some images may not be found with the same time. This part of the image is a noisy image and needs to be denoised) and other complex image processing work. Instead, the roadside data collected by different cameras can be directly fused, which can effectively improve the data processing efficiency. In addition, since roadside data no longer needs to be sent to the cloud server through the network, but is processed by a local edge computing module, the security protection effect of roadside data processing can be effectively improved.

The roadside data collected by the camera mentioned here can be understood as the original audio and video data collected by the camera. These data are also called audio and video naked data, which are collected directly from the data source (camera, microphone, etc.) and are not processed. .

After obtaining the roadside data, basic processing needs to be done, such as image signal processing (Image Signal Process, ISP). Specifically, the image signal processing method is as follows:

Using the image sensor core (image sensor core) to perform video image acquisition is to use the sensor (the photosensitive component sensor behind the camera) to convert the captured things (represented by analog signals) into a processor (such as an edge computing module) that can recognize , processed digital signal, the digital signal rawdata at this time is a video stream, and the format is rawRGB. Through the above steps, the analog-to-digital conversion of the roadside data is completed.

After obtaining the roadside data in the form of digital signals, the digital signals are further optimized and processed, including: linear correction, noise removal, black level correction, bad pixel removal, color interpolation, Gamma correction, RGB2YUV conversion, and active white balance. processing, active exposure control and more.

After completing the ISP processing, the required image data can be obtained. Of course, in some application scenarios, format conversion may be required. For example, image data in YUV format may be converted into BGR format, and edge computing processing may be performed on the image data after the format conversion is completed.

In one or more embodiments of the present application, controlling multiple roadside devices to perform roadside data collection operations includes: generating a method for controlling the multiple roadside devices based on a preset collection frequency or an acquired trigger signal. A collection instruction for a side device collection operation; controlling the plurality of roadside devices to perform a roadside data collection operation based on the control parameters included in the collection instruction.

In practical applications, the collection frequency can be preset for the edge computing module, and collection instructions are periodically sent to multiple roadside devices according to the predetermined frequency. Furthermore, multiple roadside devices simultaneously perform roadside data collection operations. The time stamp consistency of the roadside data obtained in this way is better. For example, image data is collected through a camera. There is no redundant frame of image data. There is no need to perform tedious denoising, filtering, etc., and the image data can be completed quickly. Alignment work provides a high-quality data foundation for subsequent edge computing.

In addition, the edge computing module can also respond to the trigger signal provided by other devices (such as vehicle induction coil equipment buried underground on the highway), and then the edge computing device responds to the trigger signal and sends it to multiple roadside devices to control multiple roadside devices. The roadside device collects roadside data.

It should be noted that the multiple roadside devices are controlled by the edge computing module. Therefore, relevant parameters for the roadside device to perform sensing operations (such as exposure parameters, etc.) are provided by the edge computing module. For example, when the edge computing module senses that the light at the current moment is good through weather data and/or light sensors, it will make adaptive adjustments to the exposure parameters based on the camera performance, such as reducing the exposure time. For another example, if the image data obtained at the last moment is not good and the content in the image cannot be accurately identified, the edge computing module can adjust the exposure parameters according to the needs, and control multiple roadside devices based on the adjusted exposure parameters. Resume image acquisition. In the embodiment of this application, it is preferable that when performing supplementary collection of roadside data, the edge computing module will control multiple roadside devices to perform supplementary collection of roadside data at the same time, which can effectively simplify the later alignment of roadside data. The calculation amount and calculation time are reduced, thereby improving the efficiency of roadside data processing.

In one or more embodiments of the present application, performing data preprocessing on roadside data provided by the multiple roadside devices to obtain collaborative data includes: determining the collection timestamp of the collection instruction; based on the same The collection timestamp is used to fuse the roadside data to obtain the collaborative data. If the roadside device is a camera, the method of processing the roadside data provided by the camera to obtain collaborative data includes: determining the exposure timestamp of the exposure instruction; and performing fusion processing on the original roadside data based on the same exposure timestamp. , obtain the image data.

As mentioned above, multiple roadside devices are controlled by the same edge computing module. This means that the collection instructions received by multiple roadside devices have the same collection time stamp, and the roadside data collected based on the same collection time stamp has good time consistency, and there is no need to perform tedious processing. Filter alignment operations. The collection timestamp can be used to achieve roadside data alignment, and the roadside data corresponding to the same collection timestamp can be directly fused.

It should be noted that when each roadside device collects roadside data, it is inevitable that some roadside data will be of poor quality. Therefore, before fusion processing, each roadside data needs to be collected separately. Optimize the roadside data provided by the side device. For example, the roadside collaboration device includes: a first camera facing the left, a second camera facing the right, and a third camera facing the front, where the third camera is used to complement the first camera and the second camera. The camera's blind spot. After using the first camera, the second camera and the third camera to collect images, the first original roadside data, the second original roadside data and the third original roadside data are obtained respectively. Furthermore, the first original roadside data, the second original roadside data and the third original roadside data are optimized respectively. If the quality of the image data at the first camera time ta is found to be poor, the second original roadside data will be abandoned altogether. The image data corresponding to the timestamp of the camera and the third camera are then fused with the optimized roadside data to obtain collaborative data.

Of course, you can also choose to perform image data alignment processing based on the respective timestamps generated when multiple roadside devices collect image data. In order to ensure that the timestamps of multiple roadside devices have good consistency, two timing methods can be used, one is to use GPS timing, and the other is to use the edge computing module to latch the timing. This method does not need to rely on GPS when timing the roadside device. Even when the GPS signal is poor, it can ensure that the edge computing module and the roadside device maintain the same time. Specifically: determine the execution timestamp of the roadside data; perform fusion processing on the roadside data provided by the multiple roadside devices based on the execution timestamp corresponding to the roadside data to obtain the collaborative data .

The latch timing will be described in detail below with reference to specific embodiments. Figure 5 is a schematic structural diagram of a latch timing provided by an embodiment of the present application. As can be seen from Figure 5, the edge computing module and the Complex Programmable Logic Device (CPLD) chip are connected through the GPIO (or S232, RS485) interface in addition to the communication serial port and chip transparent serial port. Clock synchronization can be performed using the GPIO interface. Specifically:

The time synchronization between the CPU and the device is performed by the CPU sensing and recording the latch time of the CPLD trigger signal GPIO, which is not affected by the GPS signal. When the system requires it, the CPU can also be timed through a custom clock source. Complete time synchronization between CPU and multiple roadside devices. This is so that after the CPU (that is, the edge computing module mentioned in the above embodiment) sends collection instructions to multiple roadside devices, the obtained roadside data corresponding to the multiple roadside devices have the same standard time. It should be noted that since the roadside data collected by each roadside device has the same collection time stamp and the execution time stamp when the collection action is performed, there is no need to update the images collected by each roadside device. Instead of filtering data, roadside data can be aligned and fused directly based on execution timestamps. Compared with traditional cumbersome processing methods such as roadside data screening, unified control of triggers through edge computing modules can effectively improve the processing efficiency of roadside data.

In order to facilitate understanding, specific examples will be given below with reference to the accompanying drawings, taking the roadside device as a camera. Figure 6 is a schematic diagram illustrating the triggering and image alignment process according to an embodiment of the present application. As can be seen from Figure 6, the edge computing module collects images at t1, t2, t3... according to the preset collection frequency. As can be seen from Figure 6, device 1 and device 2 trigger and collect roadside data (for example, image data) synchronously. In the traditional solution, device 1 and device 2 each collect images according to their own acquisition frequency, and then eliminate redundant image data (for example, find Alignment of image data can be performed only after other image data with the same timestamp is obtained). Therefore, by adopting the technical solution of the present application, roadside data with better consistency can be obtained, and the alignment processing of image data can also be simpler, more accurate and more efficient.

In one or more embodiments of the present application, performing edge computing processing based on the collaborative data to perform collaborative tasks based on edge computing processing results includes: performing data analysis on the collaborative data, and determining the data analysis result as The edge computing processing result; controlling the roadside device to perform collaborative tasks based on the edge computing processing result.

The collaborative data mentioned here is obtained after data preprocessing by the edge computing module. Then, the collaborative data is further analyzed by the edge computing module. This analysis process usually requires a large amount of calculations. For example, a trained machine learning model will be used for analysis to obtain accurate data analysis results, which are the edge computing processing results. . The edge computing module will control the roadside devices accordingly based on the edge computing processing results, for example, controlling the status and time of traffic lights.

For example, a special vehicle needs to pass through an intersection with traffic lights. When the roadside device captures the special vehicle information (for example, the license plate number or vehicle image is recognized through a camera), the characteristic vehicle is identified through the edge computing module, and the vehicle is identified based on the radar The data tests vehicle speed to estimate the time a vehicle will arrive at a traffic light based on vehicle speed. Through the above complex edge computing processing, when the special vehicle arrives at the location of the signal light, the corresponding traffic light is controlled to be green (for example, the green light display time can be extended, or the red light can be controlled to switch to green), achieving no stopping or slowing down. pass. Through the above method, the processing speed of roadside data is faster and the response is more timely.

In one or more embodiments of the present application, performing edge computing processing based on the collaborative data to perform collaborative tasks based on edge computing processing results includes: performing data analysis on the collaborative data, and determining the data analysis result as The edge computing processing result; sending the edge computing processing result to the vehicle device that performs the collaborative task.

In vehicle-road collaboration application scenarios, roadside collaboration equipment is close to data collection terminals (such as cameras, radars, etc.) and also close to audience terminals (such as traffic lights, vehicles). In this solution, data collection, data analysis, and edge computing are all It is completed in the local device, which can ensure that the audience terminal can obtain the edge computing processing results in time, so as to accurately and quickly perform collaborative tasks.

For example, vehicles generally need to slow down when traveling to an intersection to prevent collisions with pedestrians or non-motorized vehicles. However, intersection accidents still occur frequently due to drivers' visual blind spots or pedestrians' failure to obey traffic regulations. The roadside collaborative equipment proposed in this application can be used to collect real-time data. The edge computing module performs local calculations based on the real-time location of pedestrians and the status of signal lights provided by the roadside device in real time, and sends information about pedestrians at red lights or approaching lanes to relevant vehicles. , to assist the driver in making deceleration decisions. In this solution, data collection, processing, analysis, and calculation are all completed in the local roadside collaborative equipment. There is no need to perform time-consuming operations such as tedious data encoding and decoding, data network transmission, etc., which can effectively improve the roadside collaborative equipment Response speed meets the need for rapid response in vehicle-road collaboration application scenarios.

In one or more embodiments of the present application, the method further includes: sending the edge computing processing results to the cloud server.

For example, if the latest edge calculation processing result is a certain intersection (for example, a curve or A traffic accident occurred in a tunnel). If we fail to respond in time and notify the approaching vehicles from afar, more accidents may occur. Therefore, the roadside collaboration device can send the edge computing processing results (specific information such as the time and place of the accident, the affected lanes, etc.) to the cloud, so that the cloud can promptly notify vehicles that are about to go to the accident site to change lanes, routes, or slow down. stroll.

In one or more embodiments of the present application, the roadside device further includes: a camera and/or a radar. Performing data preprocessing on the roadside data provided by the plurality of roadside devices to obtain collaborative data includes: obtaining camera roadside data provided by a plurality of the cameras; obtaining radar roadside data provided by a plurality of the radars. Data; performing alignment and fusion processing on the camera roadside data and the radar roadside data to obtain the collaborative data.

Taking the roadside device as a camera as an example, the edge computing processing based on the image data includes: performing data analysis on the image data to determine the key frames and/or image content contained in the image data; The key frames and/or the image content are sent to the cloud server.

In practical applications, edge computing modules have strong data computing and processing capabilities, in order to improve data processing efficiency and shorten data processing time. Especially in vehicle-road collaboration application scenarios, the above solution is used to enable roadside equipment to provide users with the required data processing results in a timely and rapid manner. Specifically, the edge computing module can be used to perform data analysis on the image data after the alignment operation is completed. For example, the edge computing module determines key frames from the image data or identifies a certain image content after a large amount of calculations. Furthermore, the edge computing processing results (the found key frames and/or the recognized image content) are sent to the cloud server. In this way, the cloud server only needs to make global decisions based on the received results. This can improve the efficiency of roadside data collection, transmission, and processing from roadside devices. Moreover, compared with traditional solutions that transmit a large amount of roadside data to cloud servers, the amount of transmission of edge computing processing results is less, which can effectively reduce the occupation of network bandwidth and thereby improve data transmission efficiency.

In addition to the cameras mentioned above, the roadside device can also be a radar device, such as a laser radar device, an ultrasonic radar device, etc. It should be noted that these roadside devices are directly connected to the edge computing module. They do not need to perform complex processing processes such as encoding, decoding, and network transmission of the collected roadside data, but can be processed directly by the edge computing module. . In addition, a variety of roadside devices are controlled by the same edge computing module and collect roadside data. The roadside data of various roadside devices have good consistency, which facilitates the alignment of roadside data and can effectively improve the data processing speed.

Based on the same idea, embodiments of the present application also provide a data processing device. Figure 7 is a schematic structural diagram of a data processing device provided by an embodiment of the present application. The data processing device includes:

The control module 71 is used to control multiple roadside devices to perform roadside data collection operations.

The processing module 72 is used to perform data preprocessing on the roadside data provided by the multiple roadside devices to obtain collaborative data.

The execution module 73 is configured to execute edge computing processing based on the collaborative data, so as to execute collaborative tasks based on edge computing processing results.

Optionally, the control module 71 is also configured to generate collection instructions for controlling the collection operations of the multiple roadside devices according to the preset collection frequency or the acquired trigger signal;

Control the plurality of roadside devices to perform roadside data collection operations based on control parameters included in the collection instructions.

Optionally, the processing module 72 is also used to determine the collection timestamp of the collection instruction;

The roadside data is fused based on the same collection timestamp to obtain the collaborative data.

Optionally, the processing module 72 is also used to determine the execution timestamp of the roadside data;

The roadside data provided by the multiple roadside devices are fused based on the execution timestamp corresponding to the roadside data to obtain the collaborative data.

Optionally, the execution module 73 is also configured to perform data analysis on the collaborative data and determine the data analysis result as the edge computing processing result;

Control the roadside device to perform collaborative tasks based on the edge computing processing results.

The edge computing processing results are sent to vehicle devices that perform collaborative tasks.

Optionally, a sending module 74 is also included for sending the edge computing processing results to the cloud server.

Optionally, the roadside device includes: a camera. The processing module 72 is also used to obtain camera roadside data provided by multiple cameras;

The camera roadside data provided by multiple cameras are aligned and fused to obtain the collaborative data.

Optionally, the roadside device further includes: radar. The processing module 72 is also used to obtain radar roadside data provided by multiple radars;

The camera roadside data and the radar roadside data are aligned and fused to obtain the collaborative data.

An embodiment of the present application also provides an electronic device. The electronic device is the main node electronic device in the computing unit. Figure 8 is a schematic structural diagram of an electronic device provided by an embodiment of the present application. The electronic device includes a memory 801, a processor 802 and a communication component 803; wherein,

The memory 801 is used to store programs;

The processor 802 is coupled to the memory and configured to execute the program stored in the memory for:

The above-mentioned memory 801 may be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device. Memory may consist of any type of volatile or non-volatile storage device or their Combination implementations such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

Further, the processor 802 in this embodiment may be specifically: a programmable switching processing chip. The programmable switching processing chip is configured with a data copy engine and can copy the received data.

When the above-mentioned processor 802 executes the program in the memory, in addition to the above functions, it can also implement other functions. For details, please refer to the descriptions of the previous embodiments. Further, as shown in FIG. 8 , the electronic device also includes: a power supply component 804 and other components.

Embodiments of the present application also provide a non-transitory machine-readable storage medium. The non-transitory machine-readable storage medium stores executable code. When the executable code is executed by a processor of an electronic device, the The processor executes the method described in the corresponding embodiment of FIG. 4 .

An embodiment of the present application also provides a computer program product, which includes a computer program/instruction. When the computer program/instruction is executed by a processor, the processor can implement the method described in the corresponding embodiment of FIG. 4 .

Based on the above embodiments, in order to obtain various roadside related information more comprehensively, a variety of roadside devices (such as cameras, radars, etc.) are usually used to collect relevant roadside data. In this solution, an edge computing module with large computing power and multiple roadside devices are integrated into an integrated structure, and the edge computing module can control the working status of multiple roadside devices. A large amount of roadside data collected by multiple roadside devices can also be directly transferred to the edge computing module, and then the edge computing module performs edge computing processing of a large amount of roadside data, without the need to go through encoding, decoding, network transmission, etc. It saves some data processing and data transmission time, can effectively improve data processing efficiency, and thus can effectively improve the response speed of roadside collaborative equipment.

The device embodiments described above are only illustrative. The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they may be located in One location, or it can be distributed across multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Persons of ordinary skill in the art can understand and implement the method without any creative effort.

Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the part of the above technical solution that essentially contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM/RAM, magnetic disc, optical disk, etc., including a number of instructions to cause a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application, rather than to Its limitations; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent substitutions for some of the technical features. However, these modifications or substitutions do not deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

A data processing method, characterized in that the method includes:

Control multiple roadside devices to perform roadside data collection operations;

Perform data preprocessing on the roadside data provided by the multiple roadside devices to obtain collaborative data;

Edge computing processing is performed based on the collaborative data to perform collaborative tasks based on edge computing processing results.
The method according to claim 1, wherein the controlling multiple roadside devices to perform roadside data collection operations includes:

Generate collection instructions for controlling the collection operations of the multiple roadside devices according to the preset collection frequency or the obtained trigger signal;

Control the plurality of roadside devices to perform roadside data collection operations based on control parameters included in the collection instructions.
The method according to claim 2, characterized in that said performing data preprocessing on the roadside data provided by the plurality of roadside devices to obtain collaborative data includes:

Determine the collection timestamp of the collection instruction;

The roadside data is fused based on the same collection timestamp to obtain the collaborative data.
The method according to claim 2, characterized in that said performing data preprocessing on the roadside data provided by the plurality of roadside devices to obtain collaborative data includes:

Determine the execution timestamp of the roadside data;

The roadside data provided by the multiple roadside devices are fused based on the execution timestamp corresponding to the roadside data to obtain the collaborative data.
The method of claim 1, wherein performing edge computing processing based on the collaborative data to perform collaborative tasks based on edge computing processing results includes:

Perform data analysis on the collaborative data, and determine the data analysis results as the edge computing processing results;

Control the roadside device to perform collaborative tasks based on the edge computing processing results.
The method of claim 1, wherein performing edge computing processing based on the collaborative data to perform collaborative tasks based on edge computing processing results includes:

Perform data analysis on the collaborative data, and determine the data analysis results as the edge computing processing results;

The edge computing processing results are sent to vehicle devices that perform collaborative tasks.
The method according to any one of claims 1 to 6, further comprising:

Send the edge computing processing results to the cloud server.
The method according to claim 1, wherein the roadside device includes: a camera;

The data preprocessing of roadside data provided by the plurality of roadside devices to obtain collaborative data includes:

Obtain camera roadside data provided by multiple said cameras;

The camera roadside data provided by multiple cameras are aligned and fused to obtain the collaborative data.
The method according to claim 8, wherein the roadside device further includes: radar;

The data preprocessing of roadside data provided by the plurality of roadside devices to obtain collaborative data includes:

Obtain radar roadside data provided by multiple radars;

The camera roadside data and the radar roadside data are aligned and fused to obtain the collaborative data.
A roadside collaboration device, characterized in that the roadside collaboration device includes integrated: an edge computing module, and a plurality of roadside devices connected to the edge computing module;

The edge computing module is used to control multiple roadside devices to perform collection operations, preprocess data provided to the roadside through multiple roadside devices to obtain collaborative data, and perform edge computing processing on the collaborative data to facilitate edge computing based on edge computing. Process the results and perform collaborative tasks;

The plurality of roadside devices are used to collect roadside data according to the collection instructions issued by the edge computing module, and send the roadside data to the edge computing module.
The device according to claim 10, wherein the roadside coordination device includes a housing;

The edge computing module is integrated in the housing, and the multiple roadside devices are integrated on the housing and located at different positions of the housing, so that the multiple roadside devices have different perceptions. viewing angles and/or different perceptual abilities.
A roadside collaboration system, characterized in that the system includes: at least one roadside collaboration device and a cloud server; wherein,

The roadside collaboration equipment includes integrated: an edge computing module, and multiple roadside devices connected to the edge computing module; used to control multiple roadside devices to perform roadside data collection operations; to the Perform data preprocessing on roadside data provided by multiple roadside devices to obtain collaborative data; perform edge computing processing based on the collaborative data to perform collaborative tasks based on the edge computing processing results;

The cloud server is configured to receive edge computing processing results provided by the roadside collaboration device, so as to collaborate on tasks based on the edge computing processing results.
An electronic device including a memory and a processor; wherein,

The memory is used to store programs;

The processor is coupled to the memory and is configured to execute the program stored in the memory to implement the method described in any one of claims 1 to 9.
A non-transitory machine-readable storage medium, the non-transitory machine-readable storage medium The executable code is essentially stored, and when the executable code is executed by the processor of the electronic device, the processor is caused to perform the method according to any one of claims 1 to 9.