CN115311869A - Road event obtaining method and system based on intelligent network connection automobile computing power sharing and automobile - Google Patents

Road event obtaining method and system based on intelligent network connection automobile computing power sharing and automobile Download PDF

Info

Publication number
CN115311869A
CN115311869A CN202210906359.XA CN202210906359A CN115311869A CN 115311869 A CN115311869 A CN 115311869A CN 202210906359 A CN202210906359 A CN 202210906359A CN 115311869 A CN115311869 A CN 115311869A
Authority
CN
China
Prior art keywords
vehicle
computing power
computing
road
module
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202210906359.XA
Other languages
Chinese (zh)
Other versions
CN115311869B (en
Inventor
陶鹏
李增文
张清
王代毅
张俊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chongqing Changan Automobile Co Ltd
Original Assignee
Chongqing Changan Automobile Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chongqing Changan Automobile Co Ltd filed Critical Chongqing Changan Automobile Co Ltd
Priority to CN202210906359.XA priority Critical patent/CN115311869B/en
Publication of CN115311869A publication Critical patent/CN115311869A/en
Application granted granted Critical
Publication of CN115311869B publication Critical patent/CN115311869B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/01—Detecting movement of traffic to be counted or controlled
    • G08G1/04—Detecting movement of traffic to be counted or controlled using optical or ultrasonic detectors
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/09—Arrangements for giving variable traffic instructions
    • G08G1/0962—Arrangements for giving variable traffic instructions having an indicator mounted inside the vehicle, e.g. giving voice messages
    • G08G1/0967—Systems involving transmission of highway information, e.g. weather, speed limits
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/09—Arrangements for giving variable traffic instructions
    • G08G1/0962—Arrangements for giving variable traffic instructions having an indicator mounted inside the vehicle, e.g. giving voice messages
    • G08G1/0967—Systems involving transmission of highway information, e.g. weather, speed limits
    • G08G1/096766—Systems involving transmission of highway information, e.g. weather, speed limits where the system is characterised by the origin of the information transmission
    • G08G1/096775—Systems involving transmission of highway information, e.g. weather, speed limits where the system is characterised by the origin of the information transmission where the origin of the information is a central station

Abstract

The invention relates to a road event acquisition method and system based on intelligent networking automobile computing power sharing and an automobile, wherein the method comprises the following steps: the vehicle end sends the picture and the self-positioning information to the computing power sharing cloud platform through the vehicle-mounted communication unit OBU; and the cloud platform counts the residual computing power through the computing power scheduling of the cloud platform, establishes a dynamic network link, and sends the pictures related to the road scene to be processed to the vehicle end with available computing power. Because the calculation power sharing is provided by the invention, a large amount of environmental data acquired by the intelligent networked automobile can be comprehensively and accurately processed into the road events, the road events can reflect the actual road conditions around the automobile, the driving safety of the automobile can be improved, the environmental data acquired by the automobile is processed by each automobile end with the calculation power to form the road events and then is uploaded to the sharing cloud platform, so that other automobiles can make driving decisions according to the road events stored by the sharing cloud platform, and the occurrence of traffic accidents is effectively reduced.

Description

Road event obtaining method and system based on intelligent network connection automobile computing power sharing and automobile
Technical Field
The invention relates to the technical field of intelligent transportation, in particular to a road event acquisition technology and an automobile based on intelligent networked automobile computing power sharing.
Background
Along with the increasing development of urban traffic, the number of automobiles in China increases year by year, vehicles coming and going on roads increase day by year, and road traffic conditions tend to be more and more complicated, and a large number of accidents show that the main reason of the accidents is that the automobile owners cannot obtain real-time and accurate road condition data and vehicle surrounding conditions. With the development of the intelligent internet automobile technology, a large amount of information of road events can be accurately collected in real time by using a vehicle-mounted camera, then in the process of sensing the environment of the automatically-driven automobile, due to the limited computing capability of the automobile, a large amount of useful road event information is ignored in the control process of the automatically-driven automobile, even if part of the road information is identified, the information is only used for judging the driving decision of the automobile, and the sensed data information is not shared and is used for guiding other automobiles to make driving decisions, so that the occurrence of traffic accidents is reduced.
Chinese patent publication No. CN112084892B discloses a technology entitled "a road abnormal event detection management device and method thereof", which uses a camera at a road end on site to capture video graphic information around a road, and then uses a background server to process video graphic data to acquire road abnormal event information; chinese patent publication No. CN112581763A discloses a technology entitled "a method, apparatus, device and storage medium for detecting road events", which obtains road images collected and returned in a plurality of vehicle history periods, performs image processing on the uploaded images to determine road events contained in the road images, obtains all road events and road event information occurring in the history periods, and can accurately determine the types and/or locations of the road events. The two technologies do not adopt a mode of sharing the computing power of the intelligent networked automobile to calculate for the processing of a large amount of collected road data information so as to obtain road events and road environment information.
Disclosure of Invention
The invention aims to provide a road event acquisition method, a road event acquisition system and an automobile based on intelligent networking automobile computing power sharing, and solves the technical problems that: environmental data sensed by an automatic driving automobile ignores a large number of useful road events when the automatic driving automobile is controlled due to the limitation of the self computing capability, so that the driving of the automobile is unsafe, even if part of the road events are identified, the environmental data sensed by the automatic driving automobile is only used for judging the driving decision of the automobile, and the sensed environmental data is not shared and is used for guiding other automobiles to make driving decisions.
In order to solve the technical problems, the technical scheme adopted by the invention is as follows: a road event acquisition method based on intelligent networking automobile computing power sharing comprises the following steps of,
s01, a vehicle-mounted camera of a vehicle end acquires pictures related to a road scene, vehicle-mounted RTK inertial combination navigation acquires vehicle self-positioning information, and the vehicle end sends the pictures related to the road scene and the vehicle self-positioning information to a computing power sharing cloud platform through an OBU (on-board unit);
s02, the computing power sharing cloud platform counts the residual computing power of the vehicle end in a peripheral preset range through the computing power scheduling of the cloud platform, establishes a dynamic network link, and issues the pictures related to the road scene to be processed to the vehicle end with available computing power;
s03, preprocessing pictures related to a road scene by a vehicle end with available computing power, dividing the pictures related to the road scene into a training set and a verification set based on a preset proportion, performing iterative training on the training set and the verification set by using a built YOLOv4 network model to obtain a trained optimal YOLOv4 network model as an image recognition model to determine road events contained in the pictures related to the road scene;
s04, clustering multi-frame data of the pictures determined by the image recognition model, and determining road events occurring in the time stamps;
and S05, when the calculation of each vehicle end with available calculation power is completed, uploading the calculated road event to a storage sharing service cloud platform, determining vehicles in a target area according to the road event, and sending the identified road event information based on the state data of the vehicles in the target area.
Preferably, the first and second electrodes are formed of a metal,
in S03, the process of processing the picture data by the YOLOv4 network model is as follows:
s031, the Yolov4 network model obtains picture data;
s032, preprocessing picture data and extracting picture features by using CSPDarknet53 through a backbone network;
s033, the auxiliary layer adopts an SPP model to enhance the feature extraction effect;
s034, the detection layer detects the extracted road event characteristics by adopting PANet and outputs a road event result.
Preferably, the first and second electrodes are formed of a metal,
in the YOLOv4 network model, a training result is output through a full connection layer, and the training result comprises frame regression coordinates, a target classification result and confidence degree.
Preferably, the first and second liquid crystal display panels are,
the computing power sharing cloud platform comprises a computing power sharing node management module, a computing power sharing computing task counting module, a computing power sharing cloud network cooperative processing module and a computing power sharing edge vehicle monitoring module;
the computing power sharing node management module is used for unified management of the computing power sharing nodes, namely for unified management of vehicle ends with available computing power, and provides states of all computing power computing nodes, a CPU (central processing unit) system architecture, an operating system, IP (Internet protocol) addresses and CPU/memory resource use conditions in the computing power sharing node management module;
the computing power sharing computing task counting module counts all computing tasks in the managed computing power sharing cloud platform and provides computing task states, processing time, computing power expenses and processing time delay;
when a computing task needs to be processed by a cross-computing power sharing cloud platform or a road event recognition result needs to be issued by the cross-computing power sharing cloud platform, scheduling through a computing power sharing cloud network cooperative processing module under the cooperation of a mobile core network cloud platform;
the computing power sharing edge vehicle monitoring module provides state information and vehicle-mounted computing power resource information of all edge nodes accessed to the vehicle, wherein the state information comprises vehicle real-time geographic position information, driving direction data, driving speed data and vehicle energy and electric quantity data, and the vehicle-mounted computing power resource information comprises vehicle-end computing resource total amount, vehicle-end computing resource idle amount, storage resource data and network bandwidth data.
Preferably, the first and second liquid crystal display panels are,
the vehicle-mounted sensor unit of the vehicle end with the available calculation force comprises a vehicle self sensor, a PTK inertial integrated navigation, a vehicle-mounted camera and a vehicle-mounted communication unit OBU;
the PTK inertial integrated navigation system consists of an inertial navigation system and a GNSS positioning system, and utilizes a differential positioning technology to realize centimeter-level positioning accuracy of the vehicle, so as to collect positioning information of the vehicle and position road events;
the vehicle-mounted camera is a panoramic camera, a monocular camera or a binocular camera and is used for acquiring pictures related to a road scene;
and the vehicle-mounted communication unit OBU is provided with a 5G communication module and transmits and receives information by using a 5G communication technology.
Preferably, the first and second electrodes are formed of a metal,
the vehicle-mounted computing power task management module of the vehicle end with available computing power comprises a domain controller distribution computing/storage framework, an idle computing power and idle memory, a vehicle-mounted computing power virtualization and vehicle-end computing power scheduling module, a vehicle-mounted computing power resource/state data management module and a vehicle-mounted computing task receiving/returning module;
the domain controller is virtualized and containerized through a domain controller distribution computing/storage architecture technology, so that the idle computing power and the idle memory of the domain controller are scheduled and used by a domain controller sharing cloud platform, the using conditions of the computing power and the memory of the domain controller are monitored and managed in real time through a vehicle-mounted system, a vehicle-mounted computing task receiving module is used for receiving computing tasks, and a vehicle-mounted computing task returning module is used for returning computing results to a storage sharing service cloud platform.
Preferably, the first and second electrodes are formed of a metal,
the vehicle end with available computing power executes computing tasks and is a driving area controller and a cockpit area controller or a central calculator.
The invention also provides a road event acquisition system based on intelligent networking automobile computing power sharing, which comprises:
the vehicle-end sending module is used for collecting a picture related to a road scene by a vehicle-mounted camera of a vehicle end, collecting vehicle self-positioning information by vehicle-mounted RTK inertial integrated navigation, and sending the picture related to the road scene and the vehicle self-positioning information to the computing power sharing cloud platform by the vehicle end through a vehicle-mounted communication unit OBU;
the cloud computing power scheduling module is used for counting the remaining computing power of the vehicle ends in the peripheral preset range through computing power scheduling of the cloud platform by the computing power sharing cloud platform, establishing a dynamic network link, and sending the pictures related to the road scene to be processed to the vehicle ends with available computing power;
the processing module is used for preprocessing the pictures related to the road scene at the vehicle end with available computing power, dividing the pictures related to the road scene into a training set and a verification set based on a preset proportion, performing iterative training on the training set and the verification set by using the built YOLOv4 network model to obtain a trained optimal YOLOv4 network model as an image recognition model so as to determine road events contained in the pictures related to the road scene;
the clustering module is used for clustering multi-frame data of the pictures determined by the image recognition model and determining road events occurring in the time stamps;
and the uploading module is used for uploading the calculated road events to the storage sharing service cloud platform when the calculation of each vehicle end with available calculation power is completed, determining vehicles in the target area according to the road events, and sending the identified road event information based on the state data of the vehicles in the target area.
Preferably, the first and second electrodes are formed of a metal,
in the processing module, the process of processing the picture data by the YOLOv4 network model is as follows: acquiring picture data by using a YOLOv4 network model; the main network adopts CSPDarknet53 to preprocess the picture data and extract the picture characteristics; the auxiliary layer adopts an SPP model to enhance the characteristic extraction effect; and the detection layer detects the extracted road event characteristics by adopting PANet and outputs a road event result.
Preferably, the first and second electrodes are formed of a metal,
in the YOLOv4 network model, a training result is output through a full connection layer, and the training result comprises a frame regression coordinate, a target classification result and a confidence degree.
Preferably, the first and second electrodes are formed of a metal,
the computing power sharing cloud platform comprises a computing power sharing node management module, a computing power sharing computing task counting module, a computing power sharing cloud network cooperative processing module and a computing power sharing edge vehicle monitoring module;
the computing power sharing node management module is used for unified management of the computing power sharing nodes, namely for unified management of vehicle ends with available computing power, and provides states of all computing power computing nodes, a CPU (central processing unit) system architecture, an operating system, IP (Internet protocol) addresses and CPU/memory resource use conditions in the computing power sharing node management module;
the computing power sharing computing task counting module counts all computing tasks in the managed computing power sharing cloud platform and provides computing task states, processing time, computing power expenses and processing time delay;
when a computing task needs to be processed by a cross-computing power sharing cloud platform or a road event recognition result needs to be issued by the cross-computing power sharing cloud platform, scheduling through a computing power sharing cloud network cooperative processing module under the cooperation of a mobile core network cloud platform;
the computing power sharing edge vehicle monitoring module provides state information of all edge nodes accessed to the vehicle and vehicle-mounted computing power resource information, wherein the state information comprises vehicle real-time geographic position information, driving direction data, driving speed data and vehicle energy and electric quantity data, and the vehicle-mounted computing power resource information comprises vehicle-end computing resource total quantity, vehicle-end computing resource idle quantity, storage resource data and network bandwidth data.
Preferably, the first and second electrodes are formed of a metal,
the vehicle-mounted sensor unit of the vehicle end with the available calculation force comprises a vehicle self sensor, a PTK inertial integrated navigation, a vehicle-mounted camera and a vehicle-mounted communication unit OBU;
the PTK inertial integrated navigation system consists of an inertial navigation system and a GNSS positioning system, and utilizes a differential positioning technology to realize centimeter-level positioning accuracy of the vehicle, so as to collect positioning information of the vehicle and position road events;
the vehicle-mounted camera is a panoramic camera, a monocular camera or a binocular camera and is used for acquiring pictures related to a road scene;
and the vehicle-mounted communication unit OBU is provided with a 5G communication module and transmits and receives information by using a 5G communication technology.
Preferably, the first and second electrodes are formed of a metal,
the vehicle-mounted computing power task management module of the vehicle end with available computing power comprises a domain controller distribution computing/storage framework, an idle computing power and idle memory, a vehicle-mounted computing power virtualization and vehicle end computing power scheduling module, a vehicle-mounted computing power resource/state data management module and a vehicle-mounted computing task receiving/returning module;
the domain controller is virtualized and containerized through a domain controller distribution computing/storage architecture technology, so that idle computing power and idle memory of the domain controller are scheduled and used by a domain controller sharing cloud platform, the computing power and the memory use condition of the domain controller are monitored and managed in real time through a vehicle-mounted system, a vehicle-mounted computing task receiving module is used for receiving computing tasks, and a vehicle-mounted computing task returning module is used for returning computing results to a storage sharing service cloud platform.
Preferably, the first and second liquid crystal display panels are,
the vehicle end with available computing power executes computing tasks and is a driving area controller and a cockpit area controller or a central calculator.
The invention also provides an automobile, wherein the image data which is perceived by the automobile and is related to the road scene is processed into the road event by the method, so that the automobile is controlled more safely, and other automobiles make driving decisions based on the road event.
By adopting the technical scheme, the invention has the following beneficial technical effects: the method and the system utilize all intelligent networked automobiles running on the road to collect road data, upload the collected road environment crowdsourcing data to the intelligent networked automobile computing power sharing cloud platform by utilizing a V2X communication technology, and process the road data and issue information.
Drawings
FIG. 1 is an embodiment of a road event acquisition system for intelligent networked automobile computing power sharing;
FIG. 2 is a diagram illustrating architecture of intelligent networked automobile hardware and shared functions in accordance with an embodiment of the present invention;
FIG. 3 is an implementation flow chart of the intelligent networked automobile computing power sharing cloud platform for identifying and acquiring a road event;
FIG. 4 is a flow diagram of one embodiment of a process for processing video information by a computational sharing node using a YOLOv4 model;
FIG. 5 is a diagram of a YOLOv4 network model architecture provided for implementing road event recognition;
fig. 6 is a functional module structure diagram of a computing cloud platform for implementing the vehicle-mounted computing power sharing center.
Detailed Description
The invention will be further explained with reference to the drawings.
The invention aims to solve the problems that road events can be accurately acquired in real time in an intelligent traffic system, all intelligent networked automobiles running on a road are used for acquiring data, the acquired road environment crowdsourcing data is uploaded to an intelligent networked automobile computing power sharing cloud platform by using a V2X communication technology, and road data processing and information publishing are carried out.
Road events mentioned in the following embodiments include, but are not limited to, road surface state, road construction, traffic accidents, weather conditions, etc., and the images around the road are collected by the vehicle-mounted camera for image processing, road object detection, whether the road conditions are abnormal, image road surface segmentation, weather identification, and then an early warning record is generated, and early warning information is specifically sent to vehicles located in the target area.
The hardware of the intelligent networked automobile mainly comprises a vehicle-mounted computing unit, a storage unit, a vehicle-mounted communication unit, a vehicle-mounted camera and RTK inertial integrated navigation. The vehicle-mounted computing unit mainly comprises a vehicle-end driving domain controller and a cockpit domain controller, and is a central calculator if a vehicle end adopts a central computing architecture, and the part mainly provides vehicle-mounted computing resources and is a core part for executing vehicle-end computing tasks; the data storage unit is used for meeting the data storage requirement of the vehicle-end computing task; the vehicle-mounted communication unit OBU is used for communication between vehicles, between vehicles and road ends and between vehicles and network ends; the vehicle-mounted camera can be a monocular camera, a binocular camera, a panoramic camera and the like and is used for collecting road environment information around the vehicle; RTK inertial combination navigation utilizes differential positioning technique, can realize the vehicle centimeter level positioning accuracy, improves the accurate positioning of road incident.
The logical principle of the invention is set out as follows: when a large number of intelligent networked automobiles run on a road, a vehicle-mounted camera acquires a large amount of video image information around the automobiles, vehicle-mounted RTK inertial integrated navigation acquires vehicle self-positioning information, and then a large amount of video image information, vehicle self-information, driving planning information and the like acquired by a 5G communication technology are transmitted to an intelligent networked automobile computing power sharing platform in real time through an OBU (on board unit). After the intelligent internet automobile computing power sharing platform receives video graphic information, vehicle self information and driving path planning information, the cloud platform computing power scheduling system counts the residual computing power of the intelligent internet automobile within a peripheral preset range, vehicle-mounted computing power sharing is achieved through technologies such as a vehicle-mounted computing power virtualization and computing power scheduling module, a vehicle-mounted computing power resource data transmission module, a vehicle-mounted state data transmission module and a computing task management module, dynamic network connection is built for a large number of intelligent internet automobiles with idle computing power, and a large number of road peripheral video graphic information, vehicle self information and driving path planning information which need to be processed are issued to dynamic computing power network nodes.
The calculation force node is an intelligent networked automobile with residual calculation force, and under the condition that calculation force requirements are sent within a preset range, the automobile and the calculation force sharing platform are supported by means of a Uu interface or a PC5 interface, so that the time delay of end-to-end data transmission can be reduced, and network conformity caused by return of big data is reduced.
And a pattern recognition processing model is constructed by utilizing deep learning-based Yolov4 network model, iterative training is carried out through a large amount of road video image information returned by the intelligent internet automobile until a loss function is converged, and finally the trained optimal Yolov4 network model is obtained and used as an image recognition model, so that accurate recognition of a road scene is realized.
The method comprises the steps of obtaining positioning information of road event information, mainly positioning through RTK inertial navigation on an intelligent network vehicle for acquiring road events, matching the positioning information of the vehicle and the road information acquired by a camera through a timestamp, determining the accurate position of the road event, uploading acquired video image information and self positioning information to a computing power sharing cloud platform of the intelligent network vehicle by using a V2X communication technology when the intelligent network vehicle utilizes the technology, identifying the type of the road event through a YOLOv4 network model by the computing power sharing cloud platform, combining a positioning method and a positioning system for synchronizing RTK combined inertial navigation and a UTC timestamp, obtaining UTC time of unified frame data by synchronizing video streams through a timestamp matching algorithm based on UTC time correction, and sequentially combining RTK position information to obtain the actual physical position of the road event.
The road event processing process of the automobile calculation force node comprises the following steps: after each computational power node processes the computational power task issued by the computational power sharing cloud platform, the obtained road event information and positioning information are transmitted back to the intelligent networked automobile storage sharing cloud platform, and the road event information on each road is updated on the high-precision map in real time. According to self-positioning information or driving planning paths uploaded by other intelligent networked automobiles, the cloud platform sends road event information to vehicles in the target area in a targeted mode.
The automobile owner of the new energy intelligent networked automobile adds own automobile into the calculation force sharing cluster, under the premise that the functions and the performance of the automobile are not influenced, the vehicle-mounted idle time and idle calculation force of the intelligent networked automobile are provided, and the corresponding reward of the automobile owner is fed back through calculation time or calculation data quantity.
As shown in fig. 1, the base station of the road event acquiring system includes a first base station 130, a second base station 131, etc., the cloud server of the base station includes a first cloud server 140, a second cloud server 141, etc., the computing power node of the remote smart internet automobile includes a first computing power node 120, a second computing power node 121, a third computing power node 122, a fourth computing power node 123, a fifth computing power node 124, etc., the road event data acquiring vehicle includes a first vehicle 110, a second vehicle 111, a third vehicle 112, a fourth vehicle 113, etc., during the road event acquiring process, the smart internet automobile is a data acquirer and is also a road event acquirer, the cloud servers of the base station and the base station may serve as the computing power sharing cloud platform of the smart internet automobile, the computing power sharing cloud platform may be coupled to a mobile core network cloud platform 150 in a communication manner, and the mobile core network cloud platform 150 may provide services for the computing power sharing cloud platform through a 5G network. The intelligent networked automobiles in the base station signal range can be used as parallel computing nodes, computing force tasks are distributed through a computing force sharing cloud platform, the intelligent networked automobiles moving in the base station signal range can be used as acquisition vehicles of road event information, image information on roads is acquired through vehicle-mounted cameras, and the detection area 160 is the detection range of the vehicle-mounted cameras.
When a large number of intelligent networked automobiles run on a road, a vehicle-mounted camera acquires a large number of video image information around the automobiles, vehicle-mounted RTK inertial integrated navigation acquires vehicle self-positioning information, and the acquired video image information and the vehicle self-information are sent to an intelligent networked automobile computing power sharing cloud platform in real time by using a 5G communication technology through a vehicle-mounted communication unit OBU. After the intelligent networked automobile computing power sharing cloud platform receives the video graphic information, the computing power sharing platform processes the road video data through the vehicle-mounted computing power virtualization and containerization module, the computing power scheduling module, the vehicle-mounted computing power resource data transmission module, the vehicle-mounted state data transmission module and the computing task management module.
As shown in fig. 3, the present invention further provides a road event acquisition method based on intelligent internet vehicle computing power sharing, comprising the steps of,
s01, a vehicle-mounted camera of a vehicle end acquires a picture related to a road scene, vehicle-mounted RTK inertial integrated navigation acquires vehicle self-positioning information, and the vehicle end sends the picture related to the road scene and the vehicle self-positioning information to a computing power sharing cloud platform through a vehicle-mounted communication unit OBU; utilize the on-vehicle camera of intelligent networking car to gather a large amount of pictures relevant with the road scene, include: vehicles, pedestrians, road-to-ground conditions, obstacles within the road, weather conditions on the road, etc. Road image information and vehicle self information collected by a vehicle are transmitted by using a 5G communication technology and uploaded to a computing power sharing cloud platform scheduling system in real time.
S02, counting the residual calculation power of the vehicle end in a peripheral preset range through calculation power scheduling of the cloud platform by the calculation power sharing cloud platform, establishing a dynamic network link, and issuing a picture related to a road scene to be processed to the vehicle end with available calculation power; after receiving the information, the cloud platform computing power scheduling system counts the remaining computing power of the intelligent networked automobile in a peripheral preset range, establishes dynamic network connection, and sends a large amount of information to be processed to available computing power nodes to perform identification and computation of road events.
S03, preprocessing pictures related to a road scene by a vehicle end with available computing power, dividing the pictures related to the road scene into a training set and a verification set based on a preset proportion, performing iterative training on the training set and the verification set by using a built YOLOv4 network model to obtain a trained optimal YOLOv4 network model as an image recognition model so as to determine road events contained in the pictures related to the road scene; the computing power sharing cloud platform preprocesses each acquired road picture, iterative training is carried out by utilizing the built YOLOv4 network model, the trained optimal YOLOv4 network model is obtained and serves as an image recognition model, and road events contained in the road images are determined.
S04, clustering multi-frame data of the pictures determined by the image recognition model, and determining road events occurring in the time stamps; clustering multi-frame data of the road image determined by the YOLOv4 network model, and determining a road event occurring in an image data timestamp, wherein the road event information comprises: event type and location information.
And S05, when the calculation of each vehicle end with available calculation power is completed, uploading the calculated road event to a storage sharing service cloud platform, determining vehicles in a target area according to the road event, and sending the identified road event information based on the state data of the vehicles in the target area. When the calculation of each calculation force node is completed, the calculated road event information is uploaded to an intelligent network automobile storage service cloud platform, vehicles in a target area are determined according to traffic events, and the identified road event information is sent based on state data of the vehicles in the target area.
After the intelligent networked automobile uploads the collected road data information to the computing power sharing cloud platform, the road video information to be processed is issued to each computing power node.
As shown in fig. 4, in S03, the flow of processing the picture data by the YOLOv4 network model is as follows:
s031, the Yolov4 network model obtains picture data; and the road video data collected by the vehicle-mounted camera is used as input data of the YOLOv network model in each computational power node.
S032, preprocessing the picture data and extracting picture features by using CSPDarknet53 through the backbone network; the YOLOv4 backbone network adopts CSPDarknet53 to preprocess video graphics data and extract the features of road events, including: the CSPDarknet53 main network comprises 5 CSP modules, the sizes of convolution kernels in front of the modules are 3 multiplied by 3, the stride is 2, and the network learning capacity can be further enhanced.
S033, the auxiliary layer adopts an SPP model to enhance the feature extraction effect; the Yolov4 auxiliary layer mainly adopts the feature extraction enhancement effect of an SPP model, adopts a maximum pooling mode of 1 × 1,5 × 5,9 × 9 and 13 × 13, can increase some related shallow feature inputs extracted from the front trunk network, and processes and enhances the shallow features extracted from the trunk network from the part, so that the modeled features are the desired features and more important context features are separated.
S034, detecting the extracted road event characteristics by the detection layer by using PANet, and outputting a road event result; the detection layer of YOLOv4 mainly adopts PANet to detect the extracted road event characteristics and outputs a road event result, which is the most core part of the algorithm.
As shown in fig. 5, in the YOLOv4 network model, a training result is output through a full connection layer, and the training result includes a frame regression coordinate, a target classification result, and a confidence level. And outputting a training result through the full connection layer, wherein the training result comprises a frame regression coordinate, a target classification result and a confidence coefficient. The output of each layer of the Yolov4 network, how each layer gets the annotation behind each line, and what is not annotated is to convolute the diagnostic chart of the previous line. The YOLOv4 network has a total of 161 layers, and the calculated amount is 128.46BFOPS in total at 608 × 608 resolution.
As shown in fig. 6, the computing power sharing cloud platform includes a computing power sharing node management module, a computing power sharing computing task statistics module, a computing power sharing cloud network cooperative processing module, and a computing power sharing edge vehicle monitoring module; the computing cloud platform of the computing power sharing center comprises a computing power sharing node management module, a computing power sharing computing task statistic module, a computing power sharing cloud network cooperative processing module and a computing power sharing edge vehicle monitoring module.
The computing power sharing node management module is used for unified management of the computing power sharing nodes, namely for unified management of vehicle ends with available computing power, and provides states of all computing power computing nodes, a CPU system architecture, an operating system, IP addresses and CPU/memory resource use conditions in the computing power sharing node management module; the computing power sharing node management module: the module is used for the unified management of the computing power sharing nodes, and the states of all the computing power computing nodes, the CPU system architecture, the operating system, the IP address and the CPU/memory resource use condition can be checked in the module. Before the computing power sharing node is deployed on the intelligent networked automobile, a system architecture including the computing power sharing node, a system version, a CPU (central processing unit) and memory resource allocation, a network connection address and the like need to be established on the automobile, and the automobile is used as the computing power sharing node after the establishment is completed.
The computing power sharing computing task counting module counts all computing tasks in the managed computing power sharing cloud platform and provides computing task states, processing time, computing power expenditure and processing time delay; the calculation force sharing calculation task statistic module comprises: the module can count all the computing tasks in the computing power sharing platform, the computing power sharing nodes can synchronously transmit computing task data to the module, and all the information such as computing task states, processing time, computing power expenses and processing time delay in the current platform can be checked.
When a computing task needs to be processed by a cross-computing power sharing cloud platform or a road event recognition result needs to be issued by the cross-computing power sharing cloud platform, scheduling through a computing power sharing cloud network cooperative processing module under the cooperation of a mobile core network cloud platform; the computing power sharing cloud network cooperative processing module: the module is mainly applied to a cloud platform needing to be moved, a core network cloud platform and a computing power sharing cloud platform are cooperated, scheduling of computing tasks or issuing functions of different cloud ends of road event recognition results are carried out, and when the computing tasks need to be processed across platforms or the computing results need to be issued across cloud ends, scheduling needs to be carried out through the module.
The computing power sharing edge vehicle monitoring module provides state information and vehicle-mounted computing power resource information of all edge nodes accessed to the vehicle, wherein the state information comprises vehicle real-time geographic position information, driving direction data, driving speed data and vehicle energy and electric quantity data, and the vehicle-mounted computing power resource information comprises vehicle-end computing resource total amount, vehicle-end computing resource idle amount, storage resource data and network bandwidth data. Computing force sharing edge vehicle monitoring module: the module can check the state information and vehicle-mounted computing power resource information of all edge nodes accessed to the vehicle. The vehicle state information comprises vehicle real-time geographic position information, driving direction data, driving speed data, vehicle energy and electric quantity data and the like. The vehicle-mounted resource information mainly comprises: the total amount of the vehicle-end computing resources, the idle amount of the vehicle-end computing resources, the storage resource data and the network bandwidth data.
As shown in fig. 2, the vehicle-mounted sensor unit of the vehicle end with available computing power comprises a vehicle self sensor, a PTK inertial integrated navigation, a vehicle-mounted camera, and a vehicle-mounted communication unit OBU; the vehicle-mounted sensor unit 210 includes a vehicle self-sensor 220, an RTK inertial combination navigation 221, a vehicle-mounted camera 222, and a vehicle-mounted communication unit OBU223.
The PTK inertial integrated navigation system consists of an inertial navigation system and a GNSS positioning system, and utilizes a differential positioning technology to realize centimeter-level positioning precision of the vehicle, so as to acquire positioning information of the vehicle and position road events; the RTK inertial integrated navigation system consists of an inertial navigation system and a GNSS positioning system, and by utilizing a differential positioning technology, centimeter-level positioning accuracy of a vehicle can be realized, so that the RTK inertial integrated navigation system is used for acquiring current positioning information of the vehicle and accurately positioning a road event.
The vehicle-mounted camera is a panoramic camera, a monocular camera or a binocular camera and is used for acquiring pictures related to a road scene; the vehicle-mounted camera can be a panoramic camera, a monocular camera, a binocular camera and the like, and sends acquired road video graphic information to the OBU when the vehicle moves.
The vehicle-mounted communication unit OBU is provided with a 5G communication module, and sends and receives information by using a 5G communication technology; the OBU is provided with a 5G communication module, and can transmit and receive information by using a 5G communication technology.
As shown in fig. 2, the vehicle-mounted computing power task management module of the vehicle end with available computing power includes a domain controller distribution computing/storage architecture, an idle computing power and idle memory, a vehicle-mounted computing power virtualization and vehicle-end computing power scheduling module, a vehicle-mounted computing power resource/state data management module, and a vehicle-mounted computing task receiving/returning module; the vehicle-mounted computing power task management module 211 includes a domain controller distributed computing/storage framework 234, an idle computing power and idle memory 233, a vehicle-mounted computing power virtualization and computing power scheduling module 234, a vehicle-mounted computing power resource/state data management module 231, and a vehicle-mounted computing task receiving/returning module 230.
The domain controller is virtualized and containerized through a domain controller distribution computing/storage architecture technology, so that idle computing power and idle memory of the domain controller are scheduled and used by a domain controller sharing cloud platform, the computing power and the memory use condition of the domain controller are monitored and managed in real time through a vehicle-mounted system, a vehicle-mounted computing task receiving module is used for receiving computing tasks, and a vehicle-mounted computing task returning module is used for returning computing results to a storage sharing service cloud platform. The intelligent automobile domain controller is virtualized and containerized through a domain controller distributed computing/storing architecture technology, idle computing power and idle memory of the domain controller can be scheduled and used by a computing power sharing cloud platform, the using conditions of the computing power and the memory of the vehicle-mounted domain controller are monitored and managed in real time through a vehicle-mounted system, computing tasks are received through a receiving module, and computing results are returned to a computing power task distributing platform through a returning module.
Specifically, the vehicle end with available computing power executes computing tasks, namely a driving area controller and a cockpit area controller, or a central calculator. The vehicle-mounted computing unit mainly comprises a vehicle-end driving domain controller and a cockpit domain controller, and is a central calculator if a vehicle-end adopts a central computing architecture, and the part mainly provides vehicle-mounted computing resources and is a core part for executing vehicle-end computing tasks.
The invention also provides a road event acquisition system based on intelligent networking automobile computing power sharing, which is characterized by comprising the following steps:
the vehicle-end sending module is used for collecting pictures related to a road scene by a vehicle-mounted camera at a vehicle end, collecting vehicle self-positioning information by vehicle-mounted RTK inertial combination navigation, and sending the pictures related to the road scene and the vehicle self-positioning information to the computing power sharing cloud platform by the vehicle end through the vehicle-mounted communication unit OBU; utilize the on-vehicle camera of intelligent networking car to gather a large amount of pictures relevant with the road scene, include: vehicles, pedestrians, road-to-ground conditions, obstacles within the road, weather conditions on the road, etc. Road image information and vehicle self information collected by vehicles are transmitted by using a 5G communication technology and uploaded to a computing power sharing cloud platform scheduling system in real time.
The cloud computing power scheduling module is used for counting the remaining computing power of the vehicle ends in the peripheral preset range through computing power scheduling of the computing power sharing cloud platform, establishing a dynamic network link and issuing pictures related to the road scene to be processed to the vehicle ends with available computing power; after receiving the information, the cloud platform computing power scheduling system counts the residual computing power of the intelligent networked automobile in a peripheral preset range, establishes dynamic network connection, and sends a large amount of information to be processed to available computing power nodes to perform identification and computation of road events.
The processing module is used for preprocessing the pictures related to the road scene at the vehicle end with available computing power, dividing the pictures related to the road scene into a training set and a verification set based on a preset proportion, performing iterative training on the training set and the verification set by using the built YOLOv4 network model to obtain a trained optimal YOLOv4 network model as an image recognition model so as to determine road events contained in the pictures related to the road scene; the computing power sharing cloud platform preprocesses each acquired road picture, iterative training is carried out by utilizing the built YOLOv4 network model, the trained optimal YOLOv4 network model is obtained and serves as an image recognition model, and road events contained in the road image are determined.
The clustering module is used for clustering multi-frame data of the pictures determined by the image recognition model and determining road events occurring in the time stamps; clustering multi-frame data of the road image determined by the YOLOv4 network model, and determining a road event occurring in an image data timestamp, wherein the road event information comprises: event type and location information.
And the uploading module is used for uploading the calculated road events to the storage sharing service cloud platform when the calculation of each vehicle end with available calculation power is completed, determining vehicles in the target area according to the road events, and sending the identified road event information based on the state data of the vehicles in the target area. When the calculation of each calculation force node is completed, the calculated road event information is uploaded to an intelligent network-connected automobile storage service cloud platform, vehicles in a target area are determined according to traffic events, and the identified road event information is sent based on state data of the vehicles in the target area.
In particular, the amount of the solvent to be used,
in the processing module, the flow of processing the picture data by the YOLOv4 network model is as follows: acquiring picture data by using a YOLOv4 network model; the main network adopts CSPDarknet53 to preprocess the picture data and extract the picture characteristics; the auxiliary layer adopts an SPP model to enhance the characteristic extraction effect; and the detection layer detects the extracted road event characteristics by adopting PANet and outputs a road event result. Road video data collected by a vehicle-mounted camera is used as input data of a YOLOv network model in each computational power node; the YOLOv4 backbone network adopts CSPDarknet53 to preprocess video graphics data and extract features of road events, including: the CSPDarknet53 main network comprises 5 CSP modules, the sizes of convolution kernels in front of the modules are 3 multiplied by 3, the stride is 2, and the network learning capacity can be further enhanced; the YOLOv4 auxiliary layer mainly adopts an SPP model to enhance the feature extraction effect, adopts a maximum pooling mode of 1 × 1,5 × 5,9 × 9 and 13 × 13, can increase some related shallow feature inputs extracted from the front trunk network, and processes and enhances the shallow features extracted from the trunk network by the part, so that the features learned by the model are the desired features and more important context features are separated; the detection layer of YOLOv4 mainly adopts PANet to detect the extracted road event characteristics and output the road event result, and is the most core part of the algorithm.
Specifically, in the YOLOv4 network model, a training result is output through a full connection layer, and the training result includes a frame regression coordinate, a target classification result and a confidence level. And outputting a training result through the full connection layer, wherein the training result comprises a frame regression coordinate, a target classification result and a confidence coefficient. The output of each layer of the Yolov4 network, how each layer gets the annotation behind each line, and what is not annotated is to convolute the diagnostic chart of the previous line. The YOLOv4 network has a total of 161 layers, and the calculated amount is 128.46BFOPS in total at 608 × 608 resolution.
The computing power sharing cloud platform comprises a computing power sharing node management module, a computing power sharing computing task statistic module, a computing power sharing cloud network cooperative processing module and a computing power sharing edge vehicle monitoring module; the computing cloud platform of the computing power sharing center comprises a computing power sharing node management module, a computing power sharing computing task statistic module, a computing power sharing cloud network cooperative processing module and a computing power sharing edge vehicle monitoring module.
The computing power sharing node management module is used for unified management of the computing power sharing nodes, namely for unified management of vehicle ends with available computing power, and provides states of all computing power computing nodes, a CPU system architecture, an operating system, IP addresses and CPU/memory resource use conditions in the computing power sharing node management module; the computing power sharing node management module: the module is used for the unified management of the computing power sharing nodes, and the states of all the computing power computing nodes, the CPU system architecture, the operating system, the IP address and the CPU/memory resource use condition can be checked in the module. Before the computing power sharing node is deployed on the intelligent networked automobile, a system architecture including the computing power sharing node, a system version, a CPU (central processing unit) and memory resource allocation, a network connection address and the like need to be established on the automobile, and the automobile is used as the computing power sharing node after the establishment is completed.
The computing power sharing computing task counting module counts all computing tasks in the managed computing power sharing cloud platform and provides computing task states, processing time, computing power expenses and processing time delay; the calculation force sharing calculation task statistic module comprises: the module can count all the computing tasks in the computing power sharing platform, the computing power sharing nodes can synchronously transmit computing task data to the module, and all the information such as computing task states, processing time, computing power expenses, processing time delay and the like in the current platform can be checked.
When a computing task needs to be processed by a cross-computing power sharing cloud platform or a road event recognition result needs to be issued by the cross-computing power sharing cloud platform, scheduling through a computing power sharing cloud network cooperative processing module under the cooperation of a mobile core network cloud platform; the computing power sharing cloud network cooperative processing module: the module is mainly applied to a cloud platform needing to be moved, a core network cloud platform cooperates with a computing power sharing cloud platform to perform scheduling of computing tasks or issuing functions of different cloud ends of road event recognition results, and when the computing tasks need to be processed across platforms or computing results need to be issued across the cloud ends, scheduling needs to be performed through the module.
The computing power sharing edge vehicle monitoring module provides state information and vehicle-mounted computing power resource information of all edge nodes accessed to the vehicle, wherein the state information comprises vehicle real-time geographic position information, driving direction data, driving speed data and vehicle energy and electric quantity data, and the vehicle-mounted computing power resource information comprises vehicle-end computing resource total amount, vehicle-end computing resource idle amount, storage resource data and network bandwidth data. Computing force sharing edge vehicle monitoring module: the module can check the state information and vehicle-mounted computing power resource information of all edge nodes accessed to the vehicle. The vehicle state information comprises vehicle real-time geographic position information, driving direction data, driving speed data, vehicle energy and electric quantity data and the like. The vehicle-mounted resource information mainly comprises: the total amount of the vehicle-end computing resources, the idle amount of the vehicle-end computing resources, the storage resource data and the network bandwidth data.
Specifically, the vehicle-mounted sensor unit of the vehicle end with the available computing power comprises a vehicle sensor, a PTK inertial integrated navigation, a vehicle-mounted camera and a vehicle-mounted communication unit OBU; the vehicle-mounted sensor unit 210 includes a vehicle self-sensor 220, an RTK inertial combination navigation 221, a vehicle-mounted camera 222, and a vehicle-mounted communication unit OBU223.
The PTK inertial integrated navigation system consists of an inertial navigation system and a GNSS positioning system, and utilizes a differential positioning technology to realize centimeter-level positioning accuracy of the vehicle, so as to collect positioning information of the vehicle and position road events; the RTK inertial integrated navigation system consists of an inertial navigation system and a GNSS positioning system, and by utilizing a differential positioning technology, centimeter-level positioning accuracy of a vehicle can be realized, so that the current positioning information of the vehicle can be acquired, and the RTK inertial integrated navigation system can be used for accurately positioning a road event.
The vehicle-mounted camera is a panoramic camera, a monocular camera or a binocular camera and is used for acquiring pictures related to a road scene; the vehicle-mounted camera can be a panoramic camera, a monocular camera, a binocular camera and the like, and when the vehicle moves, the acquired road video graphic information is sent to the OBU.
The vehicle-mounted communication unit OBU is provided with a 5G communication module, and sends and receives information by using a 5G communication technology; the OBU is provided with a 5G communication module, and can transmit and receive information by using a 5G communication technology.
Specifically, the vehicle-mounted computing power task management module of the vehicle end with available computing power comprises a domain controller distribution computing/storage framework, an idle computing power and idle memory, a vehicle-mounted computing power virtualization and vehicle end computing power scheduling module, a vehicle-mounted computing power resource/state data management module and a vehicle-mounted computing task receiving/returning module; the vehicle computing power task management module 211 includes a domain controller distributed computing/storage architecture 234, an idle computing power and idle memory 233, a vehicle computing power virtualization and scheduling module 234, a vehicle computing power resource/status data management module 231, and a vehicle computing task receiving/returning module 230.
The domain controller is virtualized and containerized through a domain controller distribution computing/storage architecture technology, so that idle computing power and idle memory of the domain controller are scheduled and used by a domain controller sharing cloud platform, the computing power and the memory use condition of the domain controller are monitored and managed in real time through a vehicle-mounted system, a vehicle-mounted computing task receiving module is used for receiving computing tasks, and a vehicle-mounted computing task returning module is used for returning computing results to a storage sharing service cloud platform; the intelligent automobile domain controller is virtualized and containerized through a domain controller distributed computing/storing architecture technology, idle computing power and idle memory of the domain controller can be scheduled and used by a computing power sharing cloud platform, the using conditions of the computing power and the memory of the vehicle-mounted domain controller are monitored and managed in real time through a vehicle-mounted system, computing tasks are received through a receiving module, and computing results are returned to a computing power task distributing platform through a returning module.
Specifically, the vehicle end with available computing power executes computing tasks, namely a driving area controller and a cockpit area controller, or a central calculator; the vehicle-mounted computing unit mainly comprises a vehicle-end driving domain controller and a cockpit domain controller, and is a central calculator if a vehicle-end adopts a central computing architecture, and the part mainly provides vehicle-mounted computing resources and is a core part for executing vehicle-end computing tasks.
The invention also provides an automobile, wherein the image data which is perceived by the automobile and is related to the road scene is processed into the road event by the method, so that the automobile is controlled more safely, and other automobiles make driving decisions based on the road event.

Claims (15)

1. A road event acquisition method based on intelligent networked automobile computing power sharing is characterized by comprising the following steps of,
s01, a vehicle end collects pictures related to a road scene and vehicle self-positioning information, and sends the pictures related to the road scene and the vehicle self-positioning information to a computing power sharing cloud platform through an OBU (on-board unit);
s02, the computing power sharing cloud platform counts the residual computing power of the vehicle end in a peripheral preset range through the computing power scheduling of the cloud platform, establishes a dynamic network link, and issues the pictures related to the road scene to be processed to the vehicle end with available computing power;
s03, preprocessing pictures related to a road scene by a vehicle end with available computing power, dividing the pictures related to the road scene into a training set and a verification set based on a preset proportion, and performing iterative training by using a built target detection network model to obtain a trained optimal network model serving as an image recognition model so as to determine road events contained in the pictures related to the road scene;
s04, clustering multi-frame data of the pictures determined by the image recognition model, and determining road events occurring in the time stamps;
and S05, when the calculation of each vehicle end with the available calculation power is completed, uploading the calculated road events to a storage sharing service cloud platform, determining vehicles in a target area according to the road events, and sending the identified road event information based on the state data of the vehicles in the target area.
2. The method according to claim 1, wherein in S03, the target detection network model adopts a YOLOv4 network model, and a process of processing the picture data by the YOLOv4 network model is as follows:
s031, the Yolov4 network model obtains picture data;
s032, preprocessing picture data and extracting picture features by using CSPDarknet53 through a backbone network;
s033, the auxiliary layer adopts an SPP model to enhance the feature extraction effect;
and S034, detecting the extracted road event characteristics by the detection layer by using PANet, and outputting a road event result.
3. The method of claim 2,
in the YOLOv4 network model, a training result is output through a full connection layer, and the training result comprises a frame regression coordinate, a target classification result and a confidence degree.
4. The method of claim 1,
the computing power sharing cloud platform comprises a computing power sharing node management module, a computing power sharing computing task counting module, a computing power sharing cloud network cooperative processing module and a computing power sharing edge vehicle monitoring module;
the computing power sharing node management module is used for unified management of the computing power sharing nodes, namely for unified management of vehicle ends with available computing power, and provides states of all computing power computing nodes, a CPU (central processing unit) system architecture, an operating system, IP (Internet protocol) addresses and CPU/memory resource use conditions in the computing power sharing node management module;
the computing power sharing computing task counting module counts all computing tasks in the managed computing power sharing cloud platform and provides computing task states, processing time, computing power expenses and processing time delay;
when a computing task needs to be processed by a cross-computing power sharing cloud platform or a road event recognition result needs to be issued by the cross-computing power sharing cloud platform, scheduling through a computing power sharing cloud network cooperative processing module under the cooperation of a mobile core network cloud platform;
the computing power sharing edge vehicle monitoring module provides state information of all edge nodes accessed to the vehicle and vehicle-mounted computing power resource information, wherein the state information comprises vehicle real-time geographic position information, driving direction data, driving speed data and vehicle energy and electric quantity data, and the vehicle-mounted computing power resource information comprises vehicle-end computing resource total quantity, vehicle-end computing resource idle quantity, storage resource data and network bandwidth data.
5. The method of claim 1,
the vehicle end with the available calculation force comprises a vehicle-mounted sensor unit, wherein the vehicle-mounted sensor unit comprises a vehicle sensor, a PTK inertial integrated navigation system, a vehicle-mounted camera and a vehicle-mounted communication unit OBU;
the PTK inertial integrated navigation system consists of an inertial navigation system and a GNSS positioning system, and utilizes a differential positioning technology to realize centimeter-level positioning accuracy of the vehicle, so as to collect positioning information of the vehicle and position road events;
the vehicle-mounted camera is a panoramic camera, a monocular camera or a binocular camera and is used for acquiring pictures related to a road scene;
and the vehicle-mounted communication unit OBU is provided with a 5G communication module and transmits and receives information by using a 5G communication technology.
6. The method of claim 1,
the vehicle-mounted computing power task management module of the vehicle end with available computing power comprises a domain controller distribution computing/storage framework, an idle computing power and idle memory, a vehicle-mounted computing power virtualization and vehicle end computing power scheduling module, a vehicle-mounted computing power resource/state data management module and a vehicle-mounted computing task receiving/returning module;
the domain controller is virtualized and containerized through a domain controller distribution computing/storage architecture technology, so that idle computing power and idle memory of the domain controller are scheduled and used by a domain controller sharing cloud platform, the computing power and the memory use condition of the domain controller are monitored and managed in real time through a vehicle-mounted system, a vehicle-mounted computing task receiving module is used for receiving computing tasks, and a vehicle-mounted computing task returning module is used for returning computing results to a storage sharing service cloud platform.
7. The method of claim 1,
the vehicle end with available calculation force is used for executing calculation tasks by a driving area controller and a cockpit area controller or a central calculator.
8. A road event acquisition system based on intelligent networking automobile computing power sharing is characterized by comprising:
the vehicle end sending module is used for collecting the picture related to the road scene and the vehicle self-positioning information by the vehicle end and sending the picture related to the road scene and the vehicle self-positioning information to the computing power sharing cloud platform by the vehicle end through the vehicle-mounted communication unit OBU;
the cloud computing power scheduling module is used for counting the remaining computing power of the vehicle ends in the peripheral preset range through computing power scheduling of the cloud platform by the computing power sharing cloud platform, establishing a dynamic network link, and sending the pictures related to the road scene to be processed to the vehicle ends with available computing power;
the processing module is used for preprocessing the pictures related to the road scene at the vehicle end with available computing power, dividing the pictures related to the road scene into a training set and a verification set based on a preset proportion, and performing iterative training by using the established target detection network model to obtain a trained optimal network model as an image recognition model so as to determine road events contained in the pictures related to the road scene;
the clustering module is used for clustering multi-frame data of the pictures determined by the image recognition model and determining road events occurring in the time stamps;
and the uploading module is used for uploading the calculated road events to the storage sharing service cloud platform when the calculation of each vehicle end with available calculation power is completed, determining vehicles in the target area according to the road events, and sending the identified road event information based on the state data of the vehicles in the target area.
9. The system of claim 8,
in the processing module, the target detection network model adopts a YOLOv4 network model, and the process of processing the picture data by the YOLOv4 network model is as follows: acquiring picture data by using a YOLOv4 network model; the main network adopts CSPDarknet53 to preprocess the picture data and extract the picture characteristics; the auxiliary layer adopts an SPP model to enhance the characteristic extraction effect; and the detection layer detects the extracted road event characteristics by adopting the PANet and outputs a road event result.
10. The system of claim 9,
in the YOLOv4 network model, a training result is output through a full connection layer, and the training result comprises a frame regression coordinate, a target classification result and a confidence degree.
11. The system of claim 8,
the computing power sharing cloud platform comprises a computing power sharing node management module, a computing power sharing computing task counting module, a computing power sharing cloud network cooperative processing module and a computing power sharing edge vehicle monitoring module;
the computing power sharing node management module is used for unified management of the computing power sharing nodes, namely for unified management of vehicle ends with available computing power, and provides states of all computing power computing nodes, a CPU (central processing unit) architecture, an operating system, IP (Internet protocol) addresses and CPU/memory resource use conditions in the computing power sharing node management module;
the computing power sharing computing task counting module counts all computing tasks in the managed computing power sharing cloud platform and provides computing task states, processing time, computing power expenses and processing time delay;
when a computing task needs to be processed by a cross-computing power sharing cloud platform or a road event recognition result needs to be issued by the cross-computing power sharing cloud platform, scheduling through a computing power sharing cloud network cooperative processing module under the cooperation of a mobile core network cloud platform;
the computing power sharing edge vehicle monitoring module provides state information and vehicle-mounted computing power resource information of all edge nodes accessed to the vehicle, wherein the state information comprises vehicle real-time geographic position information, driving direction data, driving speed data and vehicle energy and electric quantity data, and the vehicle-mounted computing power resource information comprises vehicle-end computing resource total amount, vehicle-end computing resource idle amount, storage resource data and network bandwidth data.
12. The system of claim 8,
the vehicle end with the available computing power comprises a vehicle-mounted sensor unit, wherein the vehicle-mounted sensor unit comprises a vehicle sensor, a PTK inertial integrated navigation system, a vehicle-mounted camera and a vehicle-mounted communication unit OBU;
the PTK inertial integrated navigation system consists of an inertial navigation system and a GNSS positioning system, and utilizes a differential positioning technology to realize centimeter-level positioning accuracy of the vehicle, so as to collect positioning information of the vehicle and position road events;
the vehicle-mounted camera is a panoramic camera, a monocular camera or a binocular camera and is used for acquiring pictures related to a road scene;
and the vehicle-mounted communication unit OBU is provided with a 5G communication module and transmits and receives information by using a 5G communication technology.
13. The system of claim 8,
the vehicle-mounted computing power task management module of the vehicle end with available computing power comprises a domain controller distribution computing/storage framework, an idle computing power and idle memory, a vehicle-mounted computing power virtualization and vehicle-end computing power scheduling module, a vehicle-mounted computing power resource/state data management module and a vehicle-mounted computing task receiving/returning module;
the domain controller is virtualized and containerized through a domain controller distribution computing/storage architecture technology, so that idle computing power and idle memory of the domain controller are scheduled and used by a domain controller sharing cloud platform, the computing power and the memory use condition of the domain controller are monitored and managed in real time through a vehicle-mounted system, a vehicle-mounted computing task receiving module is used for receiving computing tasks, and a vehicle-mounted computing task returning module is used for returning computing results to a storage sharing service cloud platform.
14. The system of claim 8,
the vehicle end with available computing power executes computing tasks and is a driving area controller and a cockpit area controller or a central calculator.
15. An automobile, characterized in that picture data perceived by the automobile in relation to a road scene is processed by a method according to any one of claims 1 to 7 as a road event, so that the automobile is more safely controlled while other automobiles make driving decisions based on the road event.
CN202210906359.XA 2022-07-29 2022-07-29 Road event acquisition method and system based on intelligent network-connected automobile calculation force sharing and automobile Active CN115311869B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210906359.XA CN115311869B (en) 2022-07-29 2022-07-29 Road event acquisition method and system based on intelligent network-connected automobile calculation force sharing and automobile

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210906359.XA CN115311869B (en) 2022-07-29 2022-07-29 Road event acquisition method and system based on intelligent network-connected automobile calculation force sharing and automobile

Publications (2)

Publication Number Publication Date
CN115311869A true CN115311869A (en) 2022-11-08
CN115311869B CN115311869B (en) 2023-07-18

Family

ID=83859472

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210906359.XA Active CN115311869B (en) 2022-07-29 2022-07-29 Road event acquisition method and system based on intelligent network-connected automobile calculation force sharing and automobile

Country Status (1)

Country Link
CN (1) CN115311869B (en)

Citations (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109543617A (en) * 2018-11-23 2019-03-29 于兴虎 The detection method of intelligent vehicle movement traffic information based on YOLO target detection technique
CN109709593A (en) * 2018-12-28 2019-05-03 国汽(北京)智能网联汽车研究院有限公司 Join automobile mounted terminal platform based on " cloud-end " tightly coupled intelligent network
CN109714421A (en) * 2018-12-28 2019-05-03 国汽(北京)智能网联汽车研究院有限公司 Intelligent network based on bus or train route collaboration joins automobilism system
CN110175533A (en) * 2019-05-07 2019-08-27 平安科技（深圳）有限公司 Overpass traffic condition method of real-time, device, terminal and storage medium
AU2020102039A4 (en) * 2020-08-28 2020-10-08 Peng, Yue Miss A high-precision multi-targets visual detection method in automatic driving scene
WO2021005590A1 (en) * 2019-07-05 2021-01-14 Valerann Ltd. Traffic event and road condition identification and classification
CN112286645A (en) * 2020-12-29 2021-01-29 北京泽塔云科技股份有限公司 GPU resource pool scheduling system and method
CN112925657A (en) * 2021-01-18 2021-06-08 国汽智控(北京)科技有限公司 Vehicle road cloud cooperative processing system and method
CN113037786A (en) * 2019-12-09 2021-06-25 中国电信股份有限公司 Intelligent computing power scheduling method, device and system
CN113031035A (en) * 2021-02-07 2021-06-25 北京中交创新投资发展有限公司 Road facility data acquisition system based on artificial intelligence algorithm
CN113744524A (en) * 2021-08-16 2021-12-03 武汉理工大学 Pedestrian intention prediction method and system based on cooperative computing communication between vehicles
CN114265703A (en) * 2022-03-02 2022-04-01 梯度云科技(北京)有限公司 Cross-region computing power scheduling method, system and equipment for cloud server
CN114460923A (en) * 2022-01-28 2022-05-10 重庆长安新能源汽车科技有限公司 Vehicle-mounted distributed computing power system and method and vehicle
CN114613193A (en) * 2022-03-22 2022-06-10 重庆长安汽车股份有限公司 Calculation force sharing-based parking space acquisition method, storage medium, system and vehicle
CN114648870A (en) * 2022-02-11 2022-06-21 行云新能科技（深圳）有限公司 Edge calculation system, edge calculation decision prediction method, and computer-readable storage medium
CN114756340A (en) * 2022-03-17 2022-07-15 中国联合网络通信集团有限公司 Computing power scheduling system, method, device and storage medium

Patent Citations (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109543617A (en) * 2018-11-23 2019-03-29 于兴虎 The detection method of intelligent vehicle movement traffic information based on YOLO target detection technique
CN109709593A (en) * 2018-12-28 2019-05-03 国汽(北京)智能网联汽车研究院有限公司 Join automobile mounted terminal platform based on " cloud-end " tightly coupled intelligent network
CN109714421A (en) * 2018-12-28 2019-05-03 国汽(北京)智能网联汽车研究院有限公司 Intelligent network based on bus or train route collaboration joins automobilism system
CN110175533A (en) * 2019-05-07 2019-08-27 平安科技（深圳）有限公司 Overpass traffic condition method of real-time, device, terminal and storage medium
WO2021005590A1 (en) * 2019-07-05 2021-01-14 Valerann Ltd. Traffic event and road condition identification and classification
CN113037786A (en) * 2019-12-09 2021-06-25 中国电信股份有限公司 Intelligent computing power scheduling method, device and system
AU2020102039A4 (en) * 2020-08-28 2020-10-08 Peng, Yue Miss A high-precision multi-targets visual detection method in automatic driving scene
CN112286645A (en) * 2020-12-29 2021-01-29 北京泽塔云科技股份有限公司 GPU resource pool scheduling system and method
CN112925657A (en) * 2021-01-18 2021-06-08 国汽智控(北京)科技有限公司 Vehicle road cloud cooperative processing system and method
CN113031035A (en) * 2021-02-07 2021-06-25 北京中交创新投资发展有限公司 Road facility data acquisition system based on artificial intelligence algorithm
CN113744524A (en) * 2021-08-16 2021-12-03 武汉理工大学 Pedestrian intention prediction method and system based on cooperative computing communication between vehicles
CN114460923A (en) * 2022-01-28 2022-05-10 重庆长安新能源汽车科技有限公司 Vehicle-mounted distributed computing power system and method and vehicle
CN114648870A (en) * 2022-02-11 2022-06-21 行云新能科技（深圳）有限公司 Edge calculation system, edge calculation decision prediction method, and computer-readable storage medium
CN114265703A (en) * 2022-03-02 2022-04-01 梯度云科技(北京)有限公司 Cross-region computing power scheduling method, system and equipment for cloud server
CN114756340A (en) * 2022-03-17 2022-07-15 中国联合网络通信集团有限公司 Computing power scheduling system, method, device and storage medium
CN114613193A (en) * 2022-03-22 2022-06-10 重庆长安汽车股份有限公司 Calculation force sharing-based parking space acquisition method, storage medium, system and vehicle

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
吴钢;: "新形势下智慧交通管理信息化建设研究及实践", 警察技术, no. 02 *
戴珂泱;周建峰;严风云;: "舟山跨海大桥基于视觉感知的新一代道路事件检测预警系统应用初探", 中国交通信息化, no. 01 *

Also Published As

Publication number Publication date
CN115311869B (en) 2023-07-18

Similar Documents

Publication Publication Date Title
CN111554088B (en) Multifunctional V2X intelligent roadside base station system
US20200309538A1 (en) System for producing and/or updating a digital model of a digital map
JP7420734B2 (en) Data distribution systems, sensor devices and servers
CN112466141A (en) Vehicle-road-collaboration-oriented intelligent network connection end equipment interaction method, system and storage medium
US20220343758A1 (en) Data Transmission Method and Apparatus
WO2021155685A1 (en) Map updating method, apparatus and device
CN112926575A (en) Traffic accident recognition method, device, electronic device and medium
KR20190043396A (en) Method and system for generating and providing road weather information by using image data of roads
CN114613193A (en) Calculation force sharing-based parking space acquisition method, storage medium, system and vehicle
CN115909240A (en) Road congestion detection method based on lane line and vehicle identification
CN111429723B (en) Communication and perception data fusion method based on road side equipment
CN115311869B (en) Road event acquisition method and system based on intelligent network-connected automobile calculation force sharing and automobile
CN109874099A (en) A kind of networking vehicle mounted equipment flow control system
CN111627242B (en) Fusion positioning method for parking spaces in indoor parking garage
CN116798247A (en) Accident data prompting method and related equipment
Liu et al. Real-Time Multi-Task Environmental Perception System for Traffic Safety Empowered by Edge Artificial Intelligence
CN113850111A (en) Road condition identification method and system based on semantic segmentation and neural network technology
WO2020155075A1 (en) Navigation apparatus and method, and related device
CN116959267A (en) Driving crosswind early warning method and system based on machine vision
US11815362B2 (en) Map data generation apparatus
US20210279480A1 (en) Data collection apparatus
CN117612127B (en) Scene generation method and device, storage medium and electronic equipment
CN111881758B (en) Parking management method and system
CN115691141B (en) Urban traffic management method and system based on guardrails and network side server
CN113124752B (en) System and method for positioning automobile based on roadside visual tag

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant