CN112804278A - Traffic road condition car networking system based on image is discerned - Google Patents

Traffic road condition car networking system based on image is discerned Download PDF

Info

Publication number
CN112804278A
CN112804278A CN201911111593.8A CN201911111593A CN112804278A CN 112804278 A CN112804278 A CN 112804278A CN 201911111593 A CN201911111593 A CN 201911111593A CN 112804278 A CN112804278 A CN 112804278A
Authority
CN
China
Prior art keywords
road condition
vehicle
information
condition
unit
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201911111593.8A
Other languages
Chinese (zh)
Inventor
黄玄
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Priority to CN201911111593.8A priority Critical patent/CN112804278A/en
Publication of CN112804278A publication Critical patent/CN112804278A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/01—Detecting movement of traffic to be counted or controlled
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/01—Detecting movement of traffic to be counted or controlled
    • G08G1/017—Detecting movement of traffic to be counted or controlled identifying vehicles
    • G08G1/0175—Detecting movement of traffic to be counted or controlled identifying vehicles by photographing vehicles, e.g. when violating traffic rules

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Traffic Control Systems (AREA)

Abstract

The invention discloses a traffic road condition Internet of vehicles system based on image identification, which is provided with a plurality of vehicle-mounted devices and a rear-end platform; the method mainly applies an edge computing concept to deliver a task of identifying road conditions in driving images to each vehicle-mounted device, each vehicle-mounted device captures the driving images, analyzes the seen traffic road condition information, divides the information into static easily-perceived information, dynamic easily-perceived information and static hardly-perceived information, and submits the information to a rear-end platform through wireless transmission, and the rear-end platform mainly has the functions of confirming the validity of the traffic road condition information by a cross validation mechanism, establishing a traffic road condition map by a dynamic prediction mechanism, and transmitting the traffic road condition information to each vehicle-mounted device according to the position of each driver, and providing each driver for the attention of the driver, thereby forming a low-bandwidth and automatic road condition vehicle networking.

Description

Traffic road condition car networking system based on image is discerned
Technical Field
The invention relates to the field of vehicles, in particular to a traffic road condition Internet of vehicles system based on image identification.
Background
With the popularization of electronic devices for vehicles, more and more electronic devices are used to assist drivers, such as the most common satellite navigation system, to guide drivers to reach an input destination by means of voice and map in cooperation with Global Positioning System (GPS) and map information system. In addition, more and more vehicles are also provided with a plurality of lenses, sensing devices and program control modes, so that the aims of warning the distance between vehicles, automatically stopping vehicles and even automatically driving vehicles are fulfilled. However, traffic conditions are difficult to detect on roads, and therefore, the traditional way of broadcasting by traffic stations is still a source of information for driving to know traffic conditions.
Generally, drivers rely on visual systems to identify their traffic conditions and make appropriate driving decisions. For convenience of analysis, the relationship between traffic information and drivers can be generally classified into three categories: the first category is "easy-to-perceive information", which is road condition information that is within the visual range of the driver and can be used by the driver in making a decision directly on driving, such as the vehicle right in front and the distance to the vehicle in front; the second category is "information that is not easily perceived", which belongs to the traffic information that is within the visible range of the driver but cannot be known by the driver, and is therefore the traffic information that the driver cannot directly use in making a driving decision, such as whether the driver cannot predict that a bus ahead will change lanes or turn right or left; the third category is "imperceptible information", which belongs to traffic information that is not within the visible range of the driver, for example, the traffic lane is reduced by road construction 500 meters ahead of the driver. In addition, various information can be subdivided into dynamic information and static information, that is, whether the location of the road condition changes its position with time, for example, the static information of road construction and accident, and the dynamic information of the position of the running bicycle, the highway street sweeper and the emergency rescue vehicle (for example, ambulance and fire truck).
The operation of the conventional Warning systems, such as a Collision Warning and Full active braking System (CWFAB), an Automatic Collision Avoidance System (ACAS), a Blind Spot Warning System (BSIS), and a Lane Keeping Assist System (LKAS), is a Warning function for preventing a driver from not paying attention to surrounding road conditions through an induction and Automatic control manner. Similarly, the technology disclosed in patent application 105134417 also belongs to an early warning scheme for preventing drivers from being distracted and not paying attention to surrounding road conditions, which is harmful to driving safety. In other words, the warning systems provide the first category of "easy-to-detect information". However, for a cautious driver who is behaving properly and safety-conscious, the "perceptible information" provided by these techniques or devices can enhance the safety of driving by the driver when making his driving decisions relatively little.
In the conventional driving environment, the above-mentioned "imperceptible information" of the second category mostly needs to be provided by other drivers, for example, the driving in front is to warn the driver behind through the direction indicator that the driving dynamics will change soon. The driver to be warned is passive in acquiring the "imperceptible information", and in the case of turning of the vehicle, if the driving behavior of the driver in front is poor, the turn signal is not used correctly, or the turn signal cannot be clearly indicated by the turn signal, the driver in the rear is likely to be in danger. Although the technology disclosed in patent application 104126916 suggests that the driver needs to pay special attention to the driving direction of the driver, it fails to provide clear "imperceptible information".
The third category of "imperceptible information" that is not visible is provided and transmitted by a third party (other drivers, passersby, construction unit, etc.). The information stream is distinguished from participants of the information stream, and can be divided into an information provider and an information transfer party. For example, the foregoing broadcast stations, such as police broadcast stations or traffic broadcast stations, belong to information delivery parties; on the other hand, people who report traffic accidents and road conditions by telephone are the information providers. However, this type of information distribution or notification method mainly has the following disadvantages that one is to use manpower as the information provider, which easily causes delay or error of information notification, such as delay notification and lack of precise position; the other is that the information transmitting side has no initiative of information collection, and the strip type large number of indiscriminate information broadcasting modes do not reduce the efficiency of information transmission and reception.
Moreover, as mentioned above, there are many devices on the market that act as active information transfer parties, filtering all traffic information and only warning the traffic information related to the driver, such as Garmin Connect, Waze, etc. These devices, in conjunction with the technology disclosed in patent application 106121909, can be an effective warning system, but still cannot solve the problem of using manpower as the information provider.
In addition, in the technology disclosed in patent application 104128876, driving images are uploaded in real time and received by the rear vehicle, so that the rear driver can directly see the images seen by the front vehicle driver. However, the image streaming in this solution occupies a large amount of bandwidth, and the device itself is a visual device that is easy to distract the driver, so the risk of traffic accidents caused by distraction is not negligible.
In addition, the technology disclosed in patent application 100146222 is to analyze the driving dynamic sensing data to identify the specific road condition event, and then transmit the data to a background database through wireless transmission for data collection and update, so as to warn the driver about the specific road condition event. However, the disclosed technology cannot provide detailed traffic information, and the main reason is that the driving dynamics analyzed by the technology is the reaction of the driver in front due to some event, such as braking due to reduced vehicle speed in front, lane-switching evasion due to construction on the lane in front, etc., and the analysis of the reaction cannot reversely derive clear traffic information, such as the cause of lane-switching evasion may be road construction or a pedestrian with improper behavior, so the reason for the driving dynamics cannot be determined by analyzing the driving dynamics information of the vehicle, and the driving of the exact traffic information cannot be warned. Furthermore, when dynamic road conditions are encountered, such as dogs or wildlife running into a highway by mistake, the driving response of the driver will change with the dynamic road conditions of mobility, such as the position of the dog, and the type of the recognizable road conditions is limited. In addition, since the same vehicle cannot repeatedly recognize the same event in the disclosure, the disclosure needs to rely on multiple vehicles to repeatedly verify the obtained information, thereby reducing the system operation efficiency, failing to provide sufficient traffic information, and being difficult to promote the development thereof.
With the gradual development of the Internet of Things (Internet of Things), the same concept can also be applied to building a traffic road condition Internet of vehicles, mainly through distributed construction, for example, through a crossing monitor as a traffic road condition information source; however, such a "fixed-source" internet of vehicles with traffic conditions can only obtain equally sparse traffic condition information from a relatively sparse source, and cannot achieve the key principle of "obtaining dense information from a sparse source" to construct an effective and real-time information sharing traffic condition system. The key point of this is a fixed road condition information source, which is dense and extensive no matter how it is built, or it is not as dense information as the source moves through traffic road condition information. The current technologies proposed in the market have not yet met the critical requirement. Therefore, although conceptually attractive, the prior art has not proposed a practical technique to achieve the purpose of effectively connecting to the internet of vehicles.
Disclosure of Invention
In view of the above-mentioned shortcomings of the prior art, the present invention provides an image-recognition-based traffic information networking system, which obtains dense information through movement of the source end of the traffic information, so as to achieve the key principle of "obtaining dense information from a sparse source".
The invention also provides a traffic road condition car networking system based on image identification, which has a car-mounted device and a rear-end platform, wherein the car-mounted device and the rear-end platform mainly apply edge computing (edge computing) concept to deliver the task of traffic road condition identification in the driving image to each car-mounted device, each car-mounted device captures the driving image, analyzes the seen traffic road condition information, divides the information into static easily-perceived information, dynamic easily-perceived information and static hardly-perceived information, and submits the information to the rear-end platform through wireless transmission, the rear-end platform mainly has the functions of confirming the validity of the traffic road condition information by a cross validation mechanism, and then establishes a traffic road condition map by a dynamic prediction mechanism, and transmits the traffic road condition information to each car-mounted device according to the position of each driver, and provides each driver for the attention of the drivers to form a low bandwidth (low bandwidth), Automatic road conditions car networking.
The invention further aims to provide a traffic road condition internet of vehicles system based on image identification, which is provided with a rear-end platform, wherein a cross validation mechanism of the rear-end platform can confirm whether the traffic road condition is confirmed or not by repeatedly receiving events of the same place, and then the established road condition map is referred to determine to newly add, update or remove static traffic road condition information marked on the road condition map; or adjusting the prediction condition of the dynamic traffic road condition information and updating the predicted future position of the dynamic traffic road condition information to perform early warning.
The embodiment of the invention discloses a traffic road condition car networking system based on image identification, comprising: at least one in-vehicle device (in-vehicle device) and a back end platform (back platform), each in-vehicle device is used for capturing a plurality of road condition images, processing the captured road condition images to determine related road condition information, and transmitting the road condition information to the back end platform in an original road condition mode; the rear-end platform integrates the original road condition transmitted from each vehicle-mounted device to form a confirmed road condition, and forwards the integrated confirmed road condition to the relevant vehicle-mounted device according to the position of each vehicle-mounted device; wherein, this vehicle-mounted device still includes: a visual module for capturing a plurality of road condition images and processing the captured road condition images to determine related road condition information; a vehicle condition module (vehicle condition module) for providing a GPS positioning, a real-time timestamp, vehicle speed and other vehicle condition related information, and driving information such as optimal acceleration, deceleration and average acceleration, deceleration and the like obtained according to the vehicle condition information; an alarm module for receiving an alarm event and sending an alarm signal; a device storage module (device storage module) for storing data of each module in the vehicle-mounted device; and a control module (control module) respectively connected to the vision module, the vehicle condition module, the warning module and the storage module, and used for controlling the data transmission between the operation of the vehicle-mounted device and the rear-end platform; the backend platform further includes: a data transmission module for providing required data according to a request from each in-vehicle device; a traffic condition processing module (traffic condition processing module) for receiving and processing each original traffic condition submitted from each vehicle-mounted device, and transmitting the confirmed traffic condition of the processing result to the vehicle-mounted device according to the GPS position of each vehicle-mounted device; and a platform storage module (platform storage module) for storing data of each module of the back-end platform.
In a preferred embodiment, the vision module further comprises: an image capture unit (image capture unit) for capturing a series of multiple continuous images; a real-time image analysis and traffic information identification unit (real-time image analysis and traffic information identification unit) connected to the image capture unit for receiving and analyzing the captured multiple continuous images to identify the traffic information contained in the images; and a traffic condition submitting unit (traffic condition filtering unit) connected to the real-time image analysis and traffic information identification unit for confirming the identified traffic information.
In a preferred embodiment, the vehicle condition module further comprises a GPS unit (GPS unit) for providing the GPS location; a clock unit (clock unit) to provide the real-time timestamp; at least one sensor unit (sensor unit) for providing at least one vehicle condition information; wherein the vehicle condition information at least comprises vehicle speed information; and a vehicle condition analysis unit (vehicle condition analysis unit) for analyzing the vehicle condition information, calculating to obtain the optimal acceleration/deceleration and the average acceleration/deceleration, and storing the driving information into the device storage module.
In a preferred embodiment, the warning module further comprises at least one audio warning unit (audio alarm unit) and/or a visual warning unit (visual alarm unit); the alarm device is used for receiving an alarm event and sending an alarm signal, wherein the alarm signal can be a sound alarm signal and/or an image alarm signal.
In a preferred embodiment, the device storage module stores at least one confirmed road condition map, route and stop data of a fixed route vehicle, and location information of at least one emergency vehicle; and the position information of the emergency vehicle is presented to the vehicle-mounted device through dynamic and imperceptible information.
In a preferred embodiment, the control module further comprises: an event handling unit (event handling unit) connected to the vision module for receiving and processing the data request event and the traffic condition submission event from the vision module; a traffic condition handling unit (traffic condition handling unit) connected to the event handling unit, the vehicle condition module and the device storage module for receiving traffic conditions and determining whether to transmit a warning event to the warning module according to the vehicle conditions; and a data gateway unit (data gateway unit) connected to the event processing unit, the device storage module, and the backend platform for accessing data in response to the data request event.
In a preferred embodiment, the data transmission module further comprises: a data request receiving unit (data request receiving unit) for receiving data requests from the respective in-vehicle devices; a data request handling unit (data request handling unit) connected to the data request receiving unit for handling the data request; and a data relay unit (data relay unit) connected to the data request processing unit for relaying the data to each of the in-vehicle devices.
In a preferred embodiment, the traffic condition processing module further comprises: a traffic condition integration unit (traffic condition association unit) for receiving each traffic condition submitted from each vehicle-mounted device and integrating the submitted traffic condition with each original traffic condition stored in an original traffic condition map of the platform storage module; a traffic condition position prediction unit (traffic condition prediction unit) connected to the traffic condition integration unit and predicting possible positions of the traffic condition at different future time points for each traffic condition; an original traffic map update unit (raw traffic control map update unit) connected to the traffic position prediction unit for updating the predicted future possible position of the traffic to the original traffic map; a traffic condition confidence measure unit (traffic condition confidence measure unit) connected to the original traffic condition map update unit for calculating confidence of each traffic condition; a confirmed traffic map update unit (confirmed traffic control map update unit) connected to the traffic confidence measuring unit for updating the traffic with confidence level higher than a threshold value to a confirmed traffic map; and a traffic condition reporting unit (traffic condition reporting unit) connected to the confirmed traffic condition map updating unit for transmitting the traffic conditions in the confirmed traffic condition map to the vehicle-mounted device according to the GPS position of the vehicle-mounted device.
In a preferred embodiment, the road condition predicted by the road condition location prediction unit includes static road condition and dynamic road condition, which are processed by the road condition location prediction unit and then sent to the original road condition map updating unit.
In a preferred embodiment, the static road condition predicted by the road condition location prediction unit is regarded as a special case of dynamic road condition, and the future predicted locations of the static road condition are all set as an initial location of the road condition.
In a preferred embodiment, the platform storage module stores at least an original road condition map and a confirmed road condition map, which respectively include each original road condition and each confirmed road condition.
In a preferred embodiment, the platform storage module stores at least route and stop data of a fixed route vehicle and at least one history prediction condition of a dynamic road condition, and the history prediction condition of the dynamic road condition is used by the road condition location prediction unit.
In a preferred embodiment, the dynamic road conditions comprise at least a location of the emergency rescue vehicle, a location of the bicycle, and a location of the gravel truck, and the stored historical predicted conditions comprise at least a speed per hour of the emergency rescue vehicle over a road segment at a time.
Through the abundant road condition information of the traffic road condition Internet of vehicles system based on image identification, a driver can make the best driving decision, and the driving safety is further improved. For example, on a common road, a driver can know the road condition information outside the front visible range and can also know the possible dynamic state of a front fixed route vehicle such as a bus, so that the driver can know the dynamic state before the front vehicle is prompted by a direction light, and the driver turns to be active; and the advance reminding of the emergency rescue vehicle can be obtained through a dynamic prediction mechanism of the rear-end platform, and the emergency rescue vehicle can be appropriately avoided to improve social welfare. In addition, on narrow roads or extremely curved roads, under the condition of poor front-view conditions, the driver can also obtain the information meeting the oncoming vehicle by a dynamic prediction mechanism, thereby avoiding improper overtaking decisions.
Drawings
FIG. 1 is a schematic view of an image recognition-based traffic road condition Internet of vehicles system according to the present invention;
fig. 2 is a schematic structural diagram of a vehicle-mounted device of the image recognition-based traffic road condition internet of vehicles system of the present invention;
fig. 3 is a schematic structural diagram of a rear-end platform of the image-recognition-based traffic road condition internet of vehicles system of the present invention;
fig. 4 is a detailed schematic view of a vehicle-mounted device of the traffic road condition internet of vehicles system based on image recognition according to the present invention;
fig. 5 is a schematic structural diagram of a rear-end platform of the image-recognition-based traffic road condition internet of vehicles system of the present invention;
fig. 6 is a schematic view illustrating an actual operation of a plurality of vehicles on a road in the image recognition-based traffic information internet of vehicles system of the present invention; and
fig. 7 is a schematic diagram illustrating how the in-vehicle device determines the second type of static imperceptible information and warns the driver, taking a bus as an example.
Description of reference numerals:
100-a vehicle-mounted device; 110-a vision module; 111-an image capture unit; 112-real-time image analysis and road condition information identification unit; 113-road condition submission unit; 120-vehicle condition module; 121-a GPS unit; 122-a clock unit; 123-a sensor unit; 121-vehicle condition analysis unit; 130-an alert module; 140-device storage module; 150-a control module; 151-event handling unit; 152-road condition processing unit; 153-data gateway unit; 200-a backend platform; 210-a data transmission module; 211-a data request receiving unit; 212-a data request processing unit; 213-data point extracting unit; 220-road condition processing module; 221-road condition integration unit; 222-road condition location prediction unit; 223-original road condition map updating unit; 224-road condition confidence measuring unit; 225-confirmed road condition map updating unit; 226-road condition point-extracting unit; 230-platform storage module; 10-accident road conditions; 20-emergency rescue vehicle; 21. 22-emergency rescue vehicle predicted location; 30. 31, 32, 33, 34-vehicle; 40. 41, 42, 43, 44, 45-road condition information; 50. 51, 52-road condition information.
Detailed Description
The following description is provided for the purpose of illustrating the embodiments of the present invention and is not intended to limit the invention to the particular embodiments disclosed herein. The invention may be embodied or carried out in various other specific embodiments, and various modifications and changes in detail may be made in the present specification without departing from the spirit of the invention.
It should be understood that the structures, ratios, sizes, and the like shown in the drawings and described in the specification are only used for matching with the disclosure of the specification, so as to be understood and read by those skilled in the art, and are not used to limit the conditions of the present invention, so that the present invention has no technical significance.
Refer to fig. 1, 2, 3; FIG. 1 is a schematic view of an image recognition-based traffic road condition Internet of vehicles system according to the present invention; fig. 2 is a schematic structural diagram of a vehicle-mounted device of the image recognition-based traffic road condition internet of vehicles system of the present invention; fig. 3 is a schematic structural diagram of a rear-end platform of the image-recognition-based traffic road condition internet of vehicles system of the present invention. As shown in fig. 1, 2 and 3, the image recognition-based traffic information internet of vehicles system of the present invention comprises: at least one in-vehicle device (100) and a back end platform (200), wherein each of the in-vehicle devices (100) is configured to capture a plurality of road condition images, process the captured road condition images to determine related road condition information, and transmit the road condition information to the back end platform (200) in an original road condition manner; the rear-end platform 200 integrates the original road condition transmitted from each vehicle-mounted device 100 to become a confirmed road condition, and forwards the integrated confirmed road condition to the relevant vehicle-mounted device 100 according to the position of each vehicle-mounted device 100; wherein, this on-vehicle device 100 still includes: a visual module (visual module)110, configured to capture a plurality of road condition images and process the captured road condition images to determine relevant road condition information; a vehicle condition module (vehicle condition module) for providing a GPS positioning, a real-time timestamp, vehicle speed and other vehicle condition related information, and driving information such as optimal acceleration, deceleration and average acceleration, deceleration and the like obtained according to the vehicle condition information; an alert module (alarm module)130 for receiving an alert event and sending an alert signal; a device storage module (device storage module)140 for storing data of each module in the in-vehicle device 100; and a control module (control module)150 respectively connected to the vision module 110, the vehicle condition module 120, the warning module 130, and the storage module 140, and configured to control data transmission between the operation of the vehicle-mounted device 100 and the rear platform 200; the backend platform 200 further comprises: a data transmission module (data transmission module)210 for providing required data according to a request from each of the in-vehicle devices 100; a traffic condition processing module (traffic condition processing module)220, configured to receive and process each original traffic condition submitted from each vehicle-mounted device 100, and transmit the confirmed traffic condition of the processing result to the vehicle-mounted device 100 according to the GPS position of each vehicle-mounted device 100; and a platform storage module (platform storage module)230 for storing data of each module of the backend platform 200.
As described above, in the present invention, traffic information is classified into "easy-to-perceive information" of the first type, "imperceptible information" of the second type, and "imperceptible information" of the third type; wherein, the first type of "easily-perceived information" refers to the self-known traffic information within the visible range of the driver; the second category of "imperceptible information" refers to traffic information that is within the visible range of the driver but is not known by the driver; the third category of "imperceptible information" refers to traffic information outside the visible range of the driver. Furthermore, each type of information can be further subdivided into static and dynamic information depending on whether the location of the information changes with time.
Information such as accidents and road construction within the visual range of a driver belongs to the first category of "easily-perceived information", and the position of occurrence of the information does not change with time, so that the information is "static easily-perceived information"; the dynamic state of the front vehicle in the visible range of the driver belongs to the second category of information which is difficult to be perceived, and the position of the front vehicle changes along with time, so the front vehicle is dynamic information which is difficult to be perceived; the traffic information (dynamic and static) outside the visible range of 500 meters ahead of the driver belongs to the third category of "invisible (dynamic and static)" information. For example, the static perceptible information may include, but is not limited to, accidents, road construction, etc.; "dynamic easy-to-detect information" may include, but is not limited to, the location of a vehicle in motion, such as an ambulance, a fire truck, a street sweeper, a bus, a garbage truck, etc.; the "static imperceptible information" may include the driving dynamics of the bus, such as turning right or left, turning to an inner lane or a stop, and turning around the garbage truck, but is not limited thereto.
Under the classification framework of the traffic information, one of the technical features of the present invention is that each driver can directly obtain the first type of traffic information from the multiple traffic images captured by the vision module 110; the second type of traffic information can be identified and obtained by accessing the data stored in the device storage module 140 or the data stored in the platform storage module 230 through the control module 150 and the data transmission module 210 of the rear platform 200, for example, the route of a bus, a school bus, or a garbage collection vehicle with a fixed route and the related data of a stop. Then, each vehicle-mounted device can submit the first type and the second type of road condition information to the rear-end platform 200, and after the first type and the second type of road condition information are collected by the rear-end platform 200, each driver is prompted in a third type of road condition information mode; moreover, the first and second road conditions are submitted to the back-end platform by the event processing unit, and are directly transmitted to the road condition processing unit of the vehicle-mounted device by the event processing unit, so that the most appropriate time warning is determined. In other words, the third type of traffic information that each driver needs but cannot directly obtain is integrated from the first type and the second type of traffic information of other drivers.
The back-end platform 200 determines the validity of each submitted traffic information in a cross validation manner, and then establishes a traffic map in a dynamic prediction manner, and transmits the traffic information to each vehicle-mounted device 100 according to the position of each driver, so as to provide the driver with reference attention. Specifically, the cross validation mechanism of the backend platform 200 can determine whether the traffic road condition is true in the place by repeatedly receiving the events in the same place, and then refer to the established road condition map to determine to add, update or remove the static traffic road condition information marked on the road condition map; or adjusting the prediction condition of the dynamic traffic road condition information and updating the predicted future position of the dynamic traffic road condition information to perform early warning. In addition, the backend platform 200 also serves as a transmitter of "imperceptible information" in the system of the present invention, which refers to the driving position of the vehicle-mounted device 100 according to the established road condition map, and transmits back "imperceptible information" (dynamic, static) of the third type required by the vehicle-mounted device 100, and the road condition processing unit 152 of the vehicle-mounted device 100 determines the most appropriate warning time. The traffic condition processing unit 152 calculates an optimum warning time according to various traffic condition information and driving information provided by the traffic condition module 120, that is, according to a specific traffic condition and driving environment, and transmits an early warning message to the warning unit to warn the driver. The specific vehicle conditions include, but are not limited to, optimal acceleration/deceleration, average acceleration/deceleration, and the like; the driving environment may include the amount of traffic flow in front of and behind and whether to drive on the uphill road section and the downhill road section, but is not limited thereto. For example, the vehicle condition module 120 can provide a GPS location, a real-time timestamp, vehicle speed, and other vehicle condition related information, and the optimal acceleration/deceleration and average acceleration/deceleration obtained according to the vehicle condition information; the optimal acceleration and deceleration can be obtained by analyzing the vehicle speed and the related sensor information in a continuous time. It is worth to be noted that the driving information can be used for calculating the optimal warning time aiming at a road condition; the optimal warning time may be different for ordinary cars and sand cars on the expressway, and the time required for deceleration or braking may be different from the distance by 5 seconds or more; furthermore, there may be great differences in acceleration and deceleration of the vehicle with the cargo.
As shown in fig. 4, in a preferred embodiment, the vision module 110 further includes: an image capturing unit 111 for capturing a series of multiple continuous images; a real-time image analysis and road condition information identification unit 112 connected to the image capturing unit 111 for receiving and analyzing the captured multiple continuous images to identify the road condition information contained in the images; and a road condition submitting unit 113, connected to the real-time image analysis and road condition information identifying unit 112, for determining the identified road condition information; the image capturing unit 111 may be a camera. Specifically, the real-time image analysis and traffic information recognition unit 112 further identifies the traffic information including the first type of "easy-to-perceive information" and the second type of "difficult-to-perceive information"; for the first category of "easy-to-perceive information", the vehicle-mounted device 100 submits the information to the backend platform 200 in real time through the control module 150, and for the second category of "difficult-to-perceive information", such as static information of fixed-route vehicles, the vehicle-mounted device acquires the required related data through the control module 150 and then submits the data to the backend platform 200. In addition, the first and second road conditions are directly transmitted to the road condition processing unit 152 to determine the most appropriate warning.
Specifically, when the real-time image analysis and traffic information identification unit 112 detects the second type of traffic, it notifies the traffic submission unit 113, and the traffic submission unit 113 acquires the required data to complete the construction of the second type of traffic information. Furthermore, if no traffic condition is detected, the traffic condition submitting unit 113 can submit a timing notification to indicate that no traffic condition is detected.
Similarly, the vehicle condition module 120 further includes a GPS unit 121 for providing the GPS positioning; a clock unit 122 for providing the real-time timestamp; at least one sensor unit 123 for providing at least one vehicle condition information; wherein the vehicle condition information at least comprises vehicle speed information; and a vehicle condition analysis unit 124, which can analyze the vehicle condition information and calculate to obtain the driving information such as the optimal acceleration and deceleration and the average acceleration and deceleration. The warning module 130 further comprises at least one sound warning unit, an image warning unit, or any combination thereof; the alarm device is used for receiving an alarm event and sending an alarm signal, wherein the alarm signal can be a sound alarm signal, an image alarm signal or any combination thereof. Moreover, the device storage module 140 stores at least one confirmed road condition map, route and stop data of a fixed route vehicle, and location information of at least one emergency vehicle; the position information of the emergency vehicle is presented to the vehicle-mounted device through dynamic and imperceptible information; the confirmed road condition map includes the related road conditions of the first type of "easy-to-perceive information" and the second type of "difficult-to-perceive information" recognized by the vision module 110, and the related road conditions of the third type of "imperceptible information" transmitted from the rear-end platform 200, in other words, the road conditions submitted by other drivers, and the confirmed road condition map after cross validation by the rear-end platform 200; furthermore, the confirmed road condition map, the route and stop data of the fixed route vehicle, and the position information of at least one emergency vehicle; and the location information of the emergency vehicle is dynamic, imperceptible information presented to the vehicle-mounted device, etc. which is related to the data required by the vision module 110 when identifying the second type of road condition or the control module 150 when calculating the appropriate warning time. It is noted that if the route and parking data of the fixed-route vehicle is not stored in the device storage module 140, it can be retrieved from the platform storage module 230 of the backend platform 200 through the control module 150.
Similarly, in a preferred embodiment, the control module 150 further comprises: an event processing unit 151 connected to the vision module 110 for receiving and processing the data request event and the traffic condition submission event from the vision module 110; a traffic processing unit 152 connected to the event processing unit 151, the vehicle condition module 120, the warning module 130 and the device storage module 140 for receiving traffic, determining an optimal warning time according to the driving information and the vehicle condition stored in the device storage module 140, and storing the traffic which does not reach the warning time in the device storage module 140; and a data gateway unit 153, connected to the event processing unit 151, the device storage module 140, and performing data access with the backend platform 200 to respond data to the data request event. It should be noted that the data gateway unit 153 can obtain the required data from the device storage module 140 or from the backend platform 200 when the traffic information submission unit 113 constructs the second type of traffic information, and then transmit the data to the traffic information submission unit 113.
To sum up, the vehicle-mounted device 100 of the present invention recognizes the first type included in the multiple continuous images captured by the image capturing unit 111 through the real-time image analysis of the vision module 110 and the traffic information recognition unit 112, and prompts the traffic submission unit 113 that the second type of traffic needs to be constructed, and the traffic submission unit 113 obtains information through the control module 150 to complete construction of the second type of traffic information; finally, the traffic submitting unit 113 submits the identified first and second traffic categories, and the event processing unit 151 of the control module 150 transmits the submitted first and second traffic categories to the backend platform 200. On the other hand, after the third type of road condition transmitted from the back-end platform 200 is transmitted to the control module 150, the road condition processing unit 152 and other road conditions originally stored in the device storage module 140 are processed together according to the vehicle condition information provided by the vehicle condition module 120, so as to determine an appropriate warning time, and the warning module 130 is used to remind the driver of the road condition that should be noticed.
In other words, the traffic processing unit 152 is responsible for processing the first type and the second type of traffic recognized by the traffic submission unit 113, the third type of traffic forwarded from the backend platform 200, and the traffic processing unit 152 determines that the traffic that has not reached the appropriate warning time is temporarily stored in the device storage module 140.
Similarly, as shown in fig. 5, in a preferred embodiment, the data transmission module 210 further includes: a data request receiving unit 211 for receiving data requests from the respective in-vehicle apparatuses; a data request processing unit 212, connected to the data request receiving unit, for processing the data request; and a data point-adding unit 213 connected to the data request processing unit for adding the data point to each of the in-vehicle devices. Specifically, the data transmission module 210 receives a data request event, which is static-state-related data that requires, for example, a route of a fixed-route vehicle and a stop or the like, from the data gateway unit 153 of the control module 150 in each of the in-vehicle apparatuses 100. More specifically, when the traffic information submitting unit 113 needs to construct the second type of traffic information, it first sends a data request to the event processing unit 151, and the event processing unit 151 forwards the data request to the data gateway unit 153; the data gateway unit 153 first checks whether the required data is already stored in the storage module 140, and if so, returns the data directly, otherwise, sends a data request to the backend platform 200, and stores the data retrieved from the backend platform 200 in the device storage module 140.
Further, the traffic condition processing module 220 further comprises: a road condition integration unit 221, a road condition location prediction unit 222, an original road condition map updating unit 223, a road condition confidence measurement unit 224, a confirmed road condition map updating unit 225, and a road condition point-extracting unit 226; the cross-validation mechanism of the backend platform 200 starts with the road condition integration unit 221 and ends with the confirmed road condition map updating unit 225.
The road condition integration unit 221 is configured to receive each road condition submitted from each vehicle-mounted device 100, compare the received road condition with each original road condition in an original road condition map stored in the platform storage module 230, which is an identification action, and output the identified road condition or a new road condition to the road condition location prediction unit 222. Since a traffic condition may be submitted by different vehicle-mounted devices at different time points, the unit has a main function of integrating the same traffic condition submitted by the different reports. It should be noted that, at this time, no confirmation of any road condition is made, i.e., the confidence level calculation is performed.
The traffic location prediction unit 222 is connected to the traffic integration unit 221 and predicts possible locations of the traffic at different future time points for each traffic, where the possible locations are the current locations of the traffic if the traffic is static; the original map update unit 223 is connected to the road condition position prediction unit 222 to update the predicted future possible position of the dynamic road condition to the original map.
The traffic confidence measuring unit 224, connected to the original traffic map updating unit 223, calculates a confidence of each traffic condition, and determines whether the traffic condition actually exists based on the confidence; the confirmed road condition map updating unit 225 is connected to the road condition confidence measuring unit 224 to update the road condition with confidence level higher than a threshold to a confirmed road condition map, in other words, add new road condition, update existing road condition, or remove solved road condition; and the road condition promoting unit 226 is connected to the confirmed road condition map updating unit 225 and the platform storage module 230, so as to transmit each road condition in the confirmed road condition map to the vehicle-mounted device 100 according to the GPS position of each vehicle-mounted device 100.
It should be noted that the traffic information integrating unit 221 and the traffic confidence measuring unit 224 are the aforementioned units for performing validation of each submitted traffic information in a cross-validation manner; the traffic integration unit 221 can recognize and repeatedly receive events at the same location, determine whether the location has the traffic road condition via the traffic confidence measuring unit 224, and then refer to the established confirmed road condition map to determine to add, update or release the traffic road condition information marked on the confirmed road condition map. The road condition location prediction unit 222 adjusts the prediction conditions of the traffic road condition information by referring to the relevant data stored in the platform storage module 230 in a dynamic prediction manner, and updates the predicted future location of the traffic road condition information to perform an early warning.
The traffic confidence measuring unit 224 calculates a confidence of each traffic condition; the confidence level is a confidence level indicating whether the road condition exists; since many road conditions are excluded over time, for example, at the scene of a car accident, or at the road renovation to close a diversion, etc. Therefore, each submitted road condition in the invention is endowed with a reporting time and a releasing time; when the submitted road condition is before the release time, it can be regarded as an applicable road condition, for example, five drivers submit the same traffic accident road condition at different times respectively. And calculating the road conditions submitted at the five different times and the conditions of the release time of the road conditions, thereby obtaining the confidence level of the continuous existence of the road conditions of the traffic accident. When the confidence of the road condition is higher than a threshold, updating to a confirmed road condition map by the confirmed road condition map updating unit 225; and finally, the road condition providing unit 226 transmits the road conditions in the confirmed road condition map to the vehicle-mounted device 100 according to the GPS position of each vehicle-mounted device 100.
To sum up, the whole cross-validation mechanism of the backend platform starts with the road condition integration unit 221 and ends with the confirmed road condition map updating unit 225. The road condition integration unit 221 compares the received road condition with the existing road condition, which is an identification action, and outputs an existing road condition or a new road condition as an input of the road condition location prediction unit 222. The traffic location prediction unit 222 predicts a possible future location of the input traffic (the possible future location is the current location in case of a static traffic) and requests the original traffic map update unit 223 to update the original traffic map. After the original road condition map updating unit 223 updates the original road condition map, the road condition confidence measuring unit 224 determines whether the road condition exists, and requests the confirmed road condition map updating unit 225 to update the confirmed road condition map.
It should be noted that the calculation of the road condition confidence level can be implemented in different ways, and the following description is only a preferred embodiment, but not limited thereto.
Let a traffic condition report (report) be represented by a binary (T, χ), where T represents the time (detection time) when the traffic condition is detected, i.e., the time when the vehicle-mounted device detects and reports the traffic condition, it is noted that the traffic condition may occur earlier; the design is mainly aimed at enabling the vehicle-mounted device to report back when detecting the no-traffic condition, so as to serve as a reference basis for the subsequent cross comparison with other related traffic condition reports; that is, the driving information can be reported at a fixed time. In other words, the timing notification of the driving information means that the vehicle-mounted device can submit a special first-type traffic information when there is no traffic, which means that no traffic is detected. The timing reporting traffic information can be used for subsequent cross validation and integration of various road condition information and calculation of confidence. The resolved traffic road condition can be removed by confidence calculation including a timing announcement traffic information representing no road condition.
Furthermore, a traffic condition record (record) can be represented by a triple (T, χ, Γ), where Γ represents the expiration time (expiry time) of the traffic condition record,
Figure BDA0002272876280000151
Figure BDA0002272876280000152
indicating a validity period (valid duration). In other words, for any point in time T0If T is0<Γ, the traffic condition triplet (T, χ, Γ) is a valid traffic condition record. It should be noted that the expiration of the traffic condition record does not necessarily indicate that the traffic condition is over.
When each vehicle-mounted device detects each traffic condition, the vehicle-mounted device reports the traffic condition and generates a corresponding traffic condition record by the back-end platform.
Taking the traffic condition of a traffic accident as an example, suppose that at most M effective traffic condition records are used each time the confidence is calculated,
Figure BDA0002272876280000153
system of representationsAnd assigning a preset valid period of the road condition of each traffic accident. For time point T, let N be the total number of traffic condition records, M ═ min { M, N }, where N is the number of valid traffic condition records (N ≦ N), and let (T)1,χ1,Γ1)、(T2,χ2,Γ2)…(Tm,χm,Γm) Representing the latest m traffic condition records.
Accordingly, the confidence conf of the traffic accident road condition at the time point T can be defined as:
when the N is greater than 1, the reaction solution,
Figure BDA0002272876280000161
otherwise, conf is 0.1;
wherein when
Figure BDA0002272876280000162
When the temperature of the water is higher than the set temperature,
Figure BDA0002272876280000163
in other words, the confidence level of each traffic condition record is related to the time remaining before the expiration time of the traffic condition record. The confidence weighted average of all the valid traffic condition records included in the calculation is the confidence of the traffic condition. Of course, this is not a limitation; other functions of confidence may be used without affecting the scope of the invention.
For example, specifically, in the above example, when M is 5,
Figure BDA0002272876280000164
In time, assume that all traffic conditions associated with the traffic accident are recorded as follows:
Figure BDA0002272876280000165
when T is 14:01, N is 1; conf is 0.1;
when T is 14:04, N is 3;
Figure BDA0002272876280000166
when T is 14:34, N is 5, N is m is 2;
Figure BDA0002272876280000167
in a preferred embodiment, the platform storage module 230 at least stores an original road condition map and a confirmed road condition map, which respectively include each original road condition and each confirmed road condition; the platform storage module 230 stores at least route and stop data of a fixed route vehicle and historical predicted conditions of dynamic road conditions. The historical prediction condition of the dynamic road condition refers to a rule of dynamic road condition prediction obtained according to the traffic road conditions; for example, an emergency vehicle is detected to travel at a speed of 40KM/H in a certain road section, and the dynamic road condition is predicted based on the detected speed; after a period of time, the emergency vehicle is detected to be faster than predicted (e.g., 45KM/H), and thus, new prediction rules may be generated based on the speed, e.g., 42.5 KM/H. The traffic location prediction unit 222 predicts the future location of the traffic according to the historical prediction conditions of the dynamic traffic.
To sum up, the back-end platform 200 mainly maintains the original road condition map and the confirmed road condition map, continuously updates the road conditions submitted by the vehicle-mounted devices 100 in an integrating, dynamic predicting and road condition confidence measuring manner, and returns the updated related road condition information to the vehicle-mounted devices 100.
Fig. 6 is a schematic view of the image recognition-based traffic road condition internet of vehicles actually operating in a plurality of vehicles on a road according to the image recognition-based traffic road condition internet of vehicles system of the invention. The traffic information of fig. 6 includes the emergency rescue vehicle 20 in driving (dynamic traffic information), the accident site 10 (static traffic information), and the vehicles 30, 31, 32, 33, and 34 in driving, wherein the vehicles 30, 31, 32, 33, and 34 are all equipped with the vehicle-mounted device of the present invention.
As shown in fig. 6, where one of the events is that the vehicle 30 is traveling behind the emergency rescue vehicle 20 and, as a result of traveling following the rear thereof for a while, the on-board devices of the vehicle 30 repeatedly recognize that there is a traveling emergency rescue vehicle 20 ahead. The vehicle-mounted device combines the continuously identified dynamic and easily perceived traffic information (40, 41, 42) with the relevant positioning information and uploads the combined information to the back-end platform 200. After receiving the continuously transmitted road condition information from the vehicle 30 through the back-end platform 200, the newly added dynamic road condition information is confirmed through a cross validation mechanism, and an initial prediction condition is generated according to the movement prediction condition of the past emergency vehicle and the driving positioning information of the vehicle 30, so as to predict the future position of the dynamic road condition information (the emergency rescue vehicle 20), where the positions 21 and 22 shown in fig. 6 are the predicted positions of the emergency rescue vehicle 20 after 2 seconds and 7 seconds, respectively.
In addition, according to fig. 6, when the vehicle 31 reports the driving information at regular time, the back-end platform 200 reveals that the vehicle 31 will meet the emergency rescue vehicle 20 after 2 seconds according to the prediction result, that is, the dynamic traffic information (i.e., the traffic information of the third type of dynamic imperceptible information for the vehicle 31) is forwarded back to the on-board device of the vehicle 31, and the on-board device of the vehicle 31 determines the best warning time according to the analyzed own vehicle condition information and the driving information, so as to warn the driver.
Furthermore, the other event is that the vehicles 32 and 33 are traveling in the same direction on the same road, the vehicle-mounted device of the vehicle 33 recognizes the accident 10, and transmits the static easily-perceived traffic information 43 back to the back-end platform 200. At the same time, the onboard devices of the vehicles 34 in the opposite lane of the same roadway also detect the same accident 10 and return this information 44 to the back end platform 200.
Then, the backend platform 200 receives the events 43 and 44 from the vehicles 33 and 34, and confirms the addition of the static traffic information to the road map by using the cross validation mechanism. The traffic road condition information 50, 51, 52 is reported back to the vehicle-mounted devices of the vehicles 31, 32, 34 respectively (i.e. the traffic road condition is the static imperceptible information of the third type for the vehicle 32), and the vehicle-mounted device of the vehicle 32 determines the best warning time according to the collected vehicle condition information, so as to warn the driver. The traffic road condition information 50, 51, 52 are respectively: there is an emergency rescue vehicle 20 in front after 2 seconds, an emergency rescue vehicle 20 in front after 7 seconds, a traffic accident (road conditions confirmed by 43 and 44) at 300 meters in front, and an emergency rescue vehicle 20 in front after 7 seconds.
One of the events described in fig. 6 is that the onboard device of the vehicle 35 recognizes the emergency rescue vehicle 20 traveling in the opposite lane, and uploads the dynamic easily-perceived traffic information 45 to the backend platform 200, which verifies and updates the predicted conditions and possible future positions of the dynamic traffic information (dynamic state of the emergency rescue vehicle 20) by using a cross-validation mechanism.
The application scenario of FIG. 6 illustrates how each vehicle-mounted device handles "dynamic and static perceptible information" and how the backend platform will determine "dynamic and static imperceptible information" with respect to each vehicle-mounted device. Fig. 7 is an example of a bus, which briefly describes how the in-vehicle device determines the static imperceptible information of the second type and warns the driver. As shown in fig. 7, when the vehicle-mounted device recognizes that there is a bus in motion in front of the vehicle, the vehicle attempts to use its route plate or license plate, and refers to the current driving position and the related map data to recognize the driving route. Then obtaining the information of the route so as to warn the driver of possible movement; for example, a 130 m ahead would turn right to the XX road, a 50 m ahead would have a stop board for the bus on that route, a cut ahead to an interior line to a bus lane, etc.
It should be noted that, in the present invention, each vehicle-mounted device firstly identifies objects such as lanes, vehicles, pedestrians, accidents, road construction, etc. for a single captured image, and then identifies road condition information by combining the identification results of multiple images and related vehicle condition information such as the current vehicle speed, and calculates the confidence level for each road condition. The following description will explain the calculation of confidence level, determination of traffic road condition establishment, and dynamic and static determination by taking the detection of the ambulance as an example.
Assuming that the image capturing unit 111 of the in-vehicle device 100 can capture 60 images per second, in order to improve the accuracy, the road condition information seen by each 20 images can be determined. Firstly, the real-time image analysis and road condition information identification unit 112 of the vehicle-mounted device identifies the ambulance by aiming at a single image, and if the ambulance is detected in 10 images in 20 obtained results, the confidence level is 50%; if the confidence level is greater than the threshold value determined according to the vehicle speed, the real-time image analysis and road condition information identification unit 112 determines that the traffic road condition information of the ambulance really appears within the range; in addition, the real-time image analysis and traffic information recognition unit 112 also determines whether the ambulance is moving or not by analyzing the position of the ambulance in 10 consecutive images with reference to the current speed, and further determines whether the ambulance is dynamic information or static information.
After the real-time image analysis and the dynamic and static road condition information determined by the road condition information identification unit 112 are established, it is determined whether the front vehicle has relevant "imperceptible information" according to the road condition map stored in the vehicle-mounted device. Taking a bus as an example, once it is determined that there is a traveling bus in front of the bus, the real-time image analysis and road condition information recognition unit 112 prompts the road condition submission unit 113 to indicate the second type of road condition, and then the road condition submission unit 113 determines the bus route according to the current driving position, the map information, the identified information such as the route board and the license plate of the bus, and obtains the detailed driving route, thereby obtaining the "static imperceptible information". It is noted that this example can be applied to any vehicle whose travel route is disclosed in various ways.
The static traffic road condition information identified by the vehicle-mounted device at least comprises accidents, vehicle failure and anchorage, road construction, lane foreign matters and the like; in addition, the dynamic traffic road condition information at least comprises the quantity, types, behaviors or pedestrians with inconvenient actions of the same-direction and opposite-direction moving vehicles within the viewed range; the vehicle types at least comprise emergency rescue vehicles, such as fire trucks, ambulances and the like, bicycles, locomotives, large heavy locomotives, passenger cars, trucks, sand trucks, buses, tourist coaches and garbage trucks. In addition, when the vehicle-mounted device communicates with the rear-end platform, the vehicle-mounted device can transmit positioning information thereof, and can also comprise GPS information, altimeter information, lane information and the like.
On the other hand, the traffic information is divided into "dynamic" and "static" for the backend platform. When the static traffic road condition information from the vehicle-mounted device is received, the rear-end platform firstly checks whether the same road condition exists in the same or similar position on the road condition map. If not, giving a preset relief time and a preset confidence level to the road condition map, and marking the road condition map with the preset relief time and the preset confidence level; if yes, updating the preset release time and improving the confidence level of the event. When receiving the dynamic road condition information from the vehicle-mounted device, the rear-end platform firstly searches whether the same traffic road condition predicted position mark exists at the similar position on the road condition map. If not, determining an initial prediction condition, predicting a possible position within a fixed time according to the condition, and marking the possible position on a road condition map by a default confidence level; and otherwise, performing cross validation with the past predicted position to update the predicted condition, marking the new predicted position on the road condition map, and improving the confidence level of the event.
On the other hand, if the rear-end platform receives the timing report which is transmitted by the vehicle-mounted device and has no traffic road condition, whether the marked dynamic and static traffic road condition information exists on the road condition map or not is checked. If yes, the confidence level is reduced, and when the confidence level is reduced to a threshold value, the event is deleted from the road condition map.
In the dynamic traffic road condition prediction mechanism provided by the invention, a rear-end platform refers to the driving positioning information of the returned vehicle and historical prediction records of similar events in the past to generate an initial prediction condition, and possible positions of different time points in the future are estimated according to the initial prediction condition; then, when receiving the same dynamic road condition reported from other vehicle-mounted devices, the actual speed of the dynamic road condition can be obtained by analyzing the previous reported position, the current reported position and the relative time of the previous reported position and the current reported position, and the vehicle speed of the simulation condition can be corrected or adjusted up according to the actual speed so as to generate a new simulation result. It should be noted that the actual speed of the dynamic road condition can be obtained in different ways, and the preferred embodiment is only described herein, but not limited thereto. After receiving the information reported by the vehicle-mounted device, the rear-end platform refers to the road condition map established by the vehicle-mounted device and returns information which has confidence degree higher than a threshold value and is 'imperceptible' to the device according to the driving position of the vehicle-mounted device.
In addition, after the vehicle-mounted device acquires the confirmed road condition needing to be warned, the optimal warning time needs to be determined. However, it is determined that the information, such as the response time of the driving process information, the current vehicle speed, the current lane, the surrounding traffic conditions, the vehicle conditions, and the road conditions need to be taken into consideration. In a preferred embodiment, the road condition location information may further include a real location and a location to be processed, since the driver is less able to cope with the emergency when the driver is bent; when a road condition event occurs in a curve, the event can be regarded as occurring at a bend (namely, a position to be processed), and in this case, the real position is different from the position to be processed; on the other hand, when the true position is on the straight line, the two positions can be set to be the same.
The vehicle condition information to be considered includes at least an optimal acceleration/deceleration and an average acceleration/deceleration, and the information collection can be obtained from GPS positioning information, or accelerometer, or OBD information, sensor, or data of a combination thereof. Assuming that the vehicle-mounted device uses sound as a warning transmission medium, the road condition information to be warned is: when driving on a straight road, the real position is the same as the position to be processed, the current vehicle is driving on the inner lane at 100KM/H, the number of surrounding vehicles is small, 8 seconds are required for the vehicle to decelerate to 50KM/H, and the driver needs 2 seconds to process information. At this time, the in-vehicle device determines that it is the best time to warn driving before 8+2+ bias (bias is a fixed value) seconds near the accident site. However, if there are many surrounding vehicles, the deceleration parameter is modified to the time required for the vehicle to decelerate to 0KM/H, and the optimal time for warning driving is modified and calculated accordingly. In addition, if the vehicle is not driving on the inner lane, especially in response to the lane change dynamics of the vehicle driving on the inner lane, the optimal time for warning driving still needs to be calculated with the same parameters, but it needs to be informed that the event occurs on the inner lane, and the driving makes a decision to reduce the speed by itself, so as to avoid affecting the driving efficiency. After the vehicle-mounted device calculates the optimal early warning time according to the program, if the time is less than a threshold value, such as 5 seconds, the vehicle-mounted device immediately warns a driver; otherwise, the driver is warned at the calculated optimal early warning time.
Furthermore, the image recognition-based traffic information internet of things system of the present invention may further comprise at least one third party traffic data interface unit for receiving traffic information from a third party and other traffic information disclosed on a public transportation integrated information distribution service Platform (PTX). Taking the dynamic road condition of the bus as an example, because the real-time position of each bus is disclosed in the public transportation integrated information circulation service platform, the real-time position of each bus is obtained through the third-party data interface unit, the dynamic road condition information of the bus and the static road condition information of the bus about to turn left or right, change lanes and the like can be established by referring to the bus route information stored in the platform storage module, and then the information is submitted to the road condition processing unit for subsequent processing so as to prompt each driver. Furthermore, the third-party road condition data interface unit can be arranged on the rear-end platform or the vehicle-mounted device.
In summary, through the abundant traffic information of the image recognition-based traffic information networking system of the present invention, the driver can make the best driving decision, thereby enhancing driving safety. For example, on a common road, a driver can know the road condition information outside the front visible range and can also know the possible dynamic state of a front fixed route vehicle such as a bus, so that the driver can know the dynamic state before the front vehicle is prompted by a direction light, and the driver turns to be active; and the advance reminding of the emergency rescue vehicle can be obtained through a dynamic prediction mechanism of the rear-end platform, and the emergency rescue vehicle can be appropriately avoided to improve social welfare. In addition, on narrow roads or extremely curved roads, under the condition of poor front-view conditions, the driver can also obtain the information meeting the oncoming vehicle by a dynamic prediction mechanism, thereby avoiding improper overtaking decisions.
However, the above embodiments are only for illustrative purposes and are not intended to limit the present invention, and those skilled in the art can modify and change the above embodiments without departing from the spirit and scope of the present invention. In addition, the number of components in the above embodiments is only illustrative and not intended to limit the present invention. Therefore, the scope of the present invention should be determined by the following claims.

Claims (19)

1. The utility model provides a traffic road conditions car networking system based on image is discerned which characterized in that contains: at least one vehicle-mounted device and a rear-end platform;
each vehicle-mounted device is used for capturing a plurality of road condition images, processing the captured road condition images to determine related road condition information, and transmitting the road condition information to the rear-end platform in an original road condition mode; the rear-end platform integrates the original road condition transmitted from each vehicle-mounted device to form a confirmed road condition, and forwards the integrated confirmed road condition to the relevant vehicle-mounted device according to the position of each vehicle-mounted device;
wherein the vehicle-mounted device further includes: the visual module is used for capturing a plurality of road condition images and processing the captured road condition images to determine related road condition information; a vehicle condition module for providing a GPS positioning, a real-time timestamp, vehicle speed and/or other vehicle condition related information, and driving information such as optimal acceleration and deceleration and average acceleration and deceleration obtained according to a plurality of vehicle condition information; the warning module is used for receiving a warning event and sending a warning signal; the device storage module is used for storing data of each module in the vehicle-mounted device; the control module is respectively connected to the vision module, the vehicle condition module, the warning module and the device storage module and is used for controlling the data transmission between the operation of the vehicle-mounted device and the rear-end platform;
the backend platform further includes: the data transmission module is used for providing required data according to the requests from the vehicle-mounted devices; a road condition processing module for receiving and processing original road conditions submitted by the vehicle-mounted devices, and transmitting the confirmed road conditions of the processing result to the vehicle-mounted devices according to the GPS positions of the vehicle-mounted devices; and a platform storage module for storing data of each module of the back-end platform.
2. The image recognition-based traffic road condition internet of vehicles system of claim 1, wherein the vision module further comprises: an image capturing unit for capturing a series of multiple continuous images; the real-time image analysis and road condition information identification unit is connected with the image acquisition unit and used for receiving and analyzing the acquired multiple continuous images so as to identify the road condition information contained in the images; and a road condition submitting unit connected to the real-time image analysis and road condition information identification unit and used for confirming the identified road condition information.
3. The system of claim 1, wherein the vehicle condition module further comprises a GPS unit for providing the GPS location; a clock unit to provide the real-time timestamp; at least one sensor unit for providing at least one vehicle condition information; wherein the vehicle condition information at least comprises vehicle speed information; and the vehicle condition analysis unit can analyze a plurality of pieces of vehicle condition information, calculate to obtain the running information such as the optimal acceleration, the optimal deceleration, the average acceleration, the average deceleration and the like, and store a plurality of pieces of running information into the device storage module.
4. The system according to claim 1, wherein the warning module further comprises at least one audio warning unit and/or an image warning unit; the alarm device is used for receiving an alarm event and sending an alarm signal, wherein the alarm signal can be a sound alarm signal and/or an image alarm signal.
5. The image recognition-based traffic information vehicular networking system of claim 1, wherein the device storage module stores at least one confirmed road information map, route and stop data of a fixed route vehicle, and location information of at least one emergency vehicle; and the position information of the emergency vehicle is presented to the vehicle-mounted device through dynamic and imperceptible information.
6. The image recognition-based traffic road condition internet of vehicles system of claim 1, wherein the control module further comprises: the event processing unit is connected with the visual module and used for receiving and processing the data request event and the road condition submission event from the visual module; a road condition processing unit connected to the event processing unit, the vehicle condition module, the warning module and the device storage module for receiving road conditions and determining whether to transmit a warning event to the warning module according to the vehicle conditions; and the data gateway unit is connected with the event processing unit, the device storage module and the back-end platform to perform data access so as to respond data to the data request event.
7. The image recognition-based traffic road condition internet of vehicles system of claim 1, wherein the data transmission module further comprises: a data request receiving unit for receiving data requests from the respective vehicle-mounted devices; the data request processing unit is connected with the data request receiving unit and used for processing the data request; and the data point-lifting unit is connected with the data request processing unit and is used for lifting the data points to each vehicle-mounted device.
8. The image recognition-based traffic road condition internet of vehicles system of claim 1, wherein the road condition processing module further comprises:
a road condition integration unit for receiving each road condition submitted by each vehicle-mounted device and integrating the submitted road condition with each original road condition in an original road condition map stored in the platform storage module;
a road condition position prediction unit connected to the road condition integration unit and predicting possible positions of the road condition at different future time points for each road condition;
an original road condition map updating unit connected to the road condition position predicting unit for updating the predicted possible future position of the dynamic road condition to the original road condition map; a road condition confidence measuring unit connected to the original road condition map updating unit to calculate the confidence of each road condition;
the confirmed road condition map updating unit is connected with the road condition confidence measuring unit so as to update the road condition with the confidence higher than a threshold value to a confirmed road condition map; and
and the road condition point-lifting unit is connected with the confirmed road condition map updating unit so as to transmit the road conditions in the confirmed road condition map to the vehicle-mounted device according to the GPS position of each vehicle-mounted device.
9. The system of claim 8, wherein the road conditions predicted by the road condition location prediction unit include static road conditions and dynamic road conditions, which are processed by the road condition location prediction unit and then sent to the original road condition map updating unit.
10. The system of claim 9, wherein the static road conditions predicted by the road condition location prediction unit are considered as special cases of dynamic road conditions, and the future predicted locations of the static road conditions are set as initial locations of the road conditions.
11. The system of claim 8, wherein the original road map and the confirmed road map are stored in the platform storage module and respectively comprise original road conditions and confirmed road conditions.
12. The system of claim 9, wherein the platform storage module stores at least route and stop data of a fixed route vehicle and at least one historical prediction condition of dynamic road conditions, and the historical prediction condition of dynamic road conditions is used by the road condition location prediction unit.
13. A traffic information network-based system as claimed in claim 12, wherein the dynamic traffic information comprises at least a location of emergency rescue vehicles, a location of bicycles, and a location of sand vehicles, and the historical prediction conditions stored therein comprise at least a speed per hour of the emergency rescue vehicles at a certain section of time.
14. The system of claim 2, wherein when the vehicle-mounted device recognizes that there is a vehicle with a fixed route in front, i.e. a route sign or a license plate thereof, and refers to the current driving position and related map data to recognize the driving route, then obtains the information of the route to warn the driver of possible movement; once the fixed route vehicle in driving is determined in front, the real-time image analysis and road condition information identification unit prompts the fixed route vehicle in driving in front of the road condition submission unit, and then the bus route is judged and the detailed driving route is obtained by the road condition submission unit according to the current driving position, the current driving information, the information of the route board, the license plate and the like of the bus, and warning is prompted according to the information.
15. The system of claim 6, wherein the traffic condition processing unit of the vehicle-mounted device calculates the optimal warning time according to various specific vehicle condition information and driving environment provided by the vehicle condition module, the vehicle condition information to be considered at least comprises the optimal acceleration/deceleration and the average acceleration/deceleration, and the information collection can be obtained from GPS positioning information, or accelerometer, or OBD information, sensor, or combination thereof, and the optimal acceleration/deceleration can be obtained by analyzing the vehicle speed and related sensor information in a continuous time; the speed reducing parameter can be modified to the shortest time required by the vehicle to reduce the speed to stop according to the flow of the front and the rear vehicles, whether the vehicle runs on the uphill road section and the downhill road section and the current lane of the vehicle, and the optimal time for warning the driving is modified and calculated according to the shortest time.
16. The system of claim 8, wherein a road condition record is recorded as a triplet (T)j,χj,Γj) Represents; t isjRepresenting the time and x of the vehicle-mounted device detecting the road conditionjRepresenting the detection result χj0 denotes no traffic condition, χj1 indicates traffic condition; gamma-shapedjA preset due time representing the road condition,
Figure FDA0002272876270000044
Figure FDA0002272876270000043
indicating a validity period; m is the most effective number of road condition records used in calculating road condition confidence, for time T, N is the total number of traffic condition records, M is min { M, N }, where N is not more than N is the number of road condition records that have not yet expired, and (T)1,χ1,Γ1)、(T2,χ2,Γ2)…(Tm,χm,Γm) Representing the latest m traffic condition records;
the confidence of a road condition at time T can be defined as:
when the N is greater than 1, the reaction solution,
Figure FDA0002272876270000041
otherwise, conf is 0.1;
wherein when
Figure FDA0002272876270000045
When the temperature of the water is higher than the set temperature,
Figure FDA0002272876270000042
17. the system of claim 1, further comprising at least one third-party traffic data interface unit for receiving traffic information from a third party and other traffic information published on a public transportation integrated information distribution service Platform (PTX).
18. The image recognition-based traffic road condition internet of vehicles system of claim 17, wherein the third party road condition data interface unit is disposed on the rear end platform.
19. The system of claim 17, wherein the third party traffic data interface unit is disposed on the vehicle-mounted device.
CN201911111593.8A 2019-11-14 2019-11-14 Traffic road condition car networking system based on image is discerned Pending CN112804278A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911111593.8A CN112804278A (en) 2019-11-14 2019-11-14 Traffic road condition car networking system based on image is discerned

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911111593.8A CN112804278A (en) 2019-11-14 2019-11-14 Traffic road condition car networking system based on image is discerned

Publications (1)

Publication Number Publication Date
CN112804278A true CN112804278A (en) 2021-05-14

Family

ID=75803460

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911111593.8A Pending CN112804278A (en) 2019-11-14 2019-11-14 Traffic road condition car networking system based on image is discerned

Country Status (1)

Country Link
CN (1) CN112804278A (en)

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101729167A (en) * 2008-10-16 2010-06-09 陕西创秦电子科技有限公司 Method for processing traffic road condition information
CN101777253A (en) * 2009-12-24 2010-07-14 戴磊 Real-time road condition acquiring, analyzing and back-feeding and intelligent transportation integrated service system
CN102890869A (en) * 2012-09-25 2013-01-23 孙涛 Vehicle route predicting and notifying method and mobile intelligent terminal
CN103164986A (en) * 2011-12-14 2013-06-19 财团法人工业技术研究院 Warning system and method for specific road conditions of vehicle
CN105372654A (en) * 2015-12-14 2016-03-02 财团法人车辆研究测试中心 Obstacle classification reliability quantification method
TWI559267B (en) * 2015-12-04 2016-11-21 Method of quantifying the reliability of obstacle classification
CN106169243A (en) * 2016-08-25 2016-11-30 武汉理工大学 A kind of real-time road estimating system based on car networking and method
CN106971583A (en) * 2017-03-27 2017-07-21 宁波吉利汽车研究开发有限公司 A kind of traffic information shared system and sharing method based on vehicle-mounted networking equipment
CN110211385A (en) * 2019-06-26 2019-09-06 安徽鼎升自动化科技有限公司 A kind of traffic safety information system and method based on 5G network and navigation system
US20190311614A1 (en) * 2018-04-05 2019-10-10 Toyota Jidosha Kabushiki Kaisha Real-Time Traffic Monitoring with Connected Cars

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101729167A (en) * 2008-10-16 2010-06-09 陕西创秦电子科技有限公司 Method for processing traffic road condition information
CN101777253A (en) * 2009-12-24 2010-07-14 戴磊 Real-time road condition acquiring, analyzing and back-feeding and intelligent transportation integrated service system
CN103164986A (en) * 2011-12-14 2013-06-19 财团法人工业技术研究院 Warning system and method for specific road conditions of vehicle
CN102890869A (en) * 2012-09-25 2013-01-23 孙涛 Vehicle route predicting and notifying method and mobile intelligent terminal
TWI559267B (en) * 2015-12-04 2016-11-21 Method of quantifying the reliability of obstacle classification
CN105372654A (en) * 2015-12-14 2016-03-02 财团法人车辆研究测试中心 Obstacle classification reliability quantification method
CN106169243A (en) * 2016-08-25 2016-11-30 武汉理工大学 A kind of real-time road estimating system based on car networking and method
CN106971583A (en) * 2017-03-27 2017-07-21 宁波吉利汽车研究开发有限公司 A kind of traffic information shared system and sharing method based on vehicle-mounted networking equipment
US20190311614A1 (en) * 2018-04-05 2019-10-10 Toyota Jidosha Kabushiki Kaisha Real-Time Traffic Monitoring with Connected Cars
CN110211385A (en) * 2019-06-26 2019-09-06 安徽鼎升自动化科技有限公司 A kind of traffic safety information system and method based on 5G network and navigation system

Similar Documents

Publication Publication Date Title
TWI717102B (en) Traffic condition system for internet of vehicles based on image recognition
US11161503B2 (en) Vehicular communications network and methods of use and manufacture thereof
US11726437B2 (en) Vehicle control systems
US11009364B2 (en) Vehicular communications network and methods of use and manufacture thereof
CN107430006B (en) Method and system for avoiding accidents
Zeng et al. Potential connected vehicle applications to enhance mobility, safety, and environmental security.
US9786171B2 (en) Systems and methods for detecting and distributing hazard data by a vehicle
US7804423B2 (en) Real time traffic aide
CN108263360B (en) System and method for vehicle control in an immediate scene
US11586210B2 (en) Preemptive logical configuration of vehicle control systems
CN109606377A (en) A kind of emergency running behavior defence reminding method and system
CN108275152B (en) Vehicle system, computer-implemented method of controlling vehicle system, and storage medium
US20230118619A1 (en) Parking-stopping point management device, parking-stopping point management method, and vehicle device
US20190056230A1 (en) Driver alert system
EP4015333A1 (en) Indirect verification of speed limits based on contextual information for autonomous and semi-autonomous driving systems
US11776402B2 (en) Forecasting vehicle location occupancy
JP2020135674A (en) Traffic risk information output system and traffic risk information output program
CN108701416A (en) The method for detecting dangerous situation in road traffic
US11731645B2 (en) Driver warning system
CN110491153A (en) A kind of intelligent management system of car networking
CN112804278A (en) Traffic road condition car networking system based on image is discerned
CN116923432A (en) Driving assistance system and method
US20240067204A1 (en) Method for building an ad hoc virtual network and system
EP4303538A1 (en) System and method for an optimized routing of autonomous vehicles with risk aware maps
US20230399021A1 (en) Systems and methods for detecting restricted traffic zones for autonomous driving

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination