CN112477873A - Auxiliary driving and vehicle safety management system based on Internet of vehicles - Google Patents

Auxiliary driving and vehicle safety management system based on Internet of vehicles Download PDF

Info

Publication number
CN112477873A
CN112477873A CN202011476325.9A CN202011476325A CN112477873A CN 112477873 A CN112477873 A CN 112477873A CN 202011476325 A CN202011476325 A CN 202011476325A CN 112477873 A CN112477873 A CN 112477873A
Authority
CN
China
Prior art keywords
vehicle
module
information
unit
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202011476325.9A
Other languages
Chinese (zh)
Other versions
CN112477873B (en
Inventor
王骥
凌敏
袁志杰
林晓棠
罗颖杰
梁程锋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Ocean University
Original Assignee
Guangdong Ocean University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Ocean University filed Critical Guangdong Ocean University
Priority to CN202011476325.9A priority Critical patent/CN112477873B/en
Publication of CN112477873A publication Critical patent/CN112477873A/en
Application granted granted Critical
Publication of CN112477873B publication Critical patent/CN112477873B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R16/00—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for
    • B60R16/02—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements
    • B60R16/023—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements for transmission of signals between vehicle parts or subsystems
    • B60R16/0231—Circuits relating to the driving or the functioning of the vehicle
    • B60R16/0232—Circuits relating to the driving or the functioning of the vehicle for measuring vehicle parameters and indicating critical, abnormal or dangerous conditions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/29—Geographical information databases
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/953—Querying, e.g. by the use of web search engines
    • G06F16/9537—Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • G06V20/584—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of vehicle lights or traffic lights
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
    • G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
    • G08B21/02—Alarms for ensuring the safety of persons
    • G08B21/06—Alarms for ensuring the safety of persons indicating a condition of sleep, e.g. anti-dozing alarms
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/09—Arrangements for giving variable traffic instructions
    • G08G1/0962—Arrangements for giving variable traffic instructions having an indicator mounted inside the vehicle, e.g. giving voice messages
    • G08G1/0967—Systems involving transmission of highway information, e.g. weather, speed limits
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/09—Arrangements for giving variable traffic instructions
    • G08G1/0962—Arrangements for giving variable traffic instructions having an indicator mounted inside the vehicle, e.g. giving voice messages
    • G08G1/0968—Systems involving transmission of navigation instructions to the vehicle
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16Y—INFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]
    • G16Y20/00—Information sensed or collected by the things
    • G16Y20/10—Information sensed or collected by the things relating to the environment, e.g. temperature; relating to location
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16Y—INFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]
    • G16Y20/00—Information sensed or collected by the things
    • G16Y20/20—Information sensed or collected by the things relating to the thing itself
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16Y—INFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]
    • G16Y20/00—Information sensed or collected by the things
    • G16Y20/40—Information sensed or collected by the things relating to personal data, e.g. biometric data, records or preferences
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16Y—INFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]
    • G16Y40/00—IoT characterised by the purpose of the information processing
    • G16Y40/10—Detection; Monitoring
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16Y—INFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]
    • G16Y40/00—IoT characterised by the purpose of the information processing
    • G16Y40/20—Analytics; Diagnosis
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16Y—INFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]
    • G16Y40/00—IoT characterised by the purpose of the information processing
    • G16Y40/50—Safety; Security of things, users, data or systems
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16Y—INFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]
    • G16Y40/00—IoT characterised by the purpose of the information processing
    • G16Y40/60—Positioning; Navigation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
    • H04W4/02—Services making use of location information
    • H04W4/021—Services related to particular areas, e.g. point of interest [POI] services, venue services or geofences
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
    • H04W4/02—Services making use of location information
    • H04W4/029—Location-based management or tracking services
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
    • H04W4/30—Services specially adapted for particular environments, situations or purposes
    • H04W4/40—Services specially adapted for particular environments, situations or purposes for vehicles, e.g. vehicle-to-pedestrians [V2P]
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • B60W2040/0818—Inactivity or incapacity of driver
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • B60W2040/0818—Inactivity or incapacity of driver
    • B60W2040/0827—Inactivity or incapacity of driver due to sleepiness

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computing Systems (AREA)
  • Multimedia (AREA)
  • Databases & Information Systems (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Remote Sensing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Signal Processing (AREA)
  • Automation & Control Theory (AREA)
  • Human Computer Interaction (AREA)
  • Radar, Positioning & Navigation (AREA)
  • General Engineering & Computer Science (AREA)
  • Mechanical Engineering (AREA)
  • Data Mining & Analysis (AREA)
  • Transportation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Mathematical Physics (AREA)
  • Biomedical Technology (AREA)
  • Computer Security & Cryptography (AREA)
  • Business, Economics & Management (AREA)
  • Emergency Management (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Atmospheric Sciences (AREA)
  • Environmental & Geological Engineering (AREA)
  • Toxicology (AREA)
  • Traffic Control Systems (AREA)

Abstract

The invention discloses an auxiliary driving and vehicle safety management system based on a vehicle networking, which comprises a user end system, a vehicle data storage module, a user end communication module, a user end display module, a user end data processing unit and a user end positioning module, wherein the user end system comprises a user end data storage module, a user end communication module, a user end display module, a user end data processing unit and a user end positioning module; the vehicle end system comprises an in-vehicle data collection module, a driver driving state detection module, a lane identification detection module, a traffic road surface identification detection module, a vehicle end data storage module and a vehicle end communication module; the server end system comprises a server end data storage module, a server end data processing module and a server end communication module; the invention has the advantages of large development prospect, low cost of hardware part, hardware equipment arranged in the automobile and not eroded by the harsh environment outside the automobile, front and back end separation mode adopted in the framework of software, separated logic of data and view layer, enhanced expansibility and low cost of software development and upgrade.

Description

Auxiliary driving and vehicle safety management system based on Internet of vehicles
Technical Field
The invention belongs to the field of remote medical treatment, and particularly relates to a driving assistance and vehicle safety management system based on an internet of vehicles.
Background
At present, the traffic industry in China is rapidly developed, and vehicles become a part of daily life of people; the increase of the traffic flow causes the road to be blocked, thereby causing the low working efficiency of the whole traffic transportation system and the more serious problem of traffic accidents; the passive protection measures of the safety air bag and the crash barrier of the current vehicle cannot meet the requirements of people. Along with the development of the traffic industry and the construction and service capacity of the Beidou system in China. The technology of the Internet of vehicles starts to enter the rise period, and a practical and functional Internet of vehicles product is urgently needed to be created, so that the technology has great significance.
Disclosure of Invention
On the basis of a Beidou satellite positioning system, aiming at the problems and the defects of similar software in function and design, the method aims to provide a vehicle networking service with richer content and more complete functions for a user, obtains the position and the running track of a vehicle by analyzing and processing the positioning service provided by the Beidou navigation positioning satellite, and collects data such as the temperature, the humidity, the smoke concentration, the alcohol concentration, the distance of surrounding obstacles and the like in the vehicle by using a sensor; monitoring the driving state of a driver in real time by using a camera, and acquiring information of a road surface in front of a vehicle in real time; based on an OpenCV (open vehicle vision library) face spectrogram, the state of a driver is monitored by face detection and face characteristics, whether fatigue driving exists or the situation that attention is dispersed when the driver looks at a mobile phone and the like is detected, objects such as vehicles, roads, traffic lights and the like are identified by combining a computer vision technology, the functions of monitoring the running state of the vehicle, inquiring historical tracks, planning geo-fences, searching places, planning paths, forecasting weather and the like are realized on an independently built Web application, and a set of vehicle networking products with complete functions are provided.
The invention provides a driving assistance and vehicle safety management system based on an internet of vehicles, which comprises:
the client system comprises a client data storage module, a client communication module, a client display module, a client data processing unit and a client positioning module,
the user end data storage module is connected with the user end communication module, the user end data processing unit and the user end positioning module,
the user data processing unit is connected with the user display module,
wherein, the user terminal display module comprises,
a vehicle integrated information display unit for displaying vehicle integrated information of the target vehicle,
a driver information display early warning unit for displaying the driver information of the target vehicle and giving an early warning according to the driver information,
a location searching unit for inputting the target location, displaying the target location on the user side display module,
a path planning unit for inputting the target location, displaying the path to the target location on the user terminal display module,
the geo-fence unit is used for selecting a target area, displaying the target area on the client display module, displaying the motion condition of the target vehicle in the target area in real time through the client vehicle positioning module,
a vehicle history track unit for inquiring and displaying the history motion track of the target vehicle,
a user identification unit of a user terminal; user identification and login for the user-end system;
the vehicle end system comprises an in-vehicle data collection module, a driver driving state detection module, a lane identification detection module, a traffic road surface identification detection module, a vehicle end data storage module and a vehicle end communication module,
the vehicle-end data storage module is connected with the in-vehicle data collection module, the driver driving state detection module, the lane identification detection module, the traffic road surface identification detection module and the vehicle-end communication module,
wherein, the in-vehicle data collection module is used for collecting the environmental information of the target vehicle, positioning the vehicle and obtaining the motion trail information of the target vehicle,
the driver driving state detection module is used for collecting and detecting a driver state image of the target vehicle through an internal camera arranged on the target vehicle,
the lane recognition detection module is used for collecting and detecting the lane condition of the road surface of the target vehicle in the driving process through an external camera arranged on the target vehicle,
the traffic road surface identification detection module is used for detecting and identifying the road surface condition of the road surface through the external camera;
the server-side system comprises a server-side data storage module, a server-side data processing module and a server-side communication module,
the server data storage module is connected with the server data collection module and the server data processing module.
Preferably, the target area comprises a plurality of areas, wherein each area comprises a superposition part and a non-superposition part; adding or deleting the target area through the geo-fence unit.
Preferably, the historical movement track includes a plurality of movement tracks, the movement track information is collected through a positioning function of the in-vehicle data collection module, the movement track information is transmitted to the server data storage module through the vehicle end communication module, the historical movement track is generated through the server data processing module, and the historical movement track is displayed and inquired through the client communication module based on the server data storage module.
Preferably, the in-vehicle data collection module comprises a smoke sensor unit, a temperature sensor unit, a vehicle positioning unit, a data conversion unit,
wherein the content of the first and second substances,
the smoke alcohol sensor unit is used for detecting internal gas information of the target vehicle;
the temperature and humidity sensor unit is used for detecting internal temperature and humidity information of the target vehicle;
the vehicle positioning unit is used for positioning the target vehicle in real time to obtain the motion track information;
the data conversion unit is used for converting the internal gas information, the internal temperature and humidity information and the motion track information into digital signals;
the in-vehicle data collection module is configured to transmit the digital signal to the server-side data storage module through the vehicle-side communication module based on the digital signal, process the digital signal based on the server-side data processing module to obtain a processing result, transmit the processing result to the user-side system through the server-side communication module, and display the processing result through the vehicle integrated information display unit, where the vehicle integrated information includes the processing result, and the vehicle-side communication module is an EC204G wireless communication module.
Preferably, the smoke alcohol sensor unit comprises, an mp2 fog sensor module and an mp3 alcohol sensor module;
the data conversion unit is a mcu converter;
the temperature and humidity sensor unit is a DHT11 temperature and humidity sensor;
the vehicle positioning unit is an ATK1218-BD big dipper module;
and the vehicle positioning unit obtains the motion trail information through the data conversion unit.
Preferably, the driver driving state detection module includes a face recognition unit;
the driver driving state monitoring module is used for obtaining the face image of the driver of the target vehicle through the internal camera, carrying out eye positioning on the face image of the driver through the face recognition unit to obtain the eye positioning image of the face of the driver, carrying out recognition processing on the eye positioning image of the face of the driver to obtain the driver information, and transmitting the driver information to the driver information display early warning unit through the vehicle end communication module.
Preferably, the driver face eye positioning image includes a left eye positioning image and a right eye positioning image, wherein the left eye positioning image is obtained by selecting three left eye positioning points, and the right eye positioning image is obtained by selecting three right eye positioning points;
the face recognition unit is used for recognizing the left eye positioning image and the right eye positioning image based on a threshold value by setting the threshold value of the driver face eye positioning image, wherein when the left eye positioning image and the right eye positioning image do not meet the threshold value, early warning information is output, and when the left eye positioning image and the right eye positioning image meet the threshold value, normal information is output;
the driver information comprises early warning information and normal information;
and the driver information display early warning unit carries out early warning according to the early warning information.
Preferably, the lane recognition and detection module is configured to collect an original lane image of the road surface through the external camera, perform grayscale map processing on the original lane image for a plurality of times to obtain an original target image, perform gaussian blur processing, image contour processing, and hough direct detection processing on the original target image to obtain target lane image data, transmit the target lane image data to the user end system through the vehicle end communication module, and display the target lane image data through the vehicle integrated information display unit, where the vehicle integrated information further includes the target lane image data.
Preferably, the traffic road surface recognition and detection module is configured to obtain an original road surface image through the external camera, recognize a road surface object image of the original road surface image based on YOLO, construct a road surface condition model according to the original road surface image based on the road surface object image, obtain the road surface condition, transmit the road surface condition to the user end system through the vehicle end communication module, and display the road surface condition through the vehicle comprehensive information display unit, where the vehicle comprehensive information further includes the road surface condition.
Preferably, the user end system further comprises a convenient life module;
the convenient life module comprises a weather condition display unit and a current life index unit on the same day;
the user end system displays the temperature information, the somatosensory temperature information, the wind direction information and the wind power information of the area based on the weather condition display unit on the same day through the user end positioning module;
and the user end system displays the life information of the area based on the current life index unit through the user end positioning module.
The positive progress effects of the invention are as follows:
the invention has wide development prospect, the cost required by the hardware part is lower, and the hardware equipment is arranged in the automobile and cannot be eroded by the harsh environment outside the automobile. The durability of the hardware is greatly increased. The architecture in the aspect of software adopts a mode of separating a front end from a back end, the mode realizes the characteristic of high cohesion and low coupling, the logic of data is separated from a view layer, the development efficiency is higher, the expansibility is enhanced, and the development and upgrading cost of the software is low.
The greatest characteristic of the invention is in the aspect of computer vision recognition technology, and at present, when a traffic accident occurs, passive protection measures such as an air bag or an automobile crash barrier are difficult to better ensure the life safety of people, but if the computer vision recognition technology is combined, detection and judgment of some behaviors of a driver, for example, the fatigue state of the driver is detected, and whether the driver is in an abnormal state or not, are carried out on the driver in the driving process. If the signs of fatigue are found, the user is reminded, the purpose of preventing the user from getting ill in the bud is achieved, and the occurrence of traffic accidents is reduced. And data in the vehicle is received in real time, so that a user can see the real-time condition in the vehicle, and the driving safety is guaranteed.
Drawings
FIG. 1 is a block diagram of a system according to the present invention;
FIG. 2 is a diagram illustrating the overall architecture of the system according to the present invention;
FIG. 3 is a flow chart of a system according to the present invention;
FIG. 4 is a smoke alcohol sensing unit according to the present invention, wherein 4a is an mq2 smoke sensor and 4b is an mq3 alcohol sensor;
FIG. 5 illustrates an ATK1218-BD Beidou module of the present invention;
FIG. 6 illustrates a DHT11 temperature and humidity sensor according to the present invention;
FIG. 7 is an EC204G wireless communication module according to the present invention;
FIG. 8 is a hardware flow diagram according to the present invention;
FIG. 9 is a software flow diagram according to the present invention;
FIG. 10 is a diagram of a detection process according to the present invention;
FIG. 11 is a graphical representation of test data according to the present invention;
FIG. 12 is a schematic diagram of a location search according to the present invention;
FIG. 13 is a schematic diagram of a path planning according to the present invention;
FIG. 14 is a schematic representation of a real-time vehicle positioning according to the present invention;
FIG. 15 is a schematic view of a fenced area ready for entry in accordance with the invention;
FIG. 16 is a schematic view of an out-of-fence area according to the present invention;
FIG. 17 is a schematic illustration of a historical trajectory of a vehicle according to the present invention;
FIG. 18 is an illustration of an operator interface for screening historical vehicle trajectories, in accordance with the present invention;
FIG. 19 is a weather index map according to the present invention;
FIG. 20 is a life index graph according to the present invention;
FIG. 21 is a circuit diagram of a system according to the present invention;
fig. 22 illustrates the pen function of the present invention.
Detailed Description
The following description of the preferred embodiments of the present invention will be provided in conjunction with the accompanying drawings to describe the technical solutions of the present invention in detail, but not to limit the present invention to the scope of the embodiments described.
As shown in fig. 1 to 21, the present embodiment provides a driving assistance and vehicle safety management system based on internet of vehicles, including:
the client system comprises a client data storage module, a client communication module, a client display module, a client data processing unit and a client positioning module,
the user end data storage module is connected with the user end communication module, the user end data processing unit and the user end positioning module,
the user data processing unit is connected with the user display module,
wherein, the user terminal display module comprises,
a vehicle integrated information display unit for displaying vehicle integrated information of the target vehicle,
a driver information display early warning unit for displaying the driver information of the target vehicle and giving an early warning according to the driver information,
a location searching unit for inputting the target location, displaying the target location on the user side display module,
a path planning unit for inputting the target location, displaying the path to the target location on the user terminal display module,
the geo-fence unit is used for selecting a target area, displaying the target area on the client display module, displaying the motion condition of the target vehicle in the target area in real time through the client vehicle positioning module,
a vehicle history track unit for inquiring and displaying the history motion track of the target vehicle,
a user identification unit of a user terminal; user identification and login for the user-end system;
the vehicle end system comprises an in-vehicle data collection module, a driver driving state detection module, a lane identification detection module, a traffic road surface identification detection module, a vehicle end data storage module and a vehicle end communication module,
the vehicle-end data storage module is connected with the in-vehicle data collection module, the driver driving state detection module, the lane identification detection module, the traffic road surface identification detection module and the vehicle-end communication module,
wherein, the in-vehicle data collection module is used for collecting the environmental information of the target vehicle, positioning the vehicle and obtaining the motion trail information of the target vehicle,
the driver driving state detection module is used for collecting and detecting a driver state image of the target vehicle through an internal camera arranged on the target vehicle,
the lane recognition detection module is used for collecting and detecting the lane condition of the road surface of the target vehicle in the driving process through an external camera arranged on the target vehicle,
the traffic road surface identification detection module is used for detecting and identifying the road surface condition of the road surface through the external camera;
the server-side system comprises a server-side data storage module, a server-side data processing module and a server-side communication module,
the server data storage module is connected with the server data collection module and the server data processing module.
The target area comprises a plurality of areas, wherein each area comprises an overlapped part and a non-overlapped part; adding or deleting the target area through the geo-fence unit.
The historical movement track comprises a plurality of movement tracks, the movement track information is collected through the positioning function of the in-vehicle data collection module, the movement track information is transmitted to the server data storage module through the vehicle end communication module, the historical movement track is generated through the server data processing module, and the historical movement track is displayed and inquired through the user end communication module and based on the server data storage module.
The in-vehicle data collection module comprises a smoke sensor unit, a temperature sensor unit, a vehicle positioning unit and a data conversion unit,
wherein the content of the first and second substances,
the smoke alcohol sensor unit is used for detecting internal gas information of the target vehicle;
the temperature and humidity sensor unit is used for detecting internal temperature and humidity information of the target vehicle;
the vehicle positioning unit is used for positioning the target vehicle in real time to obtain the motion track information;
the data conversion unit is used for converting the internal gas information, the internal temperature and humidity information and the motion track information into digital signals;
the in-vehicle data collection module is configured to transmit the digital signal to the server-side data storage module through the vehicle-side communication module based on the digital signal, process the digital signal based on the server-side data processing module to obtain a processing result, transmit the processing result to the user-side system through the server-side communication module, and display the processing result through the vehicle integrated information display unit, where the vehicle integrated information includes the processing result, and the vehicle-side communication module is an EC204G wireless communication module.
The smoke alcohol sensor unit comprises an mp2 smoke sensor module and an mp3 alcohol sensor module;
the data conversion unit is a mcu converter;
the temperature and humidity sensor unit is a DHT11 temperature and humidity sensor;
the vehicle positioning unit is an ATK1218-BD big dipper module;
and the vehicle positioning unit obtains the motion trail information through the data conversion unit.
Preferably, the driver driving state detection module includes a face recognition unit;
the driver driving state monitoring module is used for obtaining the face image of the driver of the target vehicle through the internal camera, carrying out eye positioning on the face image of the driver through the face recognition unit to obtain the eye positioning image of the face of the driver, carrying out recognition processing on the eye positioning image of the face of the driver to obtain the driver information, and transmitting the driver information to the driver information display early warning unit through the vehicle end communication module.
The driver face eye positioning image comprises a left eye positioning image and a right eye positioning image, wherein the left eye positioning image is obtained by selecting three left eye positioning points, and the right eye positioning image is obtained by selecting three right eye positioning points;
the face recognition unit is used for recognizing the left eye positioning image and the right eye positioning image based on a threshold value by setting the threshold value of the driver face eye positioning image, wherein when the left eye positioning image and the right eye positioning image do not meet the threshold value, early warning information is output, and when the left eye positioning image and the right eye positioning image meet the threshold value, normal information is output;
the driver information comprises early warning information and normal information;
and the driver information display early warning unit carries out early warning according to the early warning information.
The lane identification detection module is used for collecting an original lane image of the road surface through the external camera, performing grayscale image processing on the original lane image for a plurality of times to obtain an original target image, performing Gaussian blur processing, image contour processing and Hough direct detection processing on the original target image to obtain target lane image data, transmitting the target lane image data to the user end system through the vehicle end communication module, and displaying the target lane image data through the vehicle comprehensive information display unit, wherein the vehicle comprehensive information further comprises the target lane image data.
The traffic road surface identification detection module is used for obtaining an original road surface image through the external camera, identifying a road surface object image of the original road surface image based on YOLO, constructing a road surface condition model according to the original road surface image based on the road surface object image, obtaining the road surface condition, transmitting the road surface condition to the user end system through the vehicle end communication module, and displaying the road surface condition through the vehicle comprehensive information display unit, wherein the vehicle comprehensive information further comprises the road surface condition.
The user end system also comprises a convenient life module;
the convenient life module comprises a weather condition display unit and a current life index unit on the same day;
the user end system displays the temperature information, the somatosensory temperature information, the wind direction information and the wind power information of the area based on the weather condition display unit on the same day through the user end positioning module;
and the user end system displays the life information of the area based on the current life index unit through the user end positioning module.
The following explains the design idea and technical constitution of the technical scheme in the application in detail:
on the basis of a Beidou satellite positioning system, aiming at the problems and the defects of similar software in function and design, the method aims to provide a vehicle networking service with richer content and more complete functions for a user, obtains the position and the running track of a vehicle by analyzing and processing the positioning service provided by the Beidou navigation positioning satellite, and collects data such as the temperature, the humidity, the smoke concentration, the alcohol concentration, the distance of surrounding obstacles and the like in the vehicle by using a sensor; monitoring the driving state of a driver in real time by using a camera, and acquiring information of a road surface in front of a vehicle in real time; based on an OpenCV (open vehicle vision library) face spectrogram, the state of a driver is monitored by face detection and face characteristics, whether fatigue driving exists or the situation that attention is dispersed when the driver looks at a mobile phone and the like is detected, objects such as vehicles, roads, traffic lights and the like are identified by combining a computer vision technology, the functions of monitoring the running state of the vehicle, inquiring historical tracks, planning geo-fences, searching places, planning paths, forecasting weather and the like are realized on an independently built Web application, and a set of vehicle networking products with complete functions are provided.
The overall architecture of the invention is as follows: the cloud server is matched as a data transfer station, the stm32f103 is adopted as a main control chip by hardware equipment, vehicle positioning data and vehicle internal environment data are collected through a serial port, a GPIO (general purpose input/output) and an analog-to-digital converter, and TCP (transmission control protocol) protocol communication can be carried out through the EC204G module; the web client is connected with the cloud server through the IP and the port number, and the web client and the cloud server are connected at the same time to realize remote connection. Thereby realizing communication between data and transmission between data. The communication module is designed based on TCP, WebSocket protocol and Netty framework.
The specific implementation of the framework is as follows: and the cloud server carries a network service program written based on the Netty framework and waits for the connection of the client. And after the server program of the Web application establishes connection with the cloud server according to the IP address and the port number, data processing and storage are carried out on the response of the cloud server. And the server program of the Web application sends the processed data to the browser for display through WebSocket connection.
In terms of hardware, we use the stm32rct6, ARM-Cortex-M3 core microprocessor as the main control CPU for motion, SRAM memory with maximum operating frequency of 72MHZ, 128 KB. The chip has rich peripherals such as A/D conversion, serial ports and the like, and has high cost performance. The modules of mq2 and mq3 are used, analog signals sent by the analog-to-digital conversion acquisition module are converted into digital signals, and the digital signals are processed to obtain the concentration. An ATK1218-BD positioning module is used in the positioning aspect, original Beidou positioning original data is received through a serial port, and the data format of a protocol is analyzed to obtain longitude and latitude. The temperature and humidity in the vehicle are collected through the DHT11 module, and finally the data are sent to the server through the EC204G module. A high-definition camera is built on the raspberry, through face recognition and WiFi communication, a server can obtain real-time video and perform image recognition through accessing an ip and a port number.
In terms of software, the Web application is mainly developed by Java EE and Vue. The service end uses IntelliJ IDEA as a development platform, uses Spring Boot as a core framework, combines Spring Data and MyBatiSPlus as a database interaction framework, and uses Netty as a communication framework for development. The client uses Webstorm as a development platform, uses Vue.js as a core framework and ElementUI as a UI framework, and is developed by combining the interaction of Express and a database. The cloud server program is developed using a Netty framework and installed in an environment accessible to an external network of the CentOS7. And the database aspect adopts MySQL of a relational data management system. Through the application, the user can realize the functions of monitoring the running state of the vehicle, inquiring historical tracks, planning geo-fences, searching places, planning paths, forecasting weather and the like
We choose an arilocos server on the server side, and the remote server is an arilocos server from the company arioba, which provides scalable computing capacity and has a public network unique IP, and theoretically can communicate with the server anywhere as long as it can connect to the internet. Through the Ali cloud server, the application program can be deployed rapidly, and resources such as a CPU, a memory and a hard disk are used. Our server uses the Linux centros 7 system to establish contact with hardware devices and server programs of Web applications using techniques such as Netty, process, and NIO streams. After the user accesses the application program, the connection with the cloud server can be automatically established according to the IP address and the port. After successful connection, the user can easily establish contact with the automobile equipment through the transfer station of the remote server.
In the technical selection of the front-end aspect, the Vue framework and the Vue framework have the advantages of light weight, simplicity and easiness in learning, bidirectional data binding, componentization, view, separation of data and structures, virtual DOM and high running speed. And a quick and attractive interface is provided for a front-end developer to construct a page.
In the computer vision recognition technology, an OpenCV library of Python is adopted, Python comparison has great advantages compared with other languages in the aspect of computer vision recognition, and meanwhile, a computer vision recognition framework provided by Python, namely a YOLO framework, is adopted, so that a model belonging to the computer vision recognition technology can be trained through yolv 3 to realize a corresponding vision recognition function of the computer vision recognition technology.
And the alcohol smoke detection module is used for collecting the alcohol concentration and the smoke concentration in the vehicle. The module has the advantages of high sensitivity, quick response, good stability and long service life. The method has good anti-interference performance, and can accurately eliminate the interference information of the irritant non-combustible smoke. The acquisition of MQ-2 smoke concentration is realized, and the acquisition of signals can be completed only by realizing an ADC0832 acquisition function. However, the signal acquired by the ADC0832 is only an original signal, and needs to be converted into an actual smoke concentration, and correction and formula conversion need to be performed according to the characteristics of MQ-2, so as to obtain an actual concentration value.
The vehicle positioning unit is used for positioning the vehicle position, positioning is carried out by adopting a GPS and Beidou dual mode, various parameter settings and data receiving and sending can be carried out by the module through a serial port, and the internal FLASH can be stored, so that the use is convenient. Through the ATK-S1218-BD GPS/Beidou module, any single chip microcomputer (3.3V/5V power supply) can conveniently realize GPS/Beidou positioning.
The ATK1218-BD GPS/Beidou module has the following characteristics:
the module adopts an S1216F8-BD module, and is small in size and excellent in performance.
The module can set various parameters through a serial port and can be stored in the internal FLASH, so that the use is convenient.
The module is provided with an IPX interface and can be connected with various active antennas
The module is compatible with 3.3V/5V level, and is convenient to connect various single chip microcomputer systems.
The module is provided with a rechargeable backup battery and can be powered down to keep ephemeris data
The temperature and humidity sensor unit is used for collecting the temperature and the humidity in the vehicle, and the module applies a special digital module acquisition technology and a temperature and humidity sensing technology to ensure that the product has extremely high reliability and excellent long-term stability. DATA is used for communication and synchronization between the microprocessor and the DHT11, a single-bus DATA format is adopted, one-time communication time is about 4ms, a DATA fractional part and an integer part are adopted, the specific format is described below, the current fractional part is used for later expansion, and the operation flow of reading zero is as follows:
the one-time complete data transmission is 40 bits, and the high bit is first out. The data format is that when the transmission of 8-bit humidity integer data + 8-bit humidity decimal data +8 bi-temperature integer data + 8-bit temperature decimal data + 8-bit check sum data is correct, the check sum data is equal to the last 8-bit user MCU of the result of "8-bit humidity integer data + 8-bit humidity decimal data +8 bi-temperature integer data + 8-bit temperature decimal data", after a start signal is sent by the last 8-bit user MCU, the DHT11 is switched to the high-speed mode from the low-power mode, after the host start signal is finished, the DHT11 sends a response signal, 40-bit data is sent out, signal acquisition is triggered once, a user can selectively read partial data, in the slave mode, the DHT11 receives the start signal to trigger temperature and humidity acquisition once, and if the host sends the start signal is not received, the DHT11 does not actively carry out temperature and humidity acquisition and then switches to the low-speed mode.
EC 20R 2.1 is an LTE Cat4 wireless communication module proposed by remote communication, and supports the maximum downlink rate of 150Mbps and the maximum uplink rate of 50Mbps by adopting the LTE 3GPP Rel.11 technology; meanwhile, a remote communication UMTS/HSPA + UC20 module and a multi-network LTE EC20/EC21/EC25/EG25-G module are compatible on the package, and seamless switching between a 3G network and a 4G network is realized. The module provides serial port communication, the serial port can be used for AT command or data transmission, 9600, 19200, 38400, 57600, 115200, 230400, 460800 and 921600bps baud rate are supported, the default baud rate is 115200bps, and serial port communication with mcu can be achieved.
Principle part of hardware
In the data collection module in the car: mq2 and mq3 sensors are used, which belong to tin dioxide semiconductor gas sensitive materials. When in contact with smoke, the surface conductivity changes if the potential barrier at the grain boundaries is altered by the smoke. By using the information, the existence information of the smoke can be obtained, and the larger the concentration of the smoke is, the larger the conductivity is, the lower the output resistance is, and the larger the output analog signal is. The mcu may convert the analog signal into a digital signal, converted into a concentration value. The DHT11 temperature and humidity sensor for collecting temperature and humidity is communicated with a processor by adopting a single bus data format, the one-time communication time is about 4ms, and a data fractional part and an integer part read data bits by time sequence control. The ATK1218 module Beidou module used in the positioning aspect outputs GPS/Beidou positioning data by adopting an NMEA0183 protocol in a default manner, the module can be configured by a SkyTraq protocol, the mcu receives the output original data through a serial port, and the data format of the NMEA0183 protocol is analyzed to obtain the positioning data. Vehicle data are transmitted to a server in real time, an EC204G module is used, the module is controlled by AT instructions through a serial port, the main control board is connected with a cloud server through a TCP protocol, and vehicle data are transmitted in real time.
The driver driving state detection module comprises: the method includes the steps that a high-definition camera is built by using a raspberry pi 4B, data transmitted back by the camera in a vehicle are subjected to face detection by using an opencv library of Python, 6 mark points are marked on the left eye and the right eye of a face image respectively based on face spectrogram data carried by the opencv library, and when the mark points of the left eye and the right eye are in a certain state which is beyond the limit of the user, a warning is sent through a threshold value set by the user, so that the purpose of better detecting the driving state of a driver is achieved.
The lane identification detection module comprises: and (5) carrying out lane line detection on the transmitted data by using an opencv library of Python. After the images are subjected to denoising processing, image contour, Hough line detection and the like through multiple times of gray level image processing and Gaussian blur, a considerable lane contour line set can be obtained.
In the traffic road surface discernment detection module: the velocity of YOLOv3 is higher than that of other frame visualizations when the object is identified and detected based on YOLO, and the velocity is more consistent with the item. The yolov3 training model is pre-trained, the original accuracy is maintained, the recognition speed is improved, meanwhile due to the limitation of the performance of the running machine, the original frame is modified, the recognition data amount is reduced through a data screening method, and meanwhile the recognition accuracy can be well maintained, so that the recognition speed is further improved.
Software principle part:
in a module for planning fences, the front end is responsible for providing a button for adding fences, when a user selects to add fences, the shape and size of the fences can be controlled by a mouse, finally, longitude and latitude corresponding to the fences are collected and sent to a background java client, java preprocesses the data and then adds the data to a MySQL server, and the function of storing fences is achieved.
In the fence checking module, after a user selects a fence, the system automatically starts to judge whether the current vehicle running track has operation of entering or exiting the fence through a series of algorithms, and if the vehicle enters or leaves the fence area, the browser sends a corresponding notice to the user.
In the history track module, the principle of storing the history track is judged by the connection condition of the browser to the back end java, when the browser is successfully connected with the java client, the corresponding history track is stored, the stored history track is ended after the browser is disconnected with the java client, the stored data are sent to the java client once, and the java client stores the data into MySQL for later viewing by a user.
System test scheme
The test purpose is as follows:
the report aims at the stability, the usability, the rendering time of the webpage UI and some unknown errors, finds the possible performance problems in the existing system, and provides feasible suggestions to reduce the subsequent work risk as much as possible and provide guarantee for maintaining stable operation.
Test range:
and performing system tests including function tests, performance tests, user access and safety control tests and user interface tests according to the project development specifications, the software requirement specification specifications and corresponding design documents. The remote server has the main functions of carrying out the sonna sister with the remote server and receiving data transmitted by hardware to carry out some processing, and the correct reality is carried out on a UI interface.
And (3) testing environment:
configuration: the server adopts an Ali cloud server, a CentOS7.364 bit operating system and a memory 4G;
application software: public browsers such as a Firefox browser, a Google browser and the like;
the test procedure, as shown in fig. 10;
test data, as shown in fig. 11.
The system of the invention realizes the function
And (3) searching the place: as shown in fig. 12, in the location search module, a user may search for a corresponding location in a search bar, and a landmark of an associated location in an area is displayed on a map to provide the user with a choice for viewing.
Path planning: as shown in fig. 13, the user can select a starting location and a travel mode in the route planning module, and then can search some relevant routes to achieve the navigation effect.
Real-time vehicle positioning: as shown in fig. 14, in the vehicle positioning module, the latitude and longitude of the hardware can be received, then the latitude and longitude data is sent to the server, and the client receives the data and displays the data in the browser. Real-time positioning of vehicle
The fence function: as shown in fig. 22, the user can add or delete corresponding fences in the fence according to the needs of the user; as shown in fig. 15, the successfully added fence can be selectively checked, and when the vehicle enters the fence, the user is prompted that the vehicle enters the fence range area; as shown in fig. 16, when the vehicle leaves the fence, the user is prompted that the vehicle leaves the fence area.
Looking at the historical trajectory of the vehicle, as shown in figures 17-18,
the user can check the historical driving tracks in the interface, and can also screen the historical driving tracks according to keywords to search the desired historical tracks.
In-vehicle data real-time monitoring
In the in-vehicle data monitoring module, the client can receive data transmitted by hardware, and the data is displayed on a browser for a user to view. And the data acquired by the hardware is sent to the Ali cloud server, and then the java client processes the data sent by the hardware by connecting the corresponding IP address and port number and sends the processed data to the browser for displaying to the user.
As shown in fig. 19 and 20, in the weather and life index module, a user can obtain weather information such as temperature, sensible temperature, wind direction, wind power and the like of a region where the user is located, and can view a current life index. Providing a user with an exhaustive life guide.
Computer vision recognition
In a driver fatigue detection module, driving image data of a user can be received through a camera, then the data can be subjected to Python and Python real-time data detection through opencv, then incoming images are processed, finally the processed images are transmitted back, and whether the driver dozes or not is known through some algorithms for judging eyes.
In the traffic road surface detection module, image data on a street can be sent to Python again, then the Python carries out image recognition through opencv, and vehicles, pedestrians, traffic lights and the like on the road are detected.
In the route detection module, the Python identifies and detects the lane lines on the road through opencv. And feeding back the processed result to the user.
The invention has wide development prospect, the cost required by the hardware part is lower, and the hardware equipment is arranged in the automobile and cannot be eroded by the harsh environment outside the automobile. The durability of the hardware is greatly increased. In terms of software, the framework of the invention adopts a mode of separating a front end from a back end, the mode realizes the characteristic of high cohesion and low coupling, the logic of data is separated from a view layer, the development is more efficient, the expansibility is enhanced, and the development and upgrade cost of the software is low.
The greatest characteristic of the invention is in the aspect of computer vision recognition technology, and at present, when a traffic accident occurs, passive protection measures such as an air bag or an automobile crash barrier are difficult to better ensure the life safety of people, but if the computer vision recognition technology is combined, detection and judgment of some behaviors of a driver, for example, the fatigue state of the driver is detected, and whether the driver is in an abnormal state or not, are carried out on the driver in the driving process. If the signs of fatigue are found, the user is reminded, the purpose of preventing the user from getting ill in the bud is achieved, and the occurrence of traffic accidents is reduced. And data in the vehicle is received in real time, so that a user can see the real-time condition in the vehicle, and the driving safety is guaranteed.
The above-described embodiments are merely illustrative of the preferred embodiments of the present invention, and do not limit the scope of the present invention, and various modifications and improvements of the technical solution of the present invention can be made by those skilled in the art without departing from the spirit of the present invention, and the scope of the present invention is defined by the claims.

Claims (10)

1. A driving assistance and vehicle safety management system based on the Internet of vehicles is characterized by comprising:
the client system comprises a client data storage module, a client communication module, a client display module, a client data processing unit and a client positioning module,
the user end data storage module is connected with the user end communication module, the user end data processing unit and the user end positioning module,
the user data processing unit is connected with the user display module,
wherein, the user terminal display module comprises,
a vehicle integrated information display unit for displaying vehicle integrated information of the target vehicle,
a driver information display early warning unit for displaying the driver information of the target vehicle and giving an early warning according to the driver information,
a location searching unit for inputting the target location, displaying the target location on the user side display module,
a path planning unit for inputting the target location, displaying the path to the target location on the user terminal display module,
the geo-fence unit is used for selecting a target area, displaying the target area on the client display module, displaying the motion condition of the target vehicle in the target area in real time through the client vehicle positioning module,
a vehicle history track unit for inquiring and displaying the history motion track of the target vehicle,
a user identification unit of a user terminal; user identification and login for the user-end system;
the vehicle end system comprises an in-vehicle data collection module, a driver driving state detection module, a lane identification detection module, a traffic road surface identification detection module, a vehicle end data storage module and a vehicle end communication module,
the vehicle-end data storage module is connected with the in-vehicle data collection module, the driver driving state detection module, the lane identification detection module, the traffic road surface identification detection module and the vehicle-end communication module,
wherein, the in-vehicle data collection module is used for collecting the environmental information of the target vehicle, positioning the vehicle and obtaining the motion trail information of the target vehicle,
the driver driving state detection module is used for collecting and detecting a driver state image of the target vehicle through an internal camera arranged on the target vehicle,
the lane recognition detection module is used for collecting and detecting the lane condition of the road surface of the target vehicle in the driving process through an external camera arranged on the target vehicle,
the traffic road surface identification detection module is used for detecting and identifying the road surface condition of the road surface through the external camera;
the server-side system comprises a server-side data storage module, a server-side data processing module and a server-side communication module,
the server data storage module is connected with the server data collection module and the server data processing module.
2. The Internet of vehicles based assistant driving and vehicle safety management system of claim 1,
the target area comprises a plurality of areas, wherein each area comprises an overlapped part and a non-overlapped part; adding or deleting the target area through the geo-fence unit.
3. The Internet of vehicles based assistant driving and vehicle safety management system of claim 1,
the historical movement track comprises a plurality of movement tracks, the movement track information is collected through the positioning function of the in-vehicle data collection module, the movement track information is transmitted to the server data storage module through the vehicle end communication module, the historical movement track is generated through the server data processing module, and the historical movement track is displayed and inquired through the user end communication module and based on the server data storage module.
4. The Internet of vehicles based assistant driving and vehicle safety management system of claim 1,
the in-vehicle data collection module comprises a smoke sensor unit, a temperature sensor unit, a vehicle positioning unit and a data conversion unit,
wherein the content of the first and second substances,
the smoke alcohol sensor unit is used for detecting internal gas information of the target vehicle;
the temperature and humidity sensor unit is used for detecting internal temperature and humidity information of the target vehicle;
the vehicle positioning unit is used for positioning the target vehicle in real time to obtain the motion track information;
the data conversion unit is used for converting the internal gas information, the internal temperature and humidity information and the motion track information into digital signals;
the in-vehicle data collection module is configured to transmit the digital signal to the server-side data storage module through the vehicle-side communication module based on the digital signal, process the digital signal based on the server-side data processing module to obtain a processing result, transmit the processing result to the user-side system through the server-side communication module, and display the processing result through the vehicle integrated information display unit, where the vehicle integrated information includes the processing result, and the vehicle-side communication module is an EC204G wireless communication module.
5. The Internet of vehicles based assistant driving and vehicle safety management system of claim 4,
the smoke alcohol sensor unit comprises an mp2 smoke sensor module and an mp3 alcohol sensor module;
the data conversion unit is a mcu converter;
the temperature and humidity sensor unit is a DHT11 temperature and humidity sensor;
the vehicle positioning unit is an ATK1218-BD big dipper module;
and the vehicle positioning unit obtains the motion trail information through the data conversion unit.
6. The Internet of vehicles based assistant driving and vehicle safety management system of claim 1,
the driver driving state detection module comprises a face recognition unit;
the driver driving state monitoring module is used for obtaining the face image of the driver of the target vehicle through the internal camera, carrying out eye positioning on the face image of the driver through the face recognition unit to obtain the eye positioning image of the face of the driver, carrying out recognition processing on the eye positioning image of the face of the driver to obtain the driver information, and transmitting the driver information to the driver information display early warning unit through the vehicle end communication module.
7. The Internet of vehicles based assistant driving and vehicle safety management system of claim 6,
the driver face eye positioning image comprises a left eye positioning image and a right eye positioning image, wherein the left eye positioning image is obtained by selecting three left eye positioning points, and the right eye positioning image is obtained by selecting three right eye positioning points;
the face recognition unit is used for recognizing the left eye positioning image and the right eye positioning image based on a threshold value by setting the threshold value of the driver face eye positioning image, wherein when the left eye positioning image and the right eye positioning image do not meet the threshold value, early warning information is output, and when the left eye positioning image and the right eye positioning image meet the threshold value, normal information is output;
the driver information comprises early warning information and normal information;
and the driver information display early warning unit carries out early warning according to the early warning information.
8. The Internet of vehicles based assistant driving and vehicle safety management system of claim 1,
the lane identification detection module is used for collecting an original lane image of the road surface through the external camera, performing grayscale image processing on the original lane image for a plurality of times to obtain an original target image, performing Gaussian blur processing, image contour processing and Hough direct detection processing on the original target image to obtain target lane image data, transmitting the target lane image data to the user end system through the vehicle end communication module, and displaying the target lane image data through the vehicle comprehensive information display unit, wherein the vehicle comprehensive information further comprises the target lane image data.
9. The Internet of vehicles based assistant driving and vehicle safety management system of claim 1,
the traffic road surface identification detection module is used for obtaining an original road surface image through the external camera, identifying a road surface object image of the original road surface image based on YOLO, constructing a road surface condition model according to the original road surface image based on the road surface object image, obtaining the road surface condition, transmitting the road surface condition to the user end system through the vehicle end communication module, and displaying the road surface condition through the vehicle comprehensive information display unit, wherein the vehicle comprehensive information further comprises the road surface condition.
10. The Internet of vehicles based assistant driving and vehicle safety management system of claim 1,
the user end system also comprises a convenient life module;
the convenient life module comprises a weather condition display unit and a current life index unit on the same day;
the user end system displays the temperature information, the somatosensory temperature information, the wind direction information and the wind power information of the area based on the weather condition display unit on the same day through the user end positioning module;
and the user end system displays the life information of the area based on the current life index unit through the user end positioning module.
CN202011476325.9A 2020-12-15 2020-12-15 Auxiliary driving and vehicle safety management system based on Internet of vehicles Active CN112477873B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011476325.9A CN112477873B (en) 2020-12-15 2020-12-15 Auxiliary driving and vehicle safety management system based on Internet of vehicles

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011476325.9A CN112477873B (en) 2020-12-15 2020-12-15 Auxiliary driving and vehicle safety management system based on Internet of vehicles

Publications (2)

Publication Number Publication Date
CN112477873A true CN112477873A (en) 2021-03-12
CN112477873B CN112477873B (en) 2024-04-30

Family

ID=74917513

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011476325.9A Active CN112477873B (en) 2020-12-15 2020-12-15 Auxiliary driving and vehicle safety management system based on Internet of vehicles

Country Status (1)

Country Link
CN (1) CN112477873B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113085888A (en) * 2021-04-21 2021-07-09 金陵科技学院 Intelligent networked automobile driving-assisting safety information detection system
CN113682112A (en) * 2021-09-15 2021-11-23 深圳地平线机器人科技有限公司 Method and device for controlling gas in vehicle
CN114512000A (en) * 2022-01-11 2022-05-17 海南省设计研究院有限公司 Intelligent comprehensive rod with continuous traffic information detection function

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130069773A1 (en) * 2010-05-12 2013-03-21 Beijing Its-Star Technology Co., Ltd. Intelligent traffic safety system based on comprehensive state detection and decision method thereof
WO2014017764A1 (en) * 2012-07-27 2014-01-30 에스케이플래닛 주식회사 Information providing method, and device and system for same
CN111724408A (en) * 2020-06-05 2020-09-29 广东海洋大学 Verification experiment method of abnormal driving behavior algorithm model based on 5G communication

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130069773A1 (en) * 2010-05-12 2013-03-21 Beijing Its-Star Technology Co., Ltd. Intelligent traffic safety system based on comprehensive state detection and decision method thereof
WO2014017764A1 (en) * 2012-07-27 2014-01-30 에스케이플래닛 주식회사 Information providing method, and device and system for same
CN111724408A (en) * 2020-06-05 2020-09-29 广东海洋大学 Verification experiment method of abnormal driving behavior algorithm model based on 5G communication

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113085888A (en) * 2021-04-21 2021-07-09 金陵科技学院 Intelligent networked automobile driving-assisting safety information detection system
CN113682112A (en) * 2021-09-15 2021-11-23 深圳地平线机器人科技有限公司 Method and device for controlling gas in vehicle
CN114512000A (en) * 2022-01-11 2022-05-17 海南省设计研究院有限公司 Intelligent comprehensive rod with continuous traffic information detection function

Also Published As

Publication number Publication date
CN112477873B (en) 2024-04-30

Similar Documents

Publication Publication Date Title
CN112477873B (en) Auxiliary driving and vehicle safety management system based on Internet of vehicles
US11776279B2 (en) Method and apparatus for providing unknown moving object detection
CN110753934B (en) System and method for actively selecting and tagging images for semantic segmentation
US20230406297A1 (en) System and method for vehicle occlusion detection
US10311312B2 (en) System and method for vehicle occlusion detection
CN112050792B (en) Image positioning method and device
CN107976200B (en) Method and system for operating a vehicle having an autonomous driving mode
CN111739344B (en) Early warning method and device and electronic equipment
CN105513395A (en) Intelligent information pushing system and method based on vehicle destination prediction
CN110795523B (en) Vehicle positioning method and device and intelligent vehicle
CN105405307A (en) Internet of vehicles service management system
EP3676147B1 (en) Autonomous driving apparatus and method for autonomous driving of a vehicle
CN103824457B (en) Based on express way driving safety information query system and the querying method of VANET
CN109005239A (en) Intelligent information exchange method, system and device for assisting vehicle travel
CN103247186A (en) Scene-based driver assistant system realizing method
CN107507428A (en) One kind is used for road traffic congestion degree detecting system
CN107907886A (en) Travel conditions recognition methods, device, storage medium and terminal device
CN113923775B (en) Method, device, equipment and storage medium for evaluating quality of positioning information
CN203149847U (en) Road condition sharing server
CN105913687A (en) Intelligent automobile driving system based on internet of things
EP3441960A1 (en) Intelligent lighting system, intelligent vehicle and auxiliary vehicle driving system and method therefor
CN206672365U (en) Road traffic management system applied to smart city
CN214523763U (en) Auxiliary driving and vehicle safety management system based on Internet of vehicles
Bhandari et al. Fullstop: A camera-assisted system for characterizing unsafe bus stopping
CN203133964U (en) Commercial vehicle monitoring terminal based on monocular range measurement

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant