CN115240152A - Road property and road right vehicle-mounted identification system based on video deep learning technology - Google Patents
Road property and road right vehicle-mounted identification system based on video deep learning technology Download PDFInfo
- Publication number
- CN115240152A CN115240152A CN202210724007.2A CN202210724007A CN115240152A CN 115240152 A CN115240152 A CN 115240152A CN 202210724007 A CN202210724007 A CN 202210724007A CN 115240152 A CN115240152 A CN 115240152A
- Authority
- CN
- China
- Prior art keywords
- vehicle
- road
- deep learning
- camera
- inspection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
- B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
- B60R1/22—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
- G06V10/225—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition based on a marking or identifier characterising the area
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
- B60R2300/804—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for lane monitoring
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Abstract
The invention discloses a road property right vehicle-mounted identification system based on a video deep learning technology, which is characterized in that hardware equipment mainly comprises a vehicle-mounted camera, vehicle-mounted analysis equipment and a vehicle-mounted display, wherein the vehicle-mounted camera is used for capturing and collecting video images in a vehicle driving area, the collected video images are transmitted to the vehicle-mounted analysis equipment through a data line or a network cable, the vehicle-mounted analysis equipment shoots a picture according to the current driving speed of a vehicle and the shooting frequency set in firmware, the picture content is subjected to image structured detection based on artificial intelligent deep learning, different key areas of a road surface, a guardrail and an information board in the picture are identified, and the identified result is uploaded to a cloud management center through a wireless network.
Description
Technical Field
The invention relates to the technical field of road property road right identification, in particular to a vehicle-mounted road property road right identification system based on a video deep learning technology.
Background
At present, for the inspection of highway infrastructure equipment, field workers are mainly used for driving inspection vehicles to inspect the highway at the outermost lane of the highway at a certain average speed, and in the inspection process, the workers in the vehicles observe the area where the field workers are located through eyes to visually inspect infrastructure equipment of the highway. The detected objects comprise the state of the road surface where the maintenance vehicle is located, the quality of road marking, guardrails on two sides of the road surface, and an information board on the side and above the road; the in-process of patrolling and examining needs the staff to look at six ears and listen to eight directions with the eyes, carry out long distance and patrol and examine the in-process, personnel's both eyes can produce tiredly, and can appear observing the phenomenon of careless omission in the in-process of marcing unavoidably, different patrolling and examining personnel all judge the experience according to the range estimation of oneself and patrol and examine various facilities on the highway and judge, the phenomenon that the standard is nonuniform also can appear, thereby lead to different patrolling and examining personnel report different results of patrolling and examining on same road surface, perhaps when patrolling and examining the vehicle and promoting the speed of travel, the phenomenon of missing the inspection can appear to the observer.
Disclosure of Invention
The invention aims to provide a road property and road right vehicle-mounted identification system based on a video deep learning technology, and aims to solve the problems in the background technology.
In order to achieve the purpose, the invention provides the following technical scheme: a road property road right vehicle-mounted identification system based on a video deep learning technology comprises a vehicle-mounted camera, vehicle-mounted analysis equipment and a vehicle-mounted display;
the number of the vehicle-mounted cameras is 4, the vehicle-mounted cameras are arranged at the middle rear part of the roof of the inspection vehicle and are used for respectively acquiring video images in the front direction, the rear direction, the left direction and the right direction of the inspection vehicle;
the vehicle-mounted analysis equipment and the vehicle-mounted display are arranged inside the inspection vehicle, the vehicle-mounted analysis equipment acquires the real-time moving speed of the inspection vehicle, the vehicle-mounted camera is controlled to shoot according to the set speed and the phase shooting frame rate rule according to the real-time moving speed of the inspection vehicle, structured image recognition is carried out, and the vehicle-mounted display is used for realizing man-machine interaction between the vehicle-mounted analysis equipment and inspection personnel.
Furthermore, the vehicle-mounted analysis equipment comprises an input interface module, an information storage module, an information processing module and an output interface module, wherein the information processing module is respectively in data connection with the input interface module, the information storage module, the output interface module and an external output module, the input interface module is in data connection with the vehicle-mounted camera, and the output interface module is in data connection with the vehicle-mounted display.
Furthermore, the vehicle-mounted analysis equipment adopts a machine vision technology based on deep learning, and realizes the detection of the image shot by the vehicle-mounted camera by the local information analysis terminal through material acquisition, material calibration, algorithm training, algorithm model development, model optimization and packaging of the image.
Further, the running speed of the inspection vehicle is in the range of 40-80 Km/h, and the vehicle-mounted analysis equipment controls 4 vehicle-mounted cameras erected at the middle and rear part of the roof to shoot at the shooting frequency of 3-8 m/piece.
Further, the vehicle-mounted camera at the front part is used for collecting and identifying road side identification signs, road surface unspecified throwing objects and road side slope deformation; the vehicle-mounted cameras on the left side and the right side are used for collecting roadside guard plates and slope deformation; the vehicle-mounted camera at the rear part is used for collecting pavement video images in a range of 3 x 3m behind the mobile inspection vehicle and identifying pavement diseases.
Further, the vehicle-mounted analysis equipment further comprises an external output module, the external output module is in data connection with the information processing module, and the external output module is connected with the data switch through a wireless network.
Further, the vehicle-mounted analysis equipment obtains the real-time moving speed of the inspection vehicle through GPS satellite positioning.
Compared with the prior art, the invention has the beneficial effects that: the method is characterized in that a set of hardware equipment is installed outside an inspection vehicle on the expressway, the set of hardware equipment mainly comprises a vehicle-mounted camera, vehicle-mounted analysis equipment and a vehicle-mounted display, the vehicle-mounted camera is used for capturing and collecting video images in a vehicle driving area, the collected video images are transmitted to the vehicle-mounted analysis equipment through a data line or a network cable, the vehicle-mounted analysis equipment is used for shooting and shooting a picture every 3m of the vehicle under the condition of 40km/h to 80km/h of vehicle speed according to the current driving speed of the vehicle and the shooting frequency set in firmware, the picture content is subjected to image structural detection based on artificial intelligent deep learning, different key areas of a road surface, a guardrail and an information board in the picture are identified, and the identified result is uploaded to a cloud management center through a wireless network.
1) And the image recognition system assists in inspection, so that the omission rate caused by fatigue in manual visual inspection is reduced.
2) The image structured algorithm is adopted for inspection through an artificial intelligence technology, the inspection operation standard is unified, and the influence effect caused by human factors is avoided.
3) And the detection frequency is related to the running speed of the vehicle, so that the quality of detection at different vehicle speeds is kept consistent.
4) The detection target is covered comprehensively, and the phenomenon of artificial detection omission is reduced.
5) The cost is reduced, the efficiency is improved, the labor is saved, a plurality of people are needed for one-time inspection, and the personnel investment can be reduced by using the invention.
Drawings
FIG. 1 is a schematic diagram of the system of the present invention;
FIG. 2 is a schematic view of the vehicle-mounted camera and inspection vehicle of the present invention;
FIG. 3 is a flow chart of the detection according to the first embodiment of the present invention;
FIG. 4 is a flowchart illustrating the operation of the vehicle-mounted analysis apparatus according to the present invention;
FIG. 5 is a flow chart of the detection of the road surface scattering objects in the present invention;
FIG. 6 is a flow chart of the detection of highway infrastructure according to the present invention;
FIG. 7 is a schematic diagram of the horizontal shooting range of the front-end camera according to the present invention;
FIG. 8 is a schematic diagram of a vertical shooting range of the front-end camera according to the present invention;
FIG. 9 is a schematic diagram of the horizontal shooting range of the right-hand camera according to the present invention;
FIG. 10 is a schematic diagram of a vertical shooting range of a right-hand camera according to the present invention;
FIG. 11 is a schematic diagram of the horizontal shooting range of the left-end camera according to the present invention;
FIG. 12 is a schematic diagram of the vertical shooting range of the left-end camera according to the present invention;
FIG. 13 is a flowchart illustrating a second exemplary embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1-13, the present invention provides a technical solution: a road property and road right vehicle-mounted identification system based on a video deep learning technology comprises a vehicle-mounted camera, vehicle-mounted analysis equipment and a vehicle-mounted display; the number of the vehicle-mounted cameras is 4, the vehicle-mounted cameras are arranged at the middle rear part of the roof of the inspection vehicle and are used for respectively acquiring video images in the front direction, the rear direction, the left direction and the right direction of the inspection vehicle; the vehicle-mounted analysis equipment and the vehicle-mounted display are arranged inside the inspection vehicle, the vehicle-mounted analysis equipment acquires the real-time moving speed of the inspection vehicle, controls the vehicle-mounted camera to shoot according to the set speed and the phase shooting frame rate rule according to the real-time moving speed of the inspection vehicle, and conducts image structured recognition, and the vehicle-mounted display is used for achieving human-computer interaction between the vehicle-mounted analysis equipment and inspection personnel. The vehicle-mounted analysis equipment comprises an input interface module, an information storage module, an information processing module and an output interface module, wherein the information processing module is respectively in data connection with the input interface module, the information storage module, the output interface module and the external output module, the input interface module is in data connection with the vehicle-mounted camera, and the output interface module is in data connection with the vehicle-mounted display. The vehicle-mounted analysis equipment adopts a machine vision technology based on deep learning, the materials are collected by the aid of pictures, the materials are calibrated, algorithm training is performed, algorithm models are developed, models are optimized and packaged, the local information analysis terminal is used for detecting images shot by a vehicle-mounted camera, the materials are collected, the method is a precondition of deep learning training, and the artificial intelligence deep learning method is used for repeatedly learning and training a large number of picture materials to finally achieve the purpose of recognition. Therefore, it is necessary to shoot a picture meeting the future detection requirement first according to the requirement. According to the requirement of inspection, parameters of the camera are determined, wherein the parameters mainly comprise depth of Field (FOV) and field angle (fov), and the FOV is divided into horizontal field angle and vertical field angle. The depth of field parameter influences the distance from the camera to the clear image, the horizontal field angle influences the transverse range of the camera, and the vertical field angle influences the longitudinal range of the camera. It is assumed that the vehicle-mounted camera mounted on the present system should meet the following conditions based on the following acquisition requirements and road lane conditions. The material calibration refers to the premise that the image content of the collected material is marked according to categories so that the machine can recognize the images. In this scheme, the content in the captured image in the picture is marked. The tagged objects include: traffic participants, vehicles of various models and styles, such as minibuses, trucks, motorcycles and the like. Traffic participants, pedestrians. Traffic facilities equipment, such as information boards, road markings, door frames, distribution boxes, radars, cameras and the like. Other articles, such as cone barrels for inspection maintenance and the like, manually calibrate 60000000 pictures (20000000 pictures are shot by a front camera, 20000000 pictures are shot by the left side and the right side respectively, each vehicle speed is 1000000, 3 cameras are 3000000, 20 vehicle speeds are counted, and 60000000 pictures are counted) by special artificial intelligence training software, and traffic participants, traffic road facilities, other articles and the like are marked by frames with different meanings, different colors or different shapes. For example, traffic participants 'vehicle types are marked with green frames, traffic participants' pedestrians are marked with red frames, road facilities are marked with blue frames, and other objects are marked with black frames. The traffic participant not only marks the self contour, but also marks the self contour and the shadow contour; the traffic road facilities also calibrate the profile of the traffic road facilities and the profile of the traffic road facilities together with the shadow. The algorithm training is to determine a detection object and an ignorable object after calibrating a material, for example, in the current material, an image of a blue frame is an image of a road facility, namely, an image of a key object green frame to be recognized and detected by the system, and is a vehicle traffic participant, namely, an image which can be ignored by the system. Rules, called models, are generated that identify the highway pavement and the traffic facilities on both sides. Algorithmic models typically require constant optimization. For example, when an object that does not conform to the characteristics of the traffic participants and does not belong to the characteristics of other objects is detected on the road surface, a report is made, and after the report, the object is found to be a scattered object or discarded garbage during field detection, such characteristics to be identified can be added into the algorithm. Meanwhile, if one type of article is mistakenly identified as another type of article, the algorithm model also needs supplementary material and additional training, such as scattered objects or discarded garbage shown in the following figures. The running speed of the mobile inspection vehicle is in the range of 40-80 Km/h, and the vehicle-mounted analysis equipment controls 4 vehicle-mounted cameras erected at the middle rear part of the roof to shoot at the shooting frequency of 3-8 m/piece. The vehicle-mounted camera at the front part is used for collecting and identifying road side identification signs, road surface unspecified sprinkled objects and road side slope deformation; the left and right vehicle-mounted cameras are used for collecting roadside guard plates and slope deformation; the vehicle-mounted camera at the rear part is used for collecting pavement video images in the range of 3 x 3m behind the mobile inspection vehicle and identifying pavement diseases. In the scheme, the condition that an image in the range of 20 meters ahead is required to be shot is assumed, and the inspection road section is a one-way 4-lane; in the actual inspection process, the inspection vehicle should run in the center of the outermost low-speed lane, the width of the highway lane is 3.75 meters, and 3.8 meters is taken as a whole, then, the length of the right-angled side of the vehicle is 3.8 × 3+1.9 (the inspection vehicle occupies a half lane) =13.3 meters, i.e. b =13.3 meters in the lower drawing, while the right-angled side of the running direction of the vehicle is 20 meters according to the shooting requirement, i.e. a =20 meters, the included angle between the right-angled side and the oblique side is r, tanr =13.3/20, r =33.6 °, and the angle of r is the angle at which the inspection vehicle can shoot the leftmost overtaking lane in the rightmost lane (non-emergency lane), and then the horizontal fov of the front-end camera of the vehicle should be r 2, i.e. not less than 68 °. In principle, the camera is only responsible for shooting the road surface condition of the road, and the detection work of scenes on two sides of the road does not belong to the detection range of the front-end camera. In this scheme, supposing that the image of 20 meters scope ahead was required to be shot, the portal height is 8 meters, and the information board is installed as portal top and below respectively, but only shoots portal below information board information under the long-distance condition, and the vehicle-mounted camera is installed in the top height of patrolling and examining the vehicle and is 2 meters, then: a =20 m, b = gantry height-vehicle height =8 m-2 m =6 m, tanr =6/20, r =17 °, then the lens at the front end of the vehicle, vertical fov, should be r × 2, i.e. not less than 37 °. The conclusions obtained by the above calculation are as follows: the front-end camera parameters should take: the camera with the depth of field of not less than 20 meters, the horizontal fov of not less than 68 degrees, the vertical fov of not less than 34 degrees and the frame rate of not less than 15fps is arranged on the vehicle (the installation position comprises but is not limited to the front part of the vehicle, a front machine cover and the top of the vehicle), the image in front of the vehicle can achieve the shooting effect shown in the following figures, the horizontal direction can cover the guardrails on two sides, and the vertical range can cover the information of the information board arranged below the door frame. In the scheme, it is assumed that an image of a roadside in a range of 10 meters is required to be captured, the length of a driven inspection vehicle is 5 meters, the width of an emergency lane of a highway is 3.75 meters, and 3.8 meters are rounded, then b =3.8+1.9 meters (the inspection vehicle occupies a half lane), a =5 meters, tanr = a/b =5/5.7=0.88, then r =41 ° of the right-side camera of the vehicle, and the horizontal fov = r 2=82 °. In the scheme, assuming that an image of a roadside range of 2 meters is required to be captured, and the height of a driven patrol vehicle is 2 meters, the width of an emergency lane on a highway is 3.75 meters, and 3.8 meters is taken as a whole, b =3.8+1.9 meters (the patrol vehicle occupies a half lane), a =2 meters, and tanr = a/b =2/5.7=0.35, then r =20 ° indicates that the vertical fov of the camera on the right side of the vehicle is 20 °. Right camera parameters, should be taken: the camera with the depth of field of not less than 3.8 meters, the horizontal fov of not less than 82 degrees, the vertical fov of not less than 20 degrees and the frame rate of not less than 15fps is installed on the vehicle (the installation position includes but is not limited to the front part or the side surface of the vehicle), and the shooting range of the right side of the vehicle can achieve the shooting effect of transversely covering 10 meters and longitudinally covering 2 meters. In the scheme, it is assumed that an image of a roadside range of 10 meters is required to be captured, the length of a driven inspection vehicle is 5 meters, the width of an emergency lane of a highway is 3.75 meters, and 3.8 meters is taken, then b =3.8+ 3+1.9 meters (the inspection vehicle occupies a half lane), a =5 meters, and tanr = a/b =5/13.3=0.36, then r =21 ° is the left camera of the vehicle, and the level fov = r + 2=42 °. In the scheme, it is assumed that an image of the roadside in the range of 2 meters is required to be captured, the height of a driven inspection vehicle is 2 meters, the width of an emergency lane of the expressway is 3.75 meters, and 3.8 meters are taken as a whole, so that b =3.8+ 3+1.9 meters (the inspection vehicle occupies half lane), a =2 meters, and tanr = a/b =2/13.3=0.15, and then r =9 ° means that the vertical fov of the camera on the right side of the vehicle is 9 °. Right camera parameters, should be taken: the camera with the depth of field of not less than 14 meters, the horizontal fov of not less than 42 degrees, the vertical fov of not less than 9 degrees and the frame rate of not less than 15fps is installed on the vehicle (the installation position includes but is not limited to the front part of the vehicle or the side surface of the vehicle), and the shooting range of the left side of the vehicle can achieve the shooting effect of transversely covering 10 meters and longitudinally covering 2 meters. The method comprises the steps of respectively carrying out 40km/h,41km/h and 42km/h \8230; \ 8230and analogizing in sequence, driving at the vehicle speed gradually increased by taking 1km/h as a unit under the current vehicle speed of 40km/h to 80km/h, starting the shooting function of the vehicle-mounted camera in the driving process, simultaneously starting the shooting functions of three cameras forwards, leftwards and rightwards in the driving process of each vehicle speed, starting the corresponding frame rate to shoot according to the relation between the vehicle speed and the shooting frame rate of the camera, wherein in the driving process of each vehicle speed, the number of pictures shot by each camera is not less than 1000000, and the pictures are used as basic materials for artificial intelligent deep learning. The shooting process needs various traffic environments or marks such as a traffic information board, a portal frame, a tunnel and an emergency lane and the like in the acquisition area through which the image acquisition vehicle passes, the shooting process is a road environment for formal traffic, and the shot picture content comprises various traffic environments or marks such as the traffic information board, the portal frame, the tunnel and the emergency lane and various traffic participants including a passenger car, a truck, a motorcycle and pedestrians. In addition, an image is collected, and the image is an image which has no traffic participants on the highway and only has road surface information and road marks. The vehicle-mounted analysis equipment further comprises an external output module, the external output module is in data connection with the information processing module, and the external output module is connected with the data switch through a wireless network. The vehicle-mounted analysis equipment acquires the real-time moving speed of the inspection vehicle through GPS satellite positioning, the storage module is divided into a program storage area and a data storage area, GPS and information transmitted by a camera are stored in the data storage area, the vehicle speed is acquired through GPS information, images shot by the shooting frequency camera of the vehicle-mounted camera are controlled through the vehicle speed, retrieval is carried out through a local artificial intelligence program, and the identified abnormal problems are reported. And outputting the data to a cloud management platform through a wireless network and outputting the data to a vehicle-mounted display.
And determining the shooting frequency of the camera in a specified vehicle speed interval range, and determining the working frequency of the camera in different vehicle speed interval ranges, wherein in order to avoid frame loss, the corresponding frame rate is totally evolved into an integer frame rate. The relationship between the vehicle speed and the camera frame rate is shown in the following table 1:
| speed (km/h) | Distance traveled (h/m) | Distance traveled (s/m) | Multiple of 3m | Corresponding frame rate |
| 40-50 | 40000-50000 | 11～13 | 3.7～4.3 | 5 |
| 50-60 | 50000-60000 | 13～17 | 4.3～5.7 | 6 |
| 60-70 | 60000-70000 | 17～19 | 5.7～6.3 | 7 |
| 70-80 | 70000-80000 | 19～22 | 6.3～7.3 | 8 |
| 80-90 | 80000-90000 | 22～25 | 7.3～7.8 | Irrespective of whether |
TABLE 1
The frame rate formula is: v 1000/3600/3
When the requirement is as follows: the vehicle speed ranges from 40km/h to 80km/h, 1 image is shot by 3 meters, and then:
the vehicle speed ranges from 40km/h to 50km/h, and a camera needs to take 5 pictures per second, namely 5 frames;
the vehicle speed ranges from 50km/h to 60km/h, and a camera needs to shoot 6 images per second, namely 6 frames;
the vehicle speed ranges from 60km/h to 70km/h, and a camera needs to shoot 7 images per second, namely 7 frames;
the vehicle speed ranges from 70km/h to 80km/h, and a camera needs to shoot 8 images per second, namely 8 frames;
in the camera lens model selection, a camera lens of 15 frames may be selected;
when the sensor selects the type, the frame rate can be supported to be greater than or equal to 15 frames of sensor.
In the firmware development process, in the case of a frame rate supported by the sensor, the 5 frame rate, the 6 frame rate, the 7 frame rate, and the 8 frame rate can be output by controlling the register of the sensor through the firmware.
The working principle is as follows: the invention is characterized in that a set of hardware equipment is arranged outside an expressway inspection vehicle, the hardware equipment mainly comprises a vehicle-mounted camera, the architecture comprises but is not limited to an arm architecture, the hardware equipment can also comprise an x86 architecture, vehicle-mounted analysis equipment and a vehicle-mounted display, the vehicle-mounted camera is used for capturing and collecting video images in a vehicle driving area, the collected video images are transmitted to the vehicle-mounted analysis equipment through a data line or a network cable, the vehicle-mounted analysis equipment shoots and shoots a picture every 3m when the vehicle moves under the condition of 40km/h to 80km/h of vehicle speed according to the current driving speed of the vehicle and shooting frequency set in firmware, the picture content is subjected to image structural detection based on artificial intelligent deep learning, road surfaces, guardrails and different key areas in the images are identified, and the identified result is uploaded to a cloud management center through a wireless network. A mobile inspection vehicle with the running speed of 40-80 Km/h is adopted, 4 high-definition cameras erected at the middle rear part of the roof are controlled through a vehicle-mounted processing terminal, and video images in the front direction, the rear direction, the left direction and the right direction of the vehicle are respectively collected at the shooting frequency of each camera per 3 m. The front camera is responsible for collecting and identifying roadside identification signs, road surface unspecified thrown objects (garbage) and roadside slope deformation; the left camera and the right camera are responsible for collecting roadside guard plates and slope deformation (the guard plate deformation is not required at this time); the rear camera collects road surface video images within the range of 3 × 3m behind the vehicle and is responsible for identifying road surface diseases. Algorithmic models typically require constant optimization. For example, if an object is detected on the road surface that does not match the characteristics of the traffic participants, and does not belong to the characteristics of other objects, a report is made, and the object is found to be a spill or discarded garbage during field detection after the report, such characteristics to be identified can be added to the algorithm. Also, if one type of article is mistakenly identified as another type of article, the algorithm model also requires supplementary material to increase training, such as scattered or discarded garbage. Because the objects appearing on the road surface are different, it is difficult to effectively and accurately identify all the objects appearing on the road surface by adopting a technical means, and a laser radar scanning detection mode is introduced. The working principle of the laser radar is that the outline of an object in a scanning range is built, and then the object with uniform characteristics, such as a motor vehicle, a non-motor vehicle, a lamp post, an information board, a door frame, a person and the like, can be perceived and identified by the laser radar through a training means of machine learning. The object that laser radar scanning detected in actual vehicle engineering contains but is not limited to non-motor, lamp pole, information board, portal, personnel's class possesses the object of unified characteristic, still does not possess the regular shape if scattering thing etc. simultaneously, can not adopt the object of artificial intelligence deep learning technique discernment, so among laser radar's the work, the target zone that scans in the scanning process is divided into: the object needs to be detected, and the object does not need to be detected or identified.
Example two: the implementation method is a new technical implementation method provided on the basis of the first embodiment. The implementation methods for frame rate requirement, frame rate control and image detection of the vehicle-mounted camera related to this method are not described repeatedly in this segment of implementation method 2, and this section mainly describes the technology applied to the laser radar that is different from the new laser radar in the first embodiment.
Because the objects appearing on the road surface are different, it is difficult to effectively and accurately identify all the objects appearing on the road surface by adopting a technical means, and a laser radar scanning detection mode is introduced. The working principle of the laser radar is that the outline of an object in a scanning range is constructed, and then the object with uniform characteristics, such as a motor vehicle, a non-motor vehicle, a lamp post, an information board, a door frame, a person and the like, can be perceived and identified by the laser radar through a training means of machine learning. The object that laser radar scanned and detected in actual vehicle engineering contains but is not limited to non-motor vehicle, lamp pole, information board, portal, personnel etc. and possesses the object of unified characteristic, still do not possess regular shape if scattering thing etc. simultaneously, can not adopt the object of artificial intelligence deep learning technique discernment, so laser radar's work in, the target zone that scans in-process is divided into: the object needs to be detected, and the object does not need to be detected or identified.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (7)
1. The utility model provides a highway road property right of way vehicle-mounted identification system based on video deep learning technique which characterized in that: the system comprises a vehicle-mounted camera, a vehicle-mounted analysis device and a vehicle-mounted display;
the number of the vehicle-mounted cameras is 4, the vehicle-mounted cameras are arranged at the middle rear part of the roof of the inspection vehicle and are used for respectively acquiring video images in the front direction, the rear direction, the left direction and the right direction of the inspection vehicle;
the vehicle-mounted analysis equipment and the vehicle-mounted display are arranged inside the inspection vehicle, the vehicle-mounted analysis equipment acquires the real-time moving speed of the inspection vehicle, controls the vehicle-mounted camera to shoot according to the set speed and the phase shooting frame rate rule according to the real-time moving speed of the inspection vehicle, conducts image structural identification, and the vehicle-mounted display is used for achieving human-computer interaction between the vehicle-mounted analysis equipment and inspection personnel.
2. The road property road right vehicle-mounted identification system based on the video deep learning technology as claimed in claim 1, wherein: the vehicle-mounted analysis equipment comprises an input interface module, an information storage module, an information processing module and an output interface module, wherein the information processing module is respectively in data connection with the input interface module, the information storage module, the output interface module and an external output module, the input interface module is in data connection with the vehicle-mounted camera, and the output interface module is in data connection with the vehicle-mounted display.
3. The road property road right vehicle-mounted identification system based on the video deep learning technology as claimed in claim 2, wherein: the vehicle-mounted analysis equipment adopts a machine vision technology based on deep learning, and realizes that the local information analysis terminal detects images shot by the vehicle-mounted camera by carrying out material acquisition, material calibration, algorithm training, algorithm model development, model optimization and encapsulation on the images.
4. The road property road right vehicle-mounted identification system based on the video deep learning technology as claimed in claim 1, wherein: the running speed of the inspection vehicle is in the range of 40-80 Km/h, and the vehicle-mounted analysis equipment controls 4 vehicle-mounted cameras erected at the middle rear part of the roof to shoot at the shooting frequency of 3-8 meters per vehicle.
5. The road property road right vehicle-mounted identification system based on the video deep learning technology as claimed in claim 4, wherein: the vehicle-mounted camera at the front part is used for collecting and identifying road side identification signs, road surface unspecified throwing objects and road side slope deformation; the vehicle-mounted cameras on the left side and the right side are used for collecting deformation of the roadside guard plate and the side slope; the vehicle-mounted camera at the rear part is used for collecting pavement video images in the range of 3 x 3m behind the mobile inspection vehicle and identifying pavement diseases.
6. The road property road right vehicle-mounted identification system based on the video deep learning technology as claimed in claim 2, wherein: the vehicle-mounted analysis equipment further comprises an external output module, the external output module is in data connection with the information processing module, and the external output module is connected with the data switch through a wireless network.
7. The road property road right vehicle-mounted identification system based on the video deep learning technology as claimed in claim 1, wherein: the vehicle-mounted analysis equipment obtains the real-time moving speed of the inspection vehicle through GPS satellite positioning.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210724007.2A CN115240152A (en) | 2022-06-23 | 2022-06-23 | Road property and road right vehicle-mounted identification system based on video deep learning technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210724007.2A CN115240152A (en) | 2022-06-23 | 2022-06-23 | Road property and road right vehicle-mounted identification system based on video deep learning technology |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115240152A true CN115240152A (en) | 2022-10-25 |
Family
ID=83668587
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210724007.2A Pending CN115240152A (en) | 2022-06-23 | 2022-06-23 | Road property and road right vehicle-mounted identification system based on video deep learning technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115240152A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117395378A (en) * | 2023-12-07 | 2024-01-12 | 北京道仪数慧科技有限公司 | Road product acquisition method and acquisition system |
-
2022
- 2022-06-23 CN CN202210724007.2A patent/CN115240152A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117395378A (en) * | 2023-12-07 | 2024-01-12 | 北京道仪数慧科技有限公司 | Road product acquisition method and acquisition system |
| CN117395378B (en) * | 2023-12-07 | 2024-04-09 | 北京道仪数慧科技有限公司 | Road product acquisition method and acquisition system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN202071799U (en) | Intelligent automobile safety warning device | |
| CA2747337C (en) | Multiple object speed tracking system | |
| CN109299674B (en) | Tunnel illegal lane change detection method based on car lamp | |
| CN101510356B (en) | Video detection system and data processing device thereof, video detection method | |
| CN203134149U (en) | Vehicle auxiliary driving system based on different wave band imaging fusion image processing | |
| CN103824037B (en) | Vehicle anti-tracking alarm device | |
| CN105512623A (en) | Foggy-day driving visual enhancement and visibility early warning system and method based on multiple sensors | |
| CN110097762B (en) | Road video image low visibility scale estimation method and system | |
| CN109910872B (en) | parking scene extraction system and method based on natural driving data | |
| CN104506800A (en) | Scene synthesis and comprehensive monitoring method and device for electronic police cameras in multiple directions | |
| CN103646544B (en) | Based on the vehicle behavioural analysis recognition methods of The Cloud Terrace and camera apparatus | |
| CN108830880B (en) | Video visibility detection early warning method and system suitable for expressway | |
| CN107145825A (en) | Ground level fitting, camera calibration method and system, car-mounted terminal | |
| CN102303563B (en) | System and method for prewarning front vehicle collision | |
| CN104134350A (en) | Intelligent dome camera system for traffic violation behavior recognition | |
| CN111311942A (en) | Road condition display method and system based on V2X technology, V2X terminal and V2X server | |
| CN111144301A (en) | Road pavement defect quick early warning device based on degree of depth learning | |
| CN104658261A (en) | Detection and monitoring system of cart lane occupation on expressway | |
| CN115240152A (en) | Road property and road right vehicle-mounted identification system based on video deep learning technology | |
| CN114329074B (en) | Traffic energy efficiency detection method and system for ramp road section | |
| CN106803073B (en) | Auxiliary driving system and method based on stereoscopic vision target | |
| CN102364560B (en) | Traffic sign convenient for electronic identification and method for identifying traffic sign | |
| CN106340031A (en) | Method and device for detecting moving object | |
| CN114863695B (en) | Overproof vehicle detection system and method based on vehicle-mounted laser and camera | |
| CN111126237A (en) | Safe vehicle distance detection method based on machine vision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |