CN114943940A - Method, equipment and storage medium for visually monitoring vehicles in tunnel - Google Patents
Method, equipment and storage medium for visually monitoring vehicles in tunnel Download PDFInfo
- Publication number
- CN114943940A CN114943940A CN202210881213.4A CN202210881213A CN114943940A CN 114943940 A CN114943940 A CN 114943940A CN 202210881213 A CN202210881213 A CN 202210881213A CN 114943940 A CN114943940 A CN 114943940A
- Authority
- CN
- China
- Prior art keywords
- tunnel
- vehicle
- monitored
- point cloud
- data set
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 53
- 238000012544 monitoring process Methods 0.000 title claims abstract description 35
- 230000000007 visual effect Effects 0.000 claims abstract description 28
- 238000012545 processing Methods 0.000 claims description 31
- 238000004422 calculation algorithm Methods 0.000 claims description 25
- 239000003550 marker Substances 0.000 claims description 21
- 230000009467 reduction Effects 0.000 claims description 6
- 238000004088 simulation Methods 0.000 claims description 5
- 238000000465 moulding Methods 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000007635 classification algorithm Methods 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims description 3
- 238000002372 labelling Methods 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 2
- 239000000126 substance Substances 0.000 claims 1
- 230000008447 perception Effects 0.000 abstract description 6
- 238000010586 diagram Methods 0.000 description 10
- 238000004590 computer program Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 6
- 230000006870 function Effects 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 238000010276 construction Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000007689 inspection Methods 0.000 description 3
- 230000006872 improvement Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 206010039203 Road traffic accident Diseases 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4038—Scaling the whole image or part thereof for image mosaicing, i.e. plane images composed of plane sub-images
-
- G06T5/70—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06T7/85—Stereo camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/32—Indexing scheme for image data processing or generation, in general involving image mosaicing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30232—Surveillance
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Abstract
The application discloses a method, equipment and a storage medium for visually monitoring vehicles in a tunnel. The method comprises the following steps: acquiring a tunnel image data set and a point cloud data set corresponding to a tunnel to be monitored through image acquisition equipment and a laser radar, and constructing a tunnel three-dimensional model corresponding to the tunnel to be monitored based on the tunnel image data set and the point cloud data set; associating sensor equipment in the tunnel to be monitored with the three-dimensional model of the tunnel to obtain a digital twin body corresponding to the tunnel to be monitored; receiving real-time tunnel video data, and extracting vehicle calibration information contained in the real-time tunnel video data; and generating a corresponding real-time vehicle digital object in the digital twin body corresponding to the tunnel to be monitored based on the vehicle calibration information, and displaying the digital twin body containing the real-time vehicle digital object through a large visual screen. The method realizes the visual accurate perception of the vehicles in the tunnel, so that the vehicles in the tunnel can be better visually monitored.
Description
Technical Field
The application relates to the technical field of traffic control systems, in particular to a method, equipment and a storage medium for visually monitoring vehicles in a tunnel.
Background
With the development of economy and the increasing improvement of the living standard of people, the number of vehicles is increased continuously, and the road construction is more and more perfect. The tunnel is an important way for constructing mountainous areas and cross-sea/river roads, the construction and operation mileage is gradually increased, and the tunnel is an important throat for highway and urban road traffic. The tunnel not only has large traffic flow, but also is in a semi-closed environment, has more traffic accidents than common road sections, is easy to cause secondary accidents, causes serious accident loss and traffic delay, and becomes the bottleneck of safe and smooth operation of the highway.
At present, the monitoring mode of traffic flow in the tunnel has the traditional manual inspection mode and the on-line video monitoring mode. The manual inspection not only occupies a large amount of human resources, but also has low inspection efficiency. In the video monitoring mode, due to the reasons of the camera shooting visual angle and the like, the vehicle condition in the tunnel cannot be accurately perceived. Therefore, how to realize the visual accurate perception of the vehicle in the tunnel, and better visually monitoring the vehicle in the tunnel becomes a technical problem to be solved urgently.
Disclosure of Invention
The embodiment of the application provides a method, equipment and a storage medium for visually monitoring vehicles in a tunnel, which are used for solving the following technical problems: how to realize the visual accurate perception of the vehicle in the tunnel to better carry out visual monitoring to the vehicle in the tunnel.
In a first aspect, an embodiment of the present application provides a method for visually monitoring a vehicle in a tunnel, where the method includes: respectively acquiring a tunnel image data set and a point cloud data set corresponding to a tunnel to be monitored through image acquisition equipment and a laser radar, and constructing a tunnel three-dimensional model corresponding to the tunnel to be monitored based on the tunnel image data set and the point cloud data set; associating sensor equipment in the tunnel to be monitored with the three-dimensional model of the tunnel to obtain a digital twin body corresponding to the tunnel to be monitored; receiving real-time tunnel video data, and extracting vehicle calibration information contained in the real-time tunnel video data; the method comprises the steps that tunnel video data are obtained through an intelligent camera arranged in a tunnel, and calibration information is obtained through processing of a preset identification algorithm in the intelligent camera; and generating a corresponding real-time vehicle digital object in the digital twin body corresponding to the tunnel to be monitored based on the vehicle calibration information, and displaying the digital twin body containing the real-time vehicle digital object through a large visual screen.
In an implementation manner of the present application, a tunnel three-dimensional model corresponding to a tunnel to be monitored is constructed based on a tunnel image dataset and a point cloud dataset, and specifically includes: inputting the point cloud data set into preset three-dimensional point cloud modeling software for molding so as to obtain an initial tunnel three-dimensional model corresponding to the tunnel to be monitored; processing the tunnel image data set through a preset image processing algorithm to obtain an integral visible light surface image corresponding to the tunnel to be monitored; and carrying out data superposition on the initial tunnel three-dimensional model and the whole visible light surface image so as to realize the construction of the tunnel three-dimensional model.
In one implementation of the present application, before inputting the point cloud data set into a preset three-dimensional point cloud modeling software for modeling, the method further comprises: preprocessing the point cloud data set through a preset point cloud processing algorithm to obtain a corresponding enhanced point cloud data set; determining a marker point cloud data set contained in the enhanced point cloud data set based on a preset point cloud target identification model; and determining a first marker feature point in the marker point cloud data set, and performing point cloud registration on the point cloud data set based on the marker feature point.
In an implementation manner of the present application, the point cloud data set is preprocessed through a preset point cloud processing algorithm to obtain a corresponding enhanced point cloud data set, which specifically includes: dividing the point cloud data set into two point cloud data subsets through a K nearest neighbor classification algorithm; performing projection calculation on the two point cloud data subsets through a WLOP algorithm to obtain two corresponding projection subsets; performing a preset number of iterations on the two projection subsets respectively to determine two iteration subsets corresponding to the two projection subsets; the two iterative subsets are combined to obtain an enhanced point cloud dataset.
In an implementation manner of the present application, a tunnel image dataset is processed through a preset image processing algorithm to obtain an overall visible light surface image corresponding to a tunnel to be monitored, which specifically includes: performing noise reduction processing on each tunnel image in the tunnel image data set through a nano-dimensional filtering algorithm; carrying out marker identification on each tunnel image in the tunnel image data set subjected to noise reduction treatment through a preset image target identification model; and determining second marker characteristic points corresponding to the markers in each tunnel image, and performing splicing processing on each tunnel image based on the second marker characteristic points to determine an integral visible light surface image corresponding to the tunnel to be monitored.
In an implementation manner of the present application, before receiving the real-time tunnel video data and extracting the vehicle calibration information included in the real-time tunnel video data, the method further includes: the method comprises the steps that an intelligent camera obtains initial real-time tunnel video data, and vehicle information of each vehicle contained in the initial real-time tunnel video data is determined based on a preset target recognition and tracking algorithm; wherein the vehicle information includes: vehicle attribute information, vehicle position information, and motion state information; and performing labeling processing on the initial real-time tunnel video data based on the vehicle information of each vehicle to obtain real-time tunnel video data containing vehicle calibration information.
In an implementation manner of the present application, based on vehicle calibration information, generating a corresponding real-time vehicle digital object in a digital twin corresponding to a tunnel to be monitored specifically includes: determining a corresponding vehicle digital model in a digital object model base based on vehicle attribute information in the vehicle calibration information, and adding corresponding exclusive attribute information to the vehicle digital model to obtain a corresponding vehicle digital object; the exclusive attribute information comprises license plate information; and displaying the vehicle digital object in the digital twin body corresponding to the tunnel to be monitored in real time based on the vehicle position information and the motion state information in the vehicle calibration information.
In an implementation manner of the present application, associating sensor equipment in a tunnel to be monitored with a three-dimensional model of the tunnel to obtain a digital twin corresponding to the tunnel to be monitored, specifically including: setting a corresponding environment simulation program for the tunnel three-dimensional model based on the equipment type of the sensor equipment; under the condition of triggering a visual monitoring signal of a vehicle in the tunnel, transmitting corresponding equipment data to an environment simulation program corresponding to the three-dimensional model of the tunnel in real time by using the sensor equipment; and based on the equipment data, carrying out environment simulation by an environment simulation program so as to generate a digital twin body corresponding to the tunnel to be monitored.
In a second aspect, the embodiment of the present application further provides an apparatus for visually monitoring vehicles in a tunnel, where the apparatus includes: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to: respectively acquiring a tunnel image data set and a point cloud data set corresponding to a tunnel to be monitored through image acquisition equipment and a laser radar, and constructing a tunnel three-dimensional model corresponding to the tunnel to be monitored based on the tunnel image data set and the point cloud data set; associating sensor equipment in the tunnel to be monitored with the three-dimensional model of the tunnel to obtain a digital twin body corresponding to the tunnel to be monitored; receiving real-time tunnel video data, and extracting vehicle calibration information contained in the real-time tunnel video data; the method comprises the steps that tunnel video data are obtained through an intelligent camera arranged in a tunnel, and calibration information is obtained through processing of a preset identification algorithm in the intelligent camera; and generating a corresponding real-time vehicle digital object in the digital twin body corresponding to the tunnel to be monitored based on the vehicle calibration information, and displaying the digital twin body containing the real-time vehicle digital object through a large visual screen.
In a third aspect, an embodiment of the present application further provides a non-volatile computer storage medium for visually monitoring a vehicle in a tunnel, where the non-volatile computer storage medium stores computer-executable instructions, and the computer-executable instructions are configured to: respectively acquiring a tunnel image data set and a point cloud data set corresponding to a tunnel to be monitored through image acquisition equipment and a laser radar, and constructing a tunnel three-dimensional model corresponding to the tunnel to be monitored based on the tunnel image data set and the point cloud data set; associating sensor equipment in the tunnel to be monitored with the three-dimensional model of the tunnel to obtain a digital twin body corresponding to the tunnel to be monitored; receiving real-time tunnel video data, and extracting vehicle calibration information contained in the real-time tunnel video data; the method comprises the steps that tunnel video data are obtained through an intelligent camera arranged in a tunnel, and calibration information is obtained through processing of a preset identification algorithm in the intelligent camera; and generating a corresponding real-time vehicle digital object in the digital twin body corresponding to the tunnel to be monitored based on the vehicle calibration information, and displaying the digital twin body containing the real-time vehicle digital object through a large visual screen.
According to the method, the device and the storage medium for visually monitoring the vehicles in the tunnel, the digital twin body corresponding to the tunnel to be monitored is constructed, the vehicle information in the real-time tunnel video data is extracted and then displayed in the digital twin body, and therefore the visual accurate perception of the vehicles in the tunnel is effectively achieved, and the vehicles in the tunnel can be better visually monitored.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
fig. 1 is a flowchart of a method for visually monitoring a vehicle in a tunnel according to an embodiment of the present disclosure;
fig. 2 is a schematic internal structural diagram of an apparatus for visually monitoring vehicles in a tunnel according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be described in detail and completely with reference to the following specific embodiments of the present application and the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The embodiment of the application provides a method, equipment and a storage medium for visually monitoring vehicles in a tunnel, which are used for solving the following technical problems: how to realize the visual accurate perception of the vehicle in the tunnel to better carry out visual monitoring to the vehicle in the tunnel.
The technical solutions proposed in the embodiments of the present application are described in detail below with reference to the accompanying drawings.
Fig. 1 is a flowchart of a method for visually monitoring a vehicle in a tunnel according to an embodiment of the present disclosure. As shown in fig. 1, the method for visually monitoring vehicles in a tunnel provided in the embodiment of the present application specifically includes the following steps:
In an embodiment of the application, in order to realize visual and accurate perception of vehicles in a tunnel and better visually monitor the vehicles in the tunnel, a tunnel three-dimensional model corresponding to a tunnel to be monitored needs to be constructed first.
In an embodiment of the application, a tunnel three-dimensional model corresponding to a tunnel to be monitored is constructed, and data for constructing the three-dimensional model is required to be obtained. It can be understood that when the image acquisition device acquires the image of the tunnel to be monitored, the image acquisition device cannot acquire the whole surface image of the tunnel to be monitored at one time, but acquires the image for a plurality of times to acquire a tunnel image data set containing the tunnel to be monitored at a plurality of different angles, different positions and different scales; similarly, when the laser radar collects the point cloud data of the tunnel to be monitored, the overall point cloud data of the tunnel to be monitored cannot be obtained at one time, and a point cloud data set comprising a plurality of point cloud data subsets of the tunnel to be monitored is obtained through a plurality of times of point cloud data acquisition.
In one embodiment of the application, after a tunnel image dataset and a point cloud dataset corresponding to a tunnel to be monitored are obtained, a tunnel three-dimensional model corresponding to the tunnel to be monitored is constructed based on the tunnel image dataset and the point cloud dataset.
Specifically, a point cloud data set is divided into two point cloud data subsets through a K nearest neighbor classification algorithm; performing projection calculation on the two point cloud data subsets through a WLOP algorithm to obtain two corresponding projection subsets; performing a preset number of iterations on the two projection subsets respectively to determine two iteration subsets corresponding to the two projection subsets; the two iterative subsets are combined to obtain an enhanced point cloud dataset.
Further, determining a marker point cloud data set contained in the enhanced point cloud data set based on a preset point cloud target identification model; the markers may be road surfaces, road signs, tunnel walls, etc.
Further, a first marker feature point in the marker point cloud data set is determined, and point cloud registration is performed on the point cloud data set based on the marker feature point. It can be understood that, since the point cloud data set includes a plurality of tunnel point cloud data subsets to be monitored, the tunnel point cloud data subsets to be monitored need to be registered to realize the mold so as to obtain a point cloud data set for describing the integrity of the tunnel to be monitored, and therefore, the point cloud data set needs to be subjected to point cloud registration. The specific registration method is not limited in this application and may be selected according to actual situations.
And further, inputting the point cloud data set into preset three-dimensional point cloud modeling software for molding so as to obtain an initial tunnel three-dimensional model corresponding to the tunnel to be monitored.
Further, noise reduction processing is carried out on each tunnel image in the tunnel image data set through a nano-dimensional filtering algorithm; carrying out marker identification on each tunnel image in the tunnel image data set subjected to noise reduction treatment through a preset image target identification model; and determining second marker characteristic points corresponding to the markers in each tunnel image, and performing splicing processing on each tunnel image based on the second marker characteristic points to determine an integral visible light surface image corresponding to the tunnel to be monitored.
Further, data superposition is carried out on the initial tunnel three-dimensional model and the whole visible light surface image, so that the tunnel three-dimensional model is constructed. The process of data superposition between the initial tunnel three-dimensional model and the whole visible light surface image can be immediately performed, namely, the process of adding scene information to the initial tunnel three-dimensional model to instantiate the model.
And 102, associating sensor equipment in the tunnel to be monitored with the three-dimensional model of the tunnel to obtain a digital twin body corresponding to the tunnel to be monitored.
In an embodiment of the present application, in order to realize real scene restoration of a tunnel to be monitored in a digital space to obtain a digital twin body corresponding to the tunnel to be monitored, an environment simulation of the tunnel to be monitored is also required, and therefore, in the present application, sensor equipment in the tunnel to be monitored is associated with a three-dimensional model of the tunnel to realize simulation of the tunnel to be monitored.
Specifically, setting a corresponding environment simulation program for the tunnel three-dimensional model based on the equipment type of the sensor equipment; under the condition of triggering a visual monitoring signal of a vehicle in the tunnel, transmitting corresponding equipment data to an environment simulation program corresponding to the three-dimensional model of the tunnel in real time by using the sensor equipment; and based on the equipment data, carrying out environment simulation by an environment simulation program so as to generate a digital twin body corresponding to the tunnel to be monitored.
And 103, receiving the real-time tunnel video data, and extracting vehicle calibration information contained in the real-time tunnel video data.
It can be understood that after the sensor device in the tunnel to be monitored is associated with the tunnel three-dimensional model, the intelligent camera which is preset in the tunnel is also associated with the tunnel three-dimensional model, so that real-time tunnel video data can be acquired.
In one embodiment of the application, before receiving real-time tunnel video data, an intelligent camera acquires initial real-time tunnel video data in a tunnel through a camera in sequence; then, based on a preset target recognition and tracking algorithm, determining vehicle information of each vehicle contained in the initial real-time tunnel video data; the vehicle information includes: vehicle attribute information, vehicle position information, and motion state information; the vehicle attribute information comprises model information, license plate number information, driver information and the like identified based on a face recognition technology of the vehicle, the vehicle position information comprises positioning information of the vehicle in a tunnel, lane information and the like, and the motion state information comprises vehicle speed information, motion track information and the like of the vehicle. It will be appreciated that the location of the vehicle within the tunnel can be determined by the location of the smart camera and the size information presented by the vehicle in the video.
Further, based on the vehicle information of each vehicle, the initial real-time tunnel video data is subjected to labeling processing to obtain real-time tunnel video data containing vehicle calibration information.
In an embodiment of the application, after the intelligent camera uploads the real-time tunnel video data, vehicle calibration information contained in the real-time tunnel video data can be extracted.
And 104, generating a corresponding real-time vehicle digital object in the digital twin body corresponding to the tunnel to be monitored based on the vehicle calibration information, and displaying the digital twin body containing the real-time vehicle digital object through a large visual screen.
In an embodiment of the present application, after determining vehicle calibration information included in the real-time tunnel video data, a corresponding vehicle digital model may be determined in the digital object model library based on vehicle attribute information in the vehicle calibration information, and corresponding proprietary attribute information is added to the vehicle digital model to obtain a corresponding vehicle digital object; it is understood that the unique attribute information includes license plate information, driver information, and the like.
And further, displaying the vehicle digital object in real time in the digital twin corresponding to the tunnel to be monitored based on the vehicle position information and the motion state information in the vehicle calibration information, and displaying the digital twin containing the real-time vehicle digital object through a large visual screen.
Based on the same inventive concept, the embodiment of the application also provides a device for visually monitoring the vehicles in the tunnel, and the internal structure of the device is shown in fig. 2.
Fig. 2 is a schematic internal structural diagram of an apparatus for visually monitoring vehicles in a tunnel according to an embodiment of the present application. As shown in fig. 2, the apparatus includes: a processor 201; a memory 202 having executable instructions stored thereon that, when executed, cause the processor 201 to perform a method of in-tunnel vehicle visual monitoring as described above.
In an embodiment of the present application, the processor 201 is configured to respectively acquire a tunnel image dataset and a point cloud dataset corresponding to a tunnel to be monitored through an image acquisition device and a laser radar, and construct a tunnel three-dimensional model corresponding to the tunnel to be monitored based on the tunnel image dataset and the point cloud dataset; associating sensor equipment in the tunnel to be monitored with the three-dimensional model of the tunnel to obtain a digital twin body corresponding to the tunnel to be monitored; receiving real-time tunnel video data, and extracting vehicle calibration information contained in the real-time tunnel video data; the method comprises the steps that tunnel video data are obtained through an intelligent camera arranged in a tunnel, and calibration information is obtained through processing of a preset identification algorithm in the intelligent camera; and generating a corresponding real-time vehicle digital object in the digital twin body corresponding to the tunnel to be monitored based on the vehicle calibration information, and displaying the digital twin body containing the real-time vehicle digital object through a large visual screen.
Some embodiments of the present application provide a non-transitory computer storage medium corresponding to fig. 1 for visual monitoring of vehicles in a tunnel, the medium storing computer-executable instructions configured to:
respectively acquiring a tunnel image data set and a point cloud data set corresponding to a tunnel to be monitored through image acquisition equipment and a laser radar, and constructing a tunnel three-dimensional model corresponding to the tunnel to be monitored based on the tunnel image data set and the point cloud data set;
associating sensor equipment in the tunnel to be monitored with the three-dimensional model of the tunnel to obtain a digital twin body corresponding to the tunnel to be monitored;
receiving real-time tunnel video data, and extracting vehicle calibration information contained in the real-time tunnel video data; the method comprises the steps that tunnel video data are obtained through an intelligent camera arranged in a tunnel, and calibration information is obtained through processing of a preset identification algorithm in the intelligent camera;
and generating a corresponding real-time vehicle digital object in the digital twin body corresponding to the tunnel to be monitored based on the vehicle calibration information, and displaying the digital twin body containing the real-time vehicle digital object through a large visual screen.
The embodiments in the present application are described in a progressive manner, and the same and similar parts among the embodiments can be referred to each other, and each embodiment focuses on the differences from the other embodiments. Especially, for the internet of things device and medium embodiments, since they are substantially similar to the method embodiments, the description is simple, and the relevant points can be referred to the partial description of the method embodiments.
The system and the medium provided by the embodiment of the application correspond to the method one by one, so the system and the medium also have the similar beneficial technical effects as the corresponding method, and the beneficial technical effects of the method are explained in detail above, so the beneficial technical effects of the system and the medium are not described again.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art to which the present application pertains. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims (10)
1. A method for visual monitoring of vehicles in a tunnel, the method comprising:
respectively acquiring a tunnel image data set and a point cloud data set corresponding to a tunnel to be monitored through image acquisition equipment and a laser radar, and constructing a tunnel three-dimensional model corresponding to the tunnel to be monitored based on the tunnel image data set and the point cloud data set;
associating sensor equipment in the tunnel to be monitored with the three-dimensional tunnel model to obtain a digital twin body corresponding to the tunnel to be monitored;
receiving real-time tunnel video data, and extracting vehicle calibration information contained in the real-time tunnel video data; the tunnel video data are acquired through an intelligent camera arranged in a tunnel, and the calibration information is obtained by processing a preset identification algorithm in the intelligent camera;
and generating a corresponding real-time vehicle digital object in the digital twin body corresponding to the tunnel to be monitored based on the vehicle calibration information, and displaying the digital twin body containing the real-time vehicle digital object through a large visual screen.
2. The method for visually monitoring vehicles in a tunnel according to claim 1, wherein constructing a tunnel three-dimensional model corresponding to the tunnel to be monitored based on the tunnel image dataset and the point cloud dataset specifically comprises:
inputting the point cloud data set into preset three-dimensional point cloud modeling software for molding so as to obtain an initial tunnel three-dimensional model corresponding to the tunnel to be monitored;
processing the tunnel image data set through a preset image processing algorithm to obtain an integral visible light surface image corresponding to the tunnel to be monitored;
and carrying out data superposition on the initial tunnel three-dimensional model and the integral visible light surface image so as to construct the tunnel three-dimensional model.
3. The method for visually monitoring vehicles in a tunnel according to claim 2, wherein before inputting the point cloud data set into a preset three-dimensional point cloud modeling software for molding, the method further comprises:
preprocessing the point cloud data set through a preset point cloud processing algorithm to obtain a corresponding enhanced point cloud data set;
determining a marker point cloud data set contained in the enhanced point cloud data set based on a preset point cloud target identification model;
determining a first marker feature point in the marker point cloud data set, and performing point cloud registration on the point cloud data set based on the marker feature point.
4. The method for visually monitoring the vehicles in the tunnel according to claim 3, wherein the point cloud data set is preprocessed by a preset point cloud processing algorithm to obtain a corresponding enhanced point cloud data set, and the method specifically comprises:
dividing the point cloud data set into two point cloud data subsets by a K nearest neighbor classification algorithm;
performing projection calculation on the two point cloud data subsets through a WLOP algorithm to obtain two corresponding projection subsets;
performing a preset number of iterations on the two projection subsets respectively to determine two iteration subsets corresponding to the two projection subsets;
merging the two iterative subsets to obtain the enhanced point cloud data set.
5. The method according to claim 2, wherein the tunnel image dataset is processed through a preset image processing algorithm to obtain an overall visible light surface image corresponding to the tunnel to be monitored, and specifically comprises:
performing noise reduction processing on each tunnel image in the tunnel image data set through a nano-dimensional filtering algorithm;
carrying out marker identification on each tunnel image in the tunnel image data set subjected to noise reduction treatment through a preset image target identification model;
and determining second marker feature points corresponding to the markers in each tunnel image, and performing splicing processing on each tunnel image based on the second marker feature points to determine an integral visible light surface image corresponding to the tunnel to be monitored.
6. The method for visually monitoring the vehicles in the tunnel according to claim 1, wherein before receiving the real-time tunnel video data and extracting the vehicle calibration information contained in the real-time tunnel video data, the method further comprises:
the intelligent camera acquires initial real-time tunnel video data and determines vehicle information of each vehicle contained in the initial real-time tunnel video data based on a preset target recognition and tracking algorithm; wherein the vehicle information includes: vehicle attribute information, vehicle position information, and motion state information;
and performing labeling processing on the initial real-time tunnel video data based on the vehicle information of each vehicle to obtain the real-time tunnel video data containing vehicle calibration information.
7. The method according to claim 6, wherein generating a corresponding real-time vehicle digital object in the digital twin corresponding to the tunnel to be monitored based on the vehicle calibration information specifically includes:
determining a corresponding vehicle digital model in a digital object model library based on vehicle attribute information in the vehicle calibration information, and adding corresponding exclusive attribute information to the vehicle digital model to obtain a corresponding vehicle digital object; the exclusive attribute information comprises license plate information;
and displaying the vehicle digital object in the digital twin body corresponding to the tunnel to be monitored in real time based on the vehicle position information and the motion state information in the vehicle calibration information.
8. The method for visually monitoring the vehicle in the tunnel according to claim 1, wherein associating the sensor device in the tunnel to be monitored with the three-dimensional model of the tunnel to obtain the digital twin corresponding to the tunnel to be monitored specifically comprises:
setting a corresponding environment simulation program for the tunnel three-dimensional model based on the equipment type of the sensor equipment;
under the condition of triggering a visual monitoring signal of a vehicle in the tunnel, transmitting corresponding equipment data to an environment simulation program corresponding to the three-dimensional model of the tunnel in real time by using sensor equipment;
and based on the equipment data, the environment simulation program carries out environment simulation so as to generate a digital twin body corresponding to the tunnel to be monitored.
9. An apparatus for visual monitoring of vehicles in a tunnel, the apparatus comprising:
at least one processor; and (c) a second step of,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to cause the at least one processor to:
respectively acquiring a tunnel image data set and a point cloud data set corresponding to a tunnel to be monitored through image acquisition equipment and a laser radar, and constructing a tunnel three-dimensional model corresponding to the tunnel to be monitored based on the tunnel image data set and the point cloud data set;
associating sensor equipment in the tunnel to be monitored with the tunnel three-dimensional model to obtain a digital twin body corresponding to the tunnel to be monitored;
receiving real-time tunnel video data, and extracting vehicle calibration information contained in the real-time tunnel video data; the tunnel video data are acquired through an intelligent camera arranged in a tunnel, and the calibration information is obtained by processing a preset identification algorithm in the intelligent camera;
and generating a corresponding real-time vehicle digital object in the digital twin body corresponding to the tunnel to be monitored based on the vehicle calibration information, and displaying the digital twin body containing the real-time vehicle digital object through a large visual screen.
10. A non-transitory computer storage medium for visual monitoring of vehicles in a tunnel, the medium storing computer-executable instructions configured to:
respectively acquiring a tunnel image data set and a point cloud data set corresponding to a tunnel to be monitored through image acquisition equipment and a laser radar, and constructing a tunnel three-dimensional model corresponding to the tunnel to be monitored based on the tunnel image data set and the point cloud data set;
associating sensor equipment in the tunnel to be monitored with the tunnel three-dimensional model to obtain a digital twin body corresponding to the tunnel to be monitored;
receiving real-time tunnel video data, and extracting vehicle calibration information contained in the real-time tunnel video data; the tunnel video data are acquired through an intelligent camera arranged in a tunnel, and the calibration information is obtained by processing a preset identification algorithm in the intelligent camera;
and generating a corresponding real-time vehicle digital object in the digital twin body corresponding to the tunnel to be monitored based on the vehicle calibration information, and displaying the digital twin body containing the real-time vehicle digital object through a large visual screen.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210881213.4A CN114943940A (en) | 2022-07-26 | 2022-07-26 | Method, equipment and storage medium for visually monitoring vehicles in tunnel |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210881213.4A CN114943940A (en) | 2022-07-26 | 2022-07-26 | Method, equipment and storage medium for visually monitoring vehicles in tunnel |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114943940A true CN114943940A (en) | 2022-08-26 |
Family
ID=82911550
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210881213.4A Pending CN114943940A (en) | 2022-07-26 | 2022-07-26 | Method, equipment and storage medium for visually monitoring vehicles in tunnel |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114943940A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116793245A (en) * | 2023-08-24 | 2023-09-22 | 济南瑞源智能城市开发有限公司 | Tunnel detection method, equipment and medium based on track robot |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103426165A (en) * | 2013-06-28 | 2013-12-04 | 吴立新 | Precise registration method of ground laser-point clouds and unmanned aerial vehicle image reconstruction point clouds |
| CN111145227A (en) * | 2019-12-17 | 2020-05-12 | 北京建筑大学 | Iterative integral registration method for multi-view point cloud in underground tunnel space |
| CN111275750A (en) * | 2020-01-19 | 2020-06-12 | 武汉大学 | Indoor space panoramic image generation method based on multi-sensor fusion |
| CN111445720A (en) * | 2020-04-15 | 2020-07-24 | 中国电子科技集团公司第三十八研究所 | Indoor parking method and parking system based on digital twinning |
| CN112258094A (en) * | 2020-11-27 | 2021-01-22 | 西南交通大学 | Subway train performance evaluation system construction method based on digital twinning |
| CN112862879A (en) * | 2021-02-18 | 2021-05-28 | 中国矿业大学(北京) | Method for constructing subway tunnel three-dimensional model based on TIN model |
| AU2020202540A1 (en) * | 2019-11-14 | 2021-06-03 | Latconnect 60 Pty Ltd | System and method for asset monitoring through digital twin |
| CN112990114A (en) * | 2021-04-21 | 2021-06-18 | 四川见山科技有限责任公司 | Traffic data visualization simulation method and system based on AI identification |
| CN112991742A (en) * | 2021-04-21 | 2021-06-18 | 四川见山科技有限责任公司 | Visual simulation method and system for real-time traffic data |
| CN113538863A (en) * | 2021-04-13 | 2021-10-22 | 交通运输部科学研究院 | Tunnel digital twin scene construction method and computer equipment |
| CN114399916A (en) * | 2022-01-19 | 2022-04-26 | 福建工程学院 | Virtual traffic light control reminding method for digital twin smart city traffic |
-
2022
- 2022-07-26 CN CN202210881213.4A patent/CN114943940A/en active Pending
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103426165A (en) * | 2013-06-28 | 2013-12-04 | 吴立新 | Precise registration method of ground laser-point clouds and unmanned aerial vehicle image reconstruction point clouds |
| AU2020202540A1 (en) * | 2019-11-14 | 2021-06-03 | Latconnect 60 Pty Ltd | System and method for asset monitoring through digital twin |
| CN111145227A (en) * | 2019-12-17 | 2020-05-12 | 北京建筑大学 | Iterative integral registration method for multi-view point cloud in underground tunnel space |
| CN111275750A (en) * | 2020-01-19 | 2020-06-12 | 武汉大学 | Indoor space panoramic image generation method based on multi-sensor fusion |
| CN111445720A (en) * | 2020-04-15 | 2020-07-24 | 中国电子科技集团公司第三十八研究所 | Indoor parking method and parking system based on digital twinning |
| CN112258094A (en) * | 2020-11-27 | 2021-01-22 | 西南交通大学 | Subway train performance evaluation system construction method based on digital twinning |
| CN112862879A (en) * | 2021-02-18 | 2021-05-28 | 中国矿业大学(北京) | Method for constructing subway tunnel three-dimensional model based on TIN model |
| CN113538863A (en) * | 2021-04-13 | 2021-10-22 | 交通运输部科学研究院 | Tunnel digital twin scene construction method and computer equipment |
| CN112990114A (en) * | 2021-04-21 | 2021-06-18 | 四川见山科技有限责任公司 | Traffic data visualization simulation method and system based on AI identification |
| CN112991742A (en) * | 2021-04-21 | 2021-06-18 | 四川见山科技有限责任公司 | Visual simulation method and system for real-time traffic data |
| CN114399916A (en) * | 2022-01-19 | 2022-04-26 | 福建工程学院 | Virtual traffic light control reminding method for digital twin smart city traffic |
Non-Patent Citations (1)
| Title |
|---|
| 陈华臻等: ""面向反光工件点云缺陷的点云增强算法"", 《计算机辅助设计与图形学学报》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116793245A (en) * | 2023-08-24 | 2023-09-22 | 济南瑞源智能城市开发有限公司 | Tunnel detection method, equipment and medium based on track robot |
| CN116793245B (en) * | 2023-08-24 | 2023-12-01 | 济南瑞源智能城市开发有限公司 | Tunnel detection method, equipment and medium based on track robot |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2022082476A1 (en) | Simulated traffic scene file generation method and apparatus | |
| Moers et al. | The exid dataset: A real-world trajectory dataset of highly interactive highway scenarios in germany | |
| CN111123920A (en) | Method and device for generating automatic driving simulation test scene | |
| CN113486531A (en) | Vehicle driving path planning method, equipment and system | |
| CN111797526B (en) | Simulation test scene construction method and device | |
| CN112069643A (en) | Automatic driving simulation scene generation method and device | |
| CN115357006A (en) | Intelligent networking automobile virtual and actual testing method, equipment and medium based on digital twins | |
| WO2022016311A1 (en) | Point cloud-based three-dimensional reconstruction method and apparatus, and computer device | |
| US20240017747A1 (en) | Method and system for augmenting lidar data | |
| CN111582189A (en) | Traffic signal lamp identification method and device, vehicle-mounted control terminal and motor vehicle | |
| CN111177887A (en) | Method and device for constructing simulation track data based on real driving scene | |
| Yu et al. | Autonomous vehicles digital twin: A practical paradigm for autonomous driving system development | |
| CN114943940A (en) | Method, equipment and storage medium for visually monitoring vehicles in tunnel | |
| WO2021146906A1 (en) | Test scenario simulation method and apparatus, computer device, and storage medium | |
| CN115526990A (en) | Target visualization method and device for digital twins and electronic equipment | |
| CN113867367B (en) | Processing method and device for test scene and computer program product | |
| CN113918615A (en) | Simulation-based driving experience data mining model construction method and system | |
| CN113553304A (en) | Data storage system for automatic driving | |
| JP2024019629A (en) | Prediction device, prediction method, program and vehicle control system | |
| Bruno et al. | A comparison of traffic signs detection methods in 2d and 3d images for the benefit of the navigation of autonomous vehicles | |
| Patel | A simulation environment with reduced reality gap for testing autonomous vehicles | |
| CN112556703B (en) | Method, device and system for updating high-precision map | |
| CN111881121B (en) | Automatic driving data filling method and device | |
| Gomez-Huelamo et al. | 360∘ real-time and power-efficient 3D DAMOT for autonomous driving applications | |
| CN115145253A (en) | End-to-end automatic driving method and system and training method of automatic driving model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220826 |
|
| RJ01 | Rejection of invention patent application after publication |