CN114120642B - Road traffic flow three-dimensional reconstruction method, computer equipment and storage medium - Google Patents
Road traffic flow three-dimensional reconstruction method, computer equipment and storage medium Download PDFInfo
- Publication number
- CN114120642B CN114120642B CN202111393942.7A CN202111393942A CN114120642B CN 114120642 B CN114120642 B CN 114120642B CN 202111393942 A CN202111393942 A CN 202111393942A CN 114120642 B CN114120642 B CN 114120642B
- Authority
- CN
- China
- Prior art keywords
- vehicle
- road
- dimensional
- model
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
- G08G1/0125—Traffic data processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
- G08G1/0137—Measuring and analyzing of parameters relative to traffic conditions for specific applications
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/017—Detecting movement of traffic to be counted or controlled identifying vehicles
- G08G1/0175—Detecting movement of traffic to be counted or controlled identifying vehicles by photographing vehicles, e.g. when violating traffic rules
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/04—Detecting movement of traffic to be counted or controlled using optical or ultrasonic detectors
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/16—Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
- H04L69/169—Special adaptations of TCP, UDP or IP for interworking of IP based networks with other networks
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Analytical Chemistry (AREA)
- Chemical & Material Sciences (AREA)
- Software Systems (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Computer Security & Cryptography (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention belongs to the technical field of intelligent traffic, and particularly relates to a road traffic flow three-dimensional reconstruction method, computer equipment and a storage medium. The method comprises the following steps: the server acquires data of the millimeter wave radar and the camera; estimating the actual position of the vehicle on the road and the vehicle model information by utilizing a neural network; the server encodes the actual positions and vehicle model information of all vehicles on the road into data in a JSON character string format, and pushes the encoded data to a plurality of clients; the client establishes a vehicle model database, and the client matches a three-dimensional model corresponding to the vehicle according to the decoded data; and the client side reconstructs the three-dimensional scene of the traffic flow on the road and displays the scene. The invention reduces the data transmission quantity between the server and the client, so that the client can reconstruct the traffic flow three-dimensional model of the road according to the received text data, and the traffic flow information can be shared to a plurality of clients, thereby being beneficial to the development of intelligent traffic.
Description
Technical Field
The invention belongs to the technical field of intelligent traffic, and particularly relates to a road traffic flow three-dimensional reconstruction method, computer equipment and a storage medium.
Background
With the progress of urbanization, the problem of traffic congestion becomes more and more prominent. The traffic information is the visual reflection of the road condition, the traffic information is mastered in time, the method has important significance for reasonably configuring traffic resources and relieving congestion, and meanwhile, data support is provided for the development of intelligent traffic. In the current traffic monitoring scene, the camera is mainly used for detecting vehicles running on a road in real time, obtaining the license plate number of the vehicle, judging whether the vehicle violates rules or not, and measuring the speed of the vehicle by using a radar.
The current monitoring system can only display the detected license plate number, the detected vehicle speed and the detected vehicle model in a text mode, the actual traffic flow of a road cannot be modeled, and a monitoring station shares road images with other terminals in a video flow mode, so that huge communication bandwidth is occupied, and unnecessary transmission overhead is caused. The reason for this is that:
1. the actual position information of the vehicle on the road is not utilized.
Most of the existing monitoring stations only utilize the model, license plate and speed information of vehicles, but lack the utilization of the actual position information of the vehicles.
2. A three-dimensional reconstruction system is lacking.
Because a system capable of reconstructing the road traffic flow condition is lacked, the road monitoring station is mainly shared to other user terminals in a video stream mode at present, and the transmission of the video stream brings huge communication overhead. According to the method and the system, the monitored vehicle information is encoded into a text format at the server side, the text format is shared to other terminals under the condition that smaller communication resources are used, and then the three-dimensional reconstruction system is established at the terminals. And the terminal decodes the data sent by the server and restores the actual condition of the road traffic flow by using the three-dimensional reconstruction system.
Disclosure of Invention
In order to solve the problem of huge communication overhead caused by sharing road traffic flow images by using a video flow mode in the current traffic detection system, the invention provides a method, computer equipment and a storage medium for transmitting and three-dimensional reconstruction of road traffic flow information.
The invention is realized by adopting the following technical scheme:
a road traffic flow three-dimensional reconstruction method comprises the following steps:
the server acquires data of the millimeter wave radar and the camera;
estimating the actual position of the vehicle on the road and the vehicle model information by utilizing a neural network;
the server encodes the actual positions and vehicle model information of all vehicles on the road into data in a JSON character string format, and pushes the encoded data to a plurality of clients through a network transmission protocol;
the client establishes a vehicle three-dimensional model database and is responsible for decoding the received JSON format data packet, acquiring the vehicle model and actual position information and establishing a virtual road model;
and the client matches the three-dimensional model corresponding to the vehicle in the vehicle three-dimensional model database according to the vehicle model obtained by the output decoding, maps the vehicle three-dimensional model to the virtual road model according to the actual position information of the vehicle, reestablishes the actual condition of the traffic flow on the road, and displays the traffic flow in all directions.
As a further scheme of the invention, the method for acquiring the data of the millimeter wave radar and the camera is to install the millimeter wave radar and the camera sensor on a speed measuring gantry of the road monitoring station and obtain the millimeter wave radar data and the camera image data through the measurement of the camera sensor and the millimeter wave radar respectively.
Further, the method for estimating the actual position of the vehicle on the road and the vehicle model information by using the neural network comprises the following steps:
generating a three-dimensional bounding box of the vehicle and an estimated model of the vehicle in the image data by using a target detection neural network;
projecting the vehicle three-dimensional surrounding frame to a two-dimensional image plane to form a target two-dimensional surrounding frame;
and screening the millimeter wave radar point cloud picture of each vehicle by using the formed two-dimensional bounding box, inputting the three-dimensional bounding box information containing the vehicle attitude information and the screened millimeter wave radar point cloud information into a second convolutional neural network together, and estimating the actual position of the vehicle in the real world.
Further, the camera image data adopts a first convolutional neural network to estimate the three-dimensional bounding box information and the vehicle model information of the vehicle.
Further, the method of estimating the actual position of the vehicle on the road and the vehicle model information further includes: the camera sensor and the millimeter wave radar are connected with the data processing server, and a vehicle information detector based on a neural network is deployed on the server to detect the actual position and the vehicle model information of the vehicle.
As a further scheme of the invention, the output of the detector is coded into a character string in a JSON format, and the data is sent to the client through a WebSocket.
As a further scheme of the present invention, the server encodes the detected vehicle position information and vehicle model information into a JSON character string form, and transmits data to the clients by using a WebSocket transmission protocol, and the vehicle position information and vehicle model information are shared to a plurality of clients, so that the three-dimensional display image of the road vehicle is displayed at the terminals of the plurality of clients.
As a further scheme of the invention, the road traffic flow three-dimensional reconstruction method further comprises the steps that the client side counts the number of trucks, trucks and cars and displays the number according to the decoded data; and the client side reconstructs a three-dimensional scene of traffic flow on the road and displays the scene by using WebGL, and the display content comprises a model of the vehicle, position information of the vehicle and the model of the vehicle.
As a further scheme of the invention, the client is further used for integrating a three-dimensional image display function, a vehicle number display function and a display function of the connection state of the client and the server to form a traffic flow real-time display terminal.
As a further aspect of the present invention, the method for displaying the vehicle through three-dimensional reconstruction includes:
the server side carries out character string coding on the position information and the model information of the vehicle;
the server side sends the coded vehicle position information and the coded vehicle model information to the user terminal by using a network transmission protocol;
the user terminal receives and decodes the information to obtain the actual position and model information of the vehicle;
a user terminal establishes a vehicle three-dimensional model database and matches the model of a vehicle;
and the user terminal establishes a road model, maps the vehicle three-dimensional model to the road model according to the actual position information of the vehicle, counts the number of various vehicles and displays the number.
The invention also includes a computer apparatus comprising: the system comprises at least one processor and a memory which is connected with the at least one processor in a communication mode, wherein the memory stores instructions which can be executed by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor executes the road traffic flow three-dimensional reconstruction method.
The present invention also includes a computer-readable storage medium having stored thereon computer instructions for causing the computer to execute the method for three-dimensional reconstruction of a road traffic stream.
The technical scheme provided by the invention has the following beneficial effects:
1. the invention can reduce the communication overhead occupied by the road traffic stream monitoring video transmission.
The invention codes the traffic flow information on the road into a character string form, and sends the character string form to the user terminal by using the network transmission protocol, thereby reducing the communication resources occupied by sharing the road traffic image by the video flow.
2. The invention can restore the traffic flow state on the road at the user terminal, rebuild the traffic flow on the road, can visually display the traffic flow state of the road and is beneficial to the development of intelligent traffic technology.
3. The invention can share the road traffic flow conditions to a plurality of user terminals including intelligent vehicles, road monitoring stations and law enforcement department user terminals, thereby realizing the sharing of the road traffic flow information.
The invention greatly reduces the bandwidth occupied by the conventional video stream sharing mode by utilizing the first point, and can utilize the vehicle three-dimensional model library to re-model the traffic stream condition at the user terminal according to the vehicle model and the actual position information of the vehicle, and the information of the vehicle can be used by a plurality of terminals.
These and other aspects of the invention are apparent from and will be elucidated with reference to the embodiments described hereinafter. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.
Drawings
In order to more clearly illustrate the embodiments of the present invention or technical solutions in the related art, the drawings, which are required to be used in the description of the exemplary embodiments or the related art and are provided for providing further understanding of the present invention and constitute a part of the specification, are used for explaining the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. In the drawings:
fig. 1 is a flowchart of a three-dimensional road traffic reconstruction method according to the present invention.
Fig. 2 is a schematic flow chart of vehicle data acquisition processing in a sample of a road traffic flow three-dimensional reconstruction method according to an embodiment of the present invention.
Fig. 3 is a flowchart of estimating actual position information of a vehicle on a road and a vehicle model in a road traffic stream three-dimensional reconstruction method according to an embodiment of the present invention.
Fig. 4 is a schematic diagram of a first convolutional neural network architecture in a road traffic stream three-dimensional reconstruction method according to an embodiment of the present invention.
Fig. 5 is a schematic composition diagram of modules of a first convolutional neural network architecture in the road traffic three-dimensional reconstruction method in fig. 4 according to an embodiment of the present invention.
Fig. 6 is a schematic structural diagram of a second convolutional neural network in the road traffic flow three-dimensional reconstruction method according to an embodiment of the present invention.
Fig. 7 is a diagram illustrating data processing and communication between a server and a client according to an embodiment of the present invention.
Fig. 8 is a schematic diagram of a client performing three-dimensional reconstruction of a road traffic flow according to information of a server in an embodiment of the present invention.
Fig. 9 is a schematic diagram of a traffic flow real-time display terminal implemented by a client according to the road traffic flow three-dimensional reconstruction method in an embodiment of the present invention.
Fig. 10 is another schematic view of the traffic flow real-time display terminal in fig. 9 according to another embodiment of the invention.
Fig. 11 is a schematic view of another perspective view of the traffic flow real-time display terminal in fig. 9 according to an embodiment of the invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The technical solutions in the exemplary embodiments of the present invention will be described clearly and completely with reference to the accompanying drawings in the exemplary embodiments of the present invention, and it is apparent that the described exemplary embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention provides a road traffic flow three-dimensional reconstruction method, which comprises the steps of estimating actual position and vehicle model information of a vehicle from data of a camera and a millimeter wave radar at a server by using a neural network, coding the information into character strings, transmitting the character strings to a plurality of client sides by using a network transmission protocol, establishing a vehicle three-dimensional model base at the client sides, decoding data sent by the server to obtain the actual position and model information of the vehicle, matching a vehicle model at the vehicle model base, establishing a road model, and mapping the vehicle model to the road model for displaying. The problem of huge communication overhead caused by sharing road traffic stream video images among a plurality of clients is solved. In addition, the invention enables the traffic flow on the road to be rebuilt into a three-dimensional model by the client by utilizing the received information, is beneficial to the development of intelligent traffic technology, and solves the problem that the display of the detection information of the current road detection system is not intuitive enough.
The technical scheme of the invention is further explained by combining the specific embodiment as follows:
referring to fig. 1, fig. 1 is a flowchart of a three-dimensional road traffic flow reconstruction method provided by the present invention.
An embodiment of the invention provides a road traffic stream three-dimensional reconstruction method, which aims to solve the problem of huge communication overhead caused by sharing a video stream by multiple clients of the current traffic detection system, and comprises the following steps:
s1, installing a millimeter wave radar and a camera sensor on a road speed measuring gantry, respectively connecting the camera sensor and the millimeter wave radar with a data processing server, arranging a vehicle information detector based on a neural network on the server, and detecting the actual position and the model information of a vehicle.
In this embodiment, the server obtains data of the millimeter wave radar and the camera. The equipment used for data acquisition is a millimeter wave radar and a camera which are arranged on a road portal frame. When the vehicle passes through the equipment installation road section, the millimeter wave radar acquires radar point cloud data of the vehicle, and the camera acquires camera image data of the vehicle. And the service end deploys a neural network detector for estimating the actual position information of the vehicle in the real world and the vehicle model from the millimeter wave radar and camera data.
The method comprises the following steps that a millimeter wave radar and a camera sensor are installed on a speed measuring gantry of a road monitoring station, and millimeter wave radar data and camera image data are obtained through measurement of the camera sensor and the millimeter wave radar respectively.
The acquired millimeter wave radar data is subjected to data transmission through a gigabit Ethernet interface, and the acquired camera image data is subjected to data transmission through a Universal Serial Bus (Universal Serial Bus).
It should be noted that, referring to fig. 2, this embodiment is used for processing millimeter wave radar point cloud data and camera image data, and estimating actual position information of the vehicle on the road and the model of the vehicle from the data by applying a neural network algorithm. The specific processing method is to adopt a first convolutional neural network and a second convolutional neural network for data processing.
And the camera image data adopts a first convolution neural network to estimate the three-dimensional bounding box information and the vehicle model information of the vehicle.
And the millimeter wave radar data estimates the specific position information of the three-dimensional center of the vehicle on the road by adopting a second convolutional neural network and combining the three-dimensional bounding box information of the vehicle. Referring to fig. 3, the method of estimating actual position information of a vehicle on a road includes:
s11, estimating three-dimensional bounding box information and vehicle model information of the vehicle by the first convolutional neural network;
s12, screening the point cloud pictures by utilizing the positions of the three-dimensional surrounding frames in the images;
s13, inputting the screened millimeter wave radar point cloud picture and the three-dimensional bounding box information of the vehicle into a second convolutional neural network;
and S14, estimating the actual position information of the vehicle by the second convolutional neural network.
In this embodiment, the method for estimating the actual position information of the vehicle on the road specifically includes: generating a millimeter wave radar data point cloud picture according to the acquired millimeter wave radar data; generating a three-dimensional surrounding frame of the vehicle from the image data by using a first convolution neural network, projecting the three-dimensional surrounding frame of the vehicle to a two-dimensional image plane to form a two-dimensional surrounding frame, and screening the millimeter wave radar point cloud by using the two-dimensional surrounding frame of the vehicle to screen out the radar point cloud of the same vehicle; and inputting the screened radar point cloud and the three-dimensional bounding box information of the vehicle into a second convolutional neural network, and estimating the specific position information of the three-dimensional center of the vehicle on the road. And encoding the output of the detector into a character string in a JSON format, and sending data to the client through a network transmission protocol WebSocket.
In the embodiment, the first convolutional neural network incorporates a deep aggregation feature extraction network architecture, and the overall structure of the first convolutional neural network is shown in fig. 4.
The components of the modules in fig. 4 are shown in fig. 5, and include known neural network modules:
conv is the convolutional layer;
BN Batch standardization (Batch Normalization);
relu Linear rectification function (Rectified Linear Unit);
concat is tensor splicing;
pooling;
full connection: a fully-connected layer;
flatten: a fully-connected layer;
sigmoid is the activation function of neural networks of the type S.
In the present embodiment, the structure of the second convolutional neural network is as shown in fig. 6.
And S2, encoding the output of the detector into a character string in a JSON format, and sending data to the client through a network transmission protocol Websocket.
In this embodiment, the vehicle actual position and vehicle model information estimated by the server neural network are encoded into a JSON-format character string, and then the vehicle position information and the vehicle model information are sent to a plurality of clients through a WebSocket network protocol, which is shown in fig. 7.
In this embodiment, the WebSocket is based on the TCP protocol, and can provide reliable connection between the client and the server.
And S3, establishing a plurality of clients, receiving and decoding the data sent by the server by the clients, and obtaining the models and the position information of all vehicles.
In this embodiment, based on the WebSocket transmission protocol, the data of the server may share the detected vehicle data with multiple clients at the same time, and each client may obtain the location and model information of the vehicle. The server encodes the actual positions and vehicle model information of all vehicles on the road into data in a JSON character string format, and pushes the encoded data to a plurality of clients through a network transmission protocol.
And S4, establishing a three-dimensional model database of the vehicle types at the client, and displaying the model at the corresponding position according to the obtained vehicle model and position information.
In this embodiment, a client establishes a vehicle three-dimensional model database, as shown in fig. 8, after receiving JSON character string information, the client decodes a data packet to obtain the model number and position information of a vehicle, searches for a suitable vehicle model in the three-dimensional model database, establishes a road model in a virtual three-dimensional space, maps the vehicle model to the road model according to the actual position information of the vehicle for display, reestablishes the actual condition of traffic flow on the road, and displays the actual condition in all directions.
According to the embodiment of the invention, a high-precision vehicle three-dimensional model is used to achieve a better visualization effect, a user can customize various types of vehicle model styles, and the user can change the fineness of the model according to the hardware performance of a client, so that the displayed frame rate reaches an ideal level.
The loading of the vehicle model utilizes a WebGL technology, so that a client can view a scene reconstructed in three dimensions of the vehicle only by accessing a specific webpage.
The client side counts and displays the number of the trucks, the trucks and the cars according to the decoded data; and the client side reconstructs a three-dimensional scene of traffic flow on the road and displays the scene by using WebGL, and the display content comprises a model of the vehicle, position information of the vehicle and the model of the vehicle.
And S5, integrating the vehicle quantity counting function, the server connection state display function and the three-dimensional scene display function by the client to form a complete traffic flow real-time display terminal, which is shown in fig. 9.
In the present embodiment, the vehicle counting function may display the number of different types of vehicles, and in the embodiment of the present invention, the number of cars, the number of trucks, and the number of trucks can be displayed.
In the embodiment of the invention, the three-dimensional scene is displayed in a 360-degree all-around manner, and a user can finish the switching of the visual angles by dragging a mouse at a client, which is shown in fig. 10 and 11.
It should be particularly noted that, during the three-dimensional visual display of the vehicle, the information such as the specific position, the driving speed, the vehicle model, the license plate of the vehicle and the like can be displayed according to the processed output data, wherein a correct model is screened out in an automobile three-dimensional model library according to the output vehicle model information. In this embodiment, the model can be classified into a car model, a motorcycle model, a truck model, a passenger car model. The screened model can be subjected to three-dimensional visualization by means of WebGL in combination with output position information.
In other embodiments of the present invention, when displaying the three-dimensional reconstruction of the road traffic flow, webGL may use OpenGL instead, but the effect is the same.
The invention provides a road traffic flow three-dimensional reconstruction method which can reduce huge communication overhead caused by direct sharing of road traffic flow video information among clients. The method comprises the steps that the actual position and the vehicle model of a vehicle are estimated from data of a camera and a millimeter wave radar by means of a neural network, the actual position and the vehicle model are stored in a server in a text mode, the character strings are coded into character strings in a JSON format and sent to a plurality of client sides through WebSocket, the client sides decode information, the vehicle model is matched from an established vehicle three-dimensional model library according to vehicle model information, and the vehicle model is mapped to a virtual road model by means of the actual position information of the vehicle for displaying. The invention enables the traffic flow on the road to be three-dimensionally reconstructed by the client under the condition of not sharing the video flow, reduces the overhead of signal transmission, is beneficial to the sharing and interaction of traffic information and the development of intelligent traffic technology.
It should be understood that although the above steps are described in a certain order, these steps are not necessarily performed in the order described. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, some steps of the present embodiment may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the steps or stages is not necessarily sequential, but may be performed alternately or in turns with other steps or at least a part of the steps or stages in other steps.
In an embodiment of the present invention, there is also provided a computer device, including at least one processor, and a memory communicatively connected to the at least one processor, where the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to cause the at least one processor to execute the method for reconstructing three-dimensional road traffic streams, and the processor executes the instructions to implement the method.
In one embodiment of the present invention, a computer-readable storage medium is provided, which stores computer instructions for causing the computer to execute the method for three-dimensional reconstruction of road traffic flow.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above may be implemented by a computer program represented by computer instructions and instructing relevant hardware, where the computer program may be stored in a non-volatile computer-readable storage medium, and when executed, the computer program may include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory.
Non-volatile memory may include read-only memory, magnetic tape, floppy disk, flash memory, optical storage, or the like. Volatile memory may include random access memory or external cache memory. By way of illustration, and not limitation, RAM can take many forms, such as static random access memory, dynamic random access memory, and the like.
In summary, the road traffic flow three-dimensional reconstruction method, the computer device and the storage medium provided by the invention can be used in the fields of the existing known application, such as the detection and reconstruction display of road traffic flows, and the fields of the existing intelligent transportation, automatic driving, road management and control and traffic planning.
The advantages of the present invention over the prior art are twofold. The first point is that the huge communication overhead caused by sharing the road vehicle video stream between the traffic monitoring station and the user terminal can be reduced. At present, road monitoring data is commonly shared between a traffic flow monitoring station and a client in a way of directly sharing video data streams, and huge communication overhead is brought by the increase of the number of the clients. The invention estimates the actual position and the vehicle model of the vehicle by utilizing the data of the camera and the millimeter wave radar through the neural network, codes the actual position and the vehicle model of the vehicle into data in a character string format, and reliably transmits the data to the client through the network transmission protocol Websocket, thereby reducing the communication overhead caused by data transmission between the server and the client. The second point is that the traffic flow on the road can be reconstructed and displayed in three dimensions at the client, which is beneficial to the development of intelligent transportation. The detection information of the current road detection system can only be displayed in a two-dimensional mode in a text mode, and the displayed information is not visual enough. The invention integrates the type of the vehicle by utilizing the position information of the vehicle estimated by the first point, screens the model by utilizing the vehicle three-dimensional model library, maps the vehicle model to the road model according to the actual position of the vehicle and can intuitively display the road vehicle condition.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (8)
1. A road traffic flow three-dimensional reconstruction method is characterized by comprising the following steps:
the server acquires data of the millimeter wave radar and the camera;
estimating the actual position of the vehicle on the road and the vehicle model information by using a neural network;
the server encodes the actual positions and vehicle model information of all vehicles on the road into data in a JSON character string format, and pushes the encoded data to a plurality of clients through a network transmission protocol;
the client establishes a vehicle three-dimensional model database and is responsible for decoding the received JSON format data packet, acquiring the vehicle model and actual position information and establishing a virtual road model;
the client matches a three-dimensional model corresponding to the vehicle in a vehicle three-dimensional model database according to the vehicle model obtained by the output decoding, maps the vehicle three-dimensional model to a virtual road model according to the actual position information of the vehicle, reestablishes the actual condition of traffic flow on the road, and displays the actual condition in all directions;
the method for estimating the actual position of the vehicle on the road and the vehicle model information by using the neural network comprises the following steps:
generating a three-dimensional bounding box of the vehicle and an estimated model of the vehicle in the image data by using a target detection neural network;
projecting the vehicle three-dimensional surrounding frame to a two-dimensional image plane to form a target two-dimensional surrounding frame;
and screening the millimeter wave radar point cloud picture of each vehicle by using the formed two-dimensional bounding box, and combining the three-dimensional bounding box information containing the vehicle attitude information with the screened millimeter wave radar point cloud information to estimate the actual position of the vehicle in the real world.
2. The method for three-dimensional reconstruction of road traffic flow according to claim 1, wherein the method for obtaining the data of the millimeter wave radar and the camera is to install the millimeter wave radar and the camera sensor on a speed measuring gantry of the road monitoring station, and obtain the millimeter wave radar data and the camera image data through the measurement of the camera sensor and the millimeter wave radar respectively.
3. The method for three-dimensional reconstruction of road traffic stream according to claim 2, wherein the method of estimating the actual position of the vehicle on the road and the vehicle model information further comprises: the camera sensor and the millimeter wave radar are connected with the data processing server, and a vehicle information detector based on a neural network is deployed on the server to detect the actual position and the vehicle model information of the vehicle.
4. The method for three-dimensional reconstruction of road traffic flow according to claim 1, wherein the server encodes the detected vehicle position information and vehicle model information into a JSON character string form, and transmits data to the clients by using a WebSocket transmission protocol, and the vehicle position information and the vehicle model information are shared to the clients so that three-dimensional display images of the road vehicles are displayed at terminals of the clients.
5. The three-dimensional road traffic flow reconstruction method according to claim 1, further comprising the steps of counting and displaying the number of trucks, trucks and cars by the client according to the decoded data; the client side utilizes a 3D drawing protocol WebGL (Web Graphics Library) to reconstruct and display a three-dimensional scene of traffic flow on the road, and display contents comprise models of vehicles, position information of the vehicles and vehicle models.
6. The method for three-dimensional reconstruction of road traffic flow according to claim 5, characterized in that the client is further used for integrating a three-dimensional image display function, a vehicle number display function and a display function of a connection state of the client and the server to form a real-time traffic flow display terminal.
7. A computer device, comprising: at least one processor, and a memory communicatively coupled to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executable by the at least one processor to cause the at least one processor to perform the method of three-dimensional reconstruction of roadway traffic flows as recited in any one of claims 1-6.
8. A computer-readable storage medium storing computer instructions for causing a computer to execute the method for three-dimensional reconstruction of a road flow according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111393942.7A CN114120642B (en) | 2021-11-23 | 2021-11-23 | Road traffic flow three-dimensional reconstruction method, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111393942.7A CN114120642B (en) | 2021-11-23 | 2021-11-23 | Road traffic flow three-dimensional reconstruction method, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114120642A CN114120642A (en) | 2022-03-01 |
| CN114120642B true CN114120642B (en) | 2022-12-06 |
Family
ID=80440078
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111393942.7A Active CN114120642B (en) | 2021-11-23 | 2021-11-23 | Road traffic flow three-dimensional reconstruction method, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114120642B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117492758A (en) * | 2024-01-03 | 2024-02-02 | 安徽中科星驰自动驾驶技术有限公司 | Automatic driving visualization method based on WebGL |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH0981495A (en) * | 1995-07-11 | 1997-03-28 | Sony Corp | Method and system for sharing three-dimensional virtual reality space, medium and method for information recording, medium and method for information transmission, information processing method, and shared server terminal |
Family Cites Families (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8930844B2 (en) * | 2000-08-22 | 2015-01-06 | Bruce Carlin | Network repository of digitalized 3D object models, and networked generation of photorealistic images based upon these models |
| US7305396B2 (en) * | 2002-12-31 | 2007-12-04 | Robert Bosch Gmbh | Hierarchical system and method for on-demand loading of data in a navigation system |
| CN102364523A (en) * | 2011-05-11 | 2012-02-29 | 武汉理工大学 | Method for realizing three-dimensional virtual city system based on RIA (rich Internet application) architecture |
| US10868890B2 (en) * | 2011-11-22 | 2020-12-15 | Trimble Navigation Limited | 3D modeling system distributed between a client device web browser and a server |
| JP6062041B2 (en) * | 2012-05-07 | 2017-01-18 | 本田技研工業株式会社 | A method for generating a virtual display surface from a video image of a landscape based on a road |
| CN102929969A (en) * | 2012-10-15 | 2013-02-13 | 北京师范大学 | Real-time searching and combining technology of mobile end three-dimensional city model based on Internet |
| CN104573326A (en) * | 2014-12-16 | 2015-04-29 | 北京中交兴路车联网科技有限公司 | Statistical analysis system of vehicle running data |
| CN105679079A (en) * | 2016-02-26 | 2016-06-15 | 同济大学 | Vehicle-mounted machine system facing cooperation of road network and vehicle network |
| US10261747B2 (en) * | 2016-09-09 | 2019-04-16 | The Boeing Company | Synchronized side-by-side display of live video and corresponding virtual environment images |
| CN106840178B (en) * | 2017-01-24 | 2019-05-03 | 中南大学 | A kind of map building based on ArcGIS and intelligent vehicle autonomous navigation method and system |
| CN107959687A (en) * | 2017-12-13 | 2018-04-24 | 上海曼恒数字技术股份有限公司 | A kind of data processing method, system, equipment and storage medium |
| CN109164764A (en) * | 2018-07-27 | 2019-01-08 | 中国兵器装备集团自动化研究所 | Workshop wholegrain degree three-dimensional visualization monitoring system and construction method |
| CN110135453A (en) * | 2019-03-29 | 2019-08-16 | 初速度（苏州）科技有限公司 | A kind of laser point cloud data mask method and device |
| CN110175576B (en) * | 2019-05-29 | 2022-09-16 | 电子科技大学 | Driving vehicle visual detection method combining laser point cloud data |
| US10636209B1 (en) * | 2019-09-13 | 2020-04-28 | Bongfish GmbH | Reality-based three-dimensional infrastructure reconstruction |
| CN111829535B (en) * | 2020-06-05 | 2022-05-03 | 阿波罗智能技术(北京)有限公司 | Method and device for generating offline map, electronic equipment and storage medium |
| CN111934840A (en) * | 2020-06-29 | 2020-11-13 | 北京百度网讯科技有限公司 | Communication method of client and server, gateway, electronic equipment and storage medium |
| CN112560580B (en) * | 2020-11-20 | 2022-01-28 | 腾讯科技（深圳）有限公司 | Obstacle recognition method, device, system, storage medium and electronic equipment |
| CN112669614A (en) * | 2020-11-27 | 2021-04-16 | 图达通智能科技(苏州)有限公司 | Device and method for acquiring and publishing real-time road condition information |
| CN113033412B (en) * | 2021-03-26 | 2023-08-22 | 平安国际智慧城市科技股份有限公司 | Traffic situation monitoring method, device, equipment and medium based on live-action fusion |
| CN113240939B (en) * | 2021-03-31 | 2023-03-28 | 浙江吉利控股集团有限公司 | Vehicle early warning method, device, equipment and storage medium |
| CN113192182A (en) * | 2021-04-29 | 2021-07-30 | 山东产研信息与人工智能融合研究院有限公司 | Multi-sensor-based live-action reconstruction method and system |
| CN113421330B (en) * | 2021-06-21 | 2023-09-29 | 车路通科技(成都)有限公司 | Vehicle-road cooperative road three-dimensional scene construction method, device, equipment and medium |
-
2021
- 2021-11-23 CN CN202111393942.7A patent/CN114120642B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH0981495A (en) * | 1995-07-11 | 1997-03-28 | Sony Corp | Method and system for sharing three-dimensional virtual reality space, medium and method for information recording, medium and method for information transmission, information processing method, and shared server terminal |
Non-Patent Citations (1)
| Title |
|---|
| 车载移动执法中违规车辆智能检测研究;陈刚等;《电子科技大学学报》;20180530(第03期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114120642A (en) | 2022-03-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106846863B (en) | Accident black spot warning system and method based on augmented reality and cloud intelligent decision | |
| CN112417953B (en) | Road condition detection and map data updating method, device, system and equipment | |
| CA2967495A1 (en) | System and method for compressing video data | |
| CN115187742B (en) | Method, system and related device for generating automatic driving simulation test scene | |
| CN108230437A (en) | Scene reconstruction method and device, electronic equipment, program and medium | |
| CN109598977A (en) | Vehicle checking method, device, system and server based on video monitoring | |
| JP7267363B2 (en) | Test method, device and equipment for traffic flow monitoring measurement system | |
| CN114120642B (en) | Road traffic flow three-dimensional reconstruction method, computer equipment and storage medium | |
| CN110087041B (en) | Video data processing and transmitting method and system based on 5G base station | |
| CN110175533A (en) | Overpass traffic condition method of real-time, device, terminal and storage medium | |
| CN114332809A (en) | Image identification method and device, electronic equipment and storage medium | |
| CN114663566A (en) | Loading rendering processing method and system for three-dimensional model | |
| CN112215033B (en) | Method, device and system for generating panoramic looking-around image of vehicle and storage medium | |
| CN114612544A (en) | Image processing method, device, equipment and storage medium | |
| Ahmed et al. | A Joint Perception Scheme For Connected Vehicles | |
| CN114359857A (en) | Processing method, device and system for reported information | |
| CN113094564A (en) | Data processing method and device | |
| CN112948459B (en) | Railway infrastructure inspection device, method and system | |
| US12125222B1 (en) | Systems for determining and reporting vehicle following distance | |
| CN117173693B (en) | 3D target detection method, electronic device, medium and driving device | |
| US20240331398A1 (en) | Methods for creating training data for determining vehicle following distance | |
| US20240331175A1 (en) | Systems for determining and reporting vehicle following distance | |
| US20240331186A1 (en) | Systems and methods for training a model for determining vehicle following distance | |
| CN117011787B (en) | Information processing method and device applied to gas station and electronic equipment | |
| CN115688484B (en) | V2X simulation method and system based on WebGL |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |