CN114120642B - Method for three-dimensional reconstruction of road traffic flow, computer equipment and storage medium - Google Patents
Method for three-dimensional reconstruction of road traffic flow, computer equipment and storage medium Download PDFInfo
- Publication number
- CN114120642B CN114120642B CN202111393942.7A CN202111393942A CN114120642B CN 114120642 B CN114120642 B CN 114120642B CN 202111393942 A CN202111393942 A CN 202111393942A CN 114120642 B CN114120642 B CN 114120642B
- Authority
- CN
- China
- Prior art keywords
- vehicle
- road
- dimensional
- information
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 49
- 238000013528 artificial neural network Methods 0.000 claims abstract description 20
- 230000005540 biological transmission Effects 0.000 claims abstract description 19
- 238000012544 monitoring process Methods 0.000 claims description 13
- 230000006870 function Effects 0.000 claims description 10
- 238000001514 detection method Methods 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 6
- 238000005259 measurement Methods 0.000 claims description 3
- 238000012216 screening Methods 0.000 claims 1
- 230000009286 beneficial effect Effects 0.000 abstract description 6
- 238000011161 development Methods 0.000 abstract description 6
- 238000013527 convolutional neural network Methods 0.000 description 16
- 238000004891 communication Methods 0.000 description 14
- 238000010586 diagram Methods 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 7
- 238000004590 computer program Methods 0.000 description 3
- 238000010606 normalization Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 238000011176 pooling Methods 0.000 description 2
- 238000012800 visualization Methods 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000008054 signal transmission Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
- G08G1/0125—Traffic data processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
- G08G1/0137—Measuring and analyzing of parameters relative to traffic conditions for specific applications
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/017—Detecting movement of traffic to be counted or controlled identifying vehicles
- G08G1/0175—Detecting movement of traffic to be counted or controlled identifying vehicles by photographing vehicles, e.g. when violating traffic rules
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/04—Detecting movement of traffic to be counted or controlled using optical or ultrasonic detectors
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/16—Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
- H04L69/169—Special adaptations of TCP, UDP or IP for interworking of IP based networks with other networks
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Analytical Chemistry (AREA)
- Chemical & Material Sciences (AREA)
- Software Systems (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computer Security & Cryptography (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Traffic Control Systems (AREA)
Abstract
本发明属于智能交通技术领域,具体涉及一种道路车流三维重建方法、计算机设备及存储介质。该方法包括:服务器获取毫米波雷达与摄像头的数据;利用神经网络估计车辆在道路上的实际位置和车辆型号信息;服务器将所有车辆在道路上的实际位置和车辆型号信息编码成JSON字符串格式的数据,将编码后的数据推送给多个客户端;客户端建立车辆模型数据库,客户端根据解码出来的数据匹配所述车辆对应的三维模型;客户端重建道路上车流的三维场景并进行展示。本发明降低了服务端与客户端之间的数据传输量,使客户端根据所接收的文本数据就能重建道路的车流三维模型,并且车流信息能够共享给多个客户端,有利于智能交通的发展。
The invention belongs to the technical field of intelligent transportation, and in particular relates to a three-dimensional reconstruction method of road traffic flow, computer equipment and a storage medium. The method includes: the server obtains the data of the millimeter-wave radar and the camera; the neural network is used to estimate the actual position and vehicle model information of the vehicle on the road; the server encodes the actual position and vehicle model information of all vehicles on the road into a JSON string format push the encoded data to multiple clients; the client establishes a vehicle model database, and the client matches the corresponding 3D model of the vehicle according to the decoded data; the client reconstructs the 3D scene of the traffic flow on the road and displays it . The invention reduces the amount of data transmission between the server and the client, enables the client to reconstruct the three-dimensional model of the traffic flow of the road according to the received text data, and the traffic flow information can be shared with multiple clients, which is beneficial to the development of intelligent transportation develop.
Description
技术领域technical field
本发明属于智能交通技术领域,具体涉及一种道路车流三维重建方法、计算机设备及存储介质。The invention belongs to the technical field of intelligent transportation, and in particular relates to a three-dimensional reconstruction method of road traffic flow, computer equipment and a storage medium.
背景技术Background technique
随着城市化进程的推进,交通拥堵问题日益突出。交通信息是道路状况的直观反映,及时掌握交通信息对合理的配置交通资源、缓解拥堵具有重要意义,同时也为智能交通的发展提供数据支撑。在目前的交通监测场景中,主要是通过利用摄像头对行驶在道路上的车辆进行实时检测,获取车辆车牌号码和判断车辆是否违规,利用雷达对车辆进行测速。With the advancement of urbanization, the problem of traffic congestion has become increasingly prominent. Traffic information is an intuitive reflection of road conditions. Timely grasp of traffic information is of great significance for rationally allocating traffic resources and alleviating congestion, and it also provides data support for the development of intelligent transportation. In the current traffic monitoring scene, it is mainly to use the camera to detect the vehicles driving on the road in real time, obtain the vehicle license plate number and judge whether the vehicle violates the regulations, and use the radar to measure the speed of the vehicle.
由于目前的监测系统只能够将检测到的车牌号码、车速和车辆型号以文本的方式展示,不能将道路的实际车流进行建模,并且监测站以视频流的方式与其他终端共享道路图像会占用巨大的通信带宽,造成不必要的传输开销。究其原因在于:Since the current monitoring system can only display the detected license plate number, vehicle speed, and vehicle model in text, it cannot model the actual traffic flow on the road, and the monitoring station shares road images with other terminals in the form of video streams. Huge communication bandwidth causes unnecessary transmission overhead. The reason is that:
1、没有利用车辆在道路上的实际位置信息。1. The actual position information of the vehicle on the road is not utilized.
目前的监测站大都只是利用车辆的型号、车牌和车速信息,缺乏对车辆实际位置信息的利用,本申请利用神经网络估计车辆在道路上的实际位置信息和车辆型号,将车辆的实际位置信息和车辆型号用于道路场景的三维重建。Most of the current monitoring stations only use the model, license plate and speed information of the vehicle, and lack the use of the actual location information of the vehicle. This application uses the neural network to estimate the actual location information and vehicle model of the vehicle on the road, and combines the actual location information and Vehicle models are used for 3D reconstruction of road scenes.
2、缺乏三维重建系统。2. Lack of 3D reconstruction system.
因为缺乏一个能够重建道路车流状况的系统,目前道路监测站主要是以视频流的方式共享给其他用户终端,视频流的传输会带来巨大的通信开销。本申请通过在服务端将所监测到的车辆信息编码成为文本格式,在使用较小通信资源的情况下共享给其他终端,然后在终端建立三维重建的系统。终端解码服务端所发送的数据并利用三维重建系统重新还原出道路车流的实际状况。Due to the lack of a system that can reconstruct road traffic conditions, currently road monitoring stations are mainly shared with other user terminals in the form of video streams, and the transmission of video streams will bring huge communication overhead. This application encodes the monitored vehicle information into a text format at the server end, shares it with other terminals while using relatively small communication resources, and then establishes a three-dimensional reconstruction system at the terminal. The terminal decodes the data sent by the server and uses the 3D reconstruction system to restore the actual condition of the road traffic flow.
发明内容Contents of the invention
为了对目前交通检测系统中由于利用视频流的方式共享道路车流图像造成的巨大通信开销的问题,本发明提供了一种道路车流信息的传输和三维重建的方法、计算机设备及存储介质。In order to solve the problem of huge communication overhead caused by sharing road traffic flow images in the way of video streams in current traffic detection systems, the present invention provides a method for transmission and three-dimensional reconstruction of road traffic flow information, computer equipment and storage media.
本发明采用以下技术方案实现:The present invention adopts following technical scheme to realize:
一种道路车流三维重建方法,包括以下步骤:A method for three-dimensional reconstruction of road traffic flow, comprising the following steps:
服务器获取毫米波雷达与摄像头的数据;The server obtains the data of millimeter wave radar and camera;
利用神经网络估计车辆在道路上的实际位置和车辆型号信息;Estimate the actual position of the vehicle on the road and the vehicle model information by using the neural network;
服务器将所有车辆在道路上的实际位置和车辆型号信息编码成JSON字符串格式的数据,通过网络传输协议将编码后的数据推送给多个客户端;The server encodes the actual position and vehicle model information of all vehicles on the road into data in JSON string format, and pushes the encoded data to multiple clients through the network transmission protocol;
客户端建立车辆三维模型数据库,并负责解码所接收的JSON格式的数据包,获取车辆型号和实际位置信息,并建立虚拟道路模型;The client establishes a vehicle 3D model database, and is responsible for decoding the received data packets in JSON format, obtaining vehicle model and actual location information, and establishing a virtual road model;
客户端根据输出解码所获得的车辆型号在车辆三维模型数据库内匹配所述车辆对应的三维模型,根据车辆的实际位置信息,将车辆三维模型映射到虚拟道路模型上,重新建立道路上车流的实际状况,并全方位地进行展示。According to the vehicle model obtained by output decoding, the client matches the corresponding 3D model of the vehicle in the vehicle 3D model database, maps the vehicle 3D model to the virtual road model according to the actual position information of the vehicle, and re-establishes the actual traffic flow on the road. situation and display it comprehensively.
作为本发明的进一步方案,所述获取毫米波雷达与摄像头的数据的方法为在道路监测站测速龙门架上安装毫米波雷达、摄像头传感器,并分别通过摄像头传感器、毫米波雷达测量获得毫米波雷达数据和摄像头图像数据。As a further solution of the present invention, the method for obtaining the data of the millimeter-wave radar and the camera is to install the millimeter-wave radar and the camera sensor on the speed measuring gantry of the road monitoring station, and obtain the millimeter-wave radar through the measurement of the camera sensor and the millimeter-wave radar respectively. data and camera image data.
进一步的,所述利用神经网络估计车辆在道路上的实际位置和车辆型号信息的方法为:Further, the method of using the neural network to estimate the actual position of the vehicle on the road and the vehicle model information is:
利用目标检测神经网络生成图像数据中车辆的三维包围框和估计车辆的型号;Use the target detection neural network to generate the three-dimensional bounding box of the vehicle in the image data and estimate the model of the vehicle;
将车辆三维包围框投影到二维图像平面,形成目标的二维包围框;Project the 3D bounding box of the vehicle onto the 2D image plane to form the 2D bounding box of the target;
利用所形成的二维包围框对每辆车的毫米波雷达点云图进行筛选,将包含了车辆姿态信息的三维包围框信息与所筛选出来的毫米波雷达点云信息一同输入到第二卷积神经网络,估计出车辆在现实世界中的实际位置。Use the formed two-dimensional bounding box to filter the millimeter-wave radar point cloud image of each vehicle, and input the three-dimensional bounding box information containing the vehicle attitude information and the filtered millimeter-wave radar point cloud information to the second convolution A neural network that estimates the actual location of the vehicle in the real world.
进一步的,所述摄像头图像数据采用第一卷积神经网络估计车辆的三维包围框信息和车辆型号信息。Further, the camera image data uses the first convolutional neural network to estimate the three-dimensional bounding box information and vehicle model information of the vehicle.
进一步的,估计车辆在道路上的实际位置和车辆型号信息的方法还包括:将摄像头传感器、毫米波雷达与数据处理服务器连接,在服务器上部署基于神经网络的车辆信息检测器,检测车辆的实际位置与车辆型号信息。Further, the method for estimating the actual position of the vehicle on the road and the vehicle model information also includes: connecting the camera sensor, the millimeter-wave radar and the data processing server, deploying a neural network-based vehicle information detector on the server, and detecting the actual position of the vehicle. Location and vehicle model information.
作为本发明的进一步方案,将检测器的输出编码为JSON格式的字符串,通过网络传输协议WebSocket发送数据到客户端。As a further solution of the present invention, the output of the detector is encoded into a string in JSON format, and the data is sent to the client through the network transmission protocol WebSocket.
作为本发明的进一步方案,所述服务端将检测出的车辆位置信息和车辆型号信息编码成JSON字符串的形式,并利用WebSocket传输协议向客户端传输数据,所述车辆位置信息和车辆型号信息共享给多个客户端,以使道路车辆的三维显示图像在多个客户端的终端进行显示。As a further solution of the present invention, the server encodes the detected vehicle location information and vehicle model information into the form of a JSON character string, and uses the WebSocket transmission protocol to transmit data to the client. The vehicle location information and vehicle model information Shared to multiple clients, so that the three-dimensional display images of road vehicles can be displayed on the terminals of multiple clients.
作为本发明的进一步方案,本发明的道路车流三维重建方法,还包括客户端根据解码出来的数据,统计货车、卡车和轿车的数量并展示;客户端利用WebGL,重建道路上车流的三维场景并进行展示,展示内容包括车辆的模型、车辆的位置信息和车辆型号。As a further solution of the present invention, the road traffic flow three-dimensional reconstruction method of the present invention also includes that the client calculates and displays the quantity of trucks, trucks and cars according to the decoded data; the client uses WebGL to reconstruct the three-dimensional scene of the traffic flow on the road and Display, the display content includes the model of the vehicle, the location information of the vehicle and the model of the vehicle.
作为本发明的进一步方案,所述客户端还用于整合三维图像显示功能、车辆数目显示功能和客户端与服务端连接状态的显示功能,形成一个车流实时展示终端。As a further solution of the present invention, the client is also used to integrate the functions of displaying three-dimensional images, displaying the number of vehicles and displaying the connection status between the client and the server to form a real-time display terminal for traffic flow.
作为本发明的进一步方案,所述车辆三维重建展示的方法,包括:As a further solution of the present invention, the method for the three-dimensional reconstruction and display of the vehicle includes:
服务端对车辆的位置信息和型号信息进行字符串编码;The server encodes the location information and model information of the vehicle as strings;
服务端利用网络传输协议将编码后的车辆位置信息与型号信息发送给用户终端;The server uses the network transmission protocol to send the encoded vehicle location information and model information to the user terminal;
用户终端接收并解码信息,获取车辆的实际位置和型号信息;The user terminal receives and decodes the information to obtain the actual location and model information of the vehicle;
用户终端建立车辆三维模型数据库,与车辆的型号进行匹配;The user terminal establishes a vehicle 3D model database and matches it with the vehicle model;
用户终端建立道路模型,根据车辆的实际位置信息将车辆三维模型映射到道路模型,统计各类车辆的数目并进行显示。The user terminal builds a road model, maps the three-dimensional vehicle model to the road model according to the actual location information of the vehicle, counts and displays the number of various vehicles.
本发明还包括一种计算机设备,包括:至少一个处理器,以及与所述至少一个处理器通信连接的存储器,其中,所述存储器存储有可被所述至少一个处理器执行的指令,所述指令被所述至少一个处理器执行,以使所述至少一个处理器执行所述的道路车流三维重建方法。The present invention also includes a computer device, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the The instructions are executed by the at least one processor, so that the at least one processor executes the method for three-dimensional reconstruction of road traffic flow.
本发明还包括一种计算机可读存储介质,所述计算机可读存储介质存储有计算机指令,所述计算机指令用于使所述计算机执行所述的道路车流三维重建方法。The present invention also includes a computer-readable storage medium, where the computer-readable storage medium stores computer instructions, and the computer instructions are used to make the computer execute the method for three-dimensional reconstruction of road traffic flow.
本发明提供的技术方案,具有如下有益效果:The technical scheme provided by the invention has the following beneficial effects:
1、本发明能够减少道路车流监控视频传输所占用的通信开销。1. The present invention can reduce the communication overhead occupied by road traffic monitoring video transmission.
本发明将道路上的车流信息编码为字符串的形式,利用网络传输协议发送给用户终端,减少了以视频流共享道路交通图像所占用的通信资源。The invention encodes the vehicle flow information on the road into a character string, and sends it to the user terminal by using the network transmission protocol, thereby reducing the communication resources occupied by sharing the road traffic image with the video stream.
2、本发明能够在用户终端重新还原道路上的车流状态,对道路上的车流进行重新建模,能够直观地显示道路的车流状况,有利于智能交通技术的发展。2. The present invention can restore the state of traffic flow on the road at the user terminal, remodel the traffic flow on the road, and can intuitively display the state of traffic flow on the road, which is beneficial to the development of intelligent transportation technology.
3、本发明能够将道路车流状况共享给多个用户终端,包括智能车辆、道路监测站和执法部门用户终端,实现道路车流信息的共享。3. The present invention can share road traffic flow conditions with multiple user terminals, including smart vehicles, road monitoring stations, and law enforcement department user terminals, so as to realize the sharing of road traffic flow information.
本发明利用第一点大大降低以往视频流共享方式所占用的带宽,并且能够根据车辆型号和车辆实际位置信息利用车辆三维模型库在用户终端对车流状况进行重新建模,车辆的信息能够供多个终端使用。The present invention utilizes the first point to greatly reduce the bandwidth occupied by the previous video stream sharing method, and can use the vehicle three-dimensional model library to remodel the traffic flow at the user terminal according to the vehicle model and the actual position information of the vehicle, and the information of the vehicle can be provided for multiple terminal use.
本发明的这些方面或其他方面在以下实施例的描述中会更加简明易懂。应当理解的是,以上的一般描述和后文的细节描述仅是示例性和解释性的,并不能限制本发明。These or other aspects of the present invention will be more clearly understood in the description of the following embodiments. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention.
附图说明Description of drawings
为了更清楚地说明本发明实施例或相关技术中的技术方案,下面将对示例性实施例或相关技术描述中所需要使用的附图作一简单地介绍,附图用来提供对本发明的进一步理解,并且构成说明书的一部分,与本发明的实施例一起用于解释本发明,并不构成对本发明的限制。在附图中:In order to more clearly illustrate the technical solutions in the embodiments of the present invention or related technologies, the following will briefly introduce the accompanying drawings that need to be used in the description of exemplary embodiments or related technologies. The accompanying drawings are used to provide a further description of the present invention It should be understood that it constitutes a part of the description, and is used to explain the present invention together with the embodiments of the present invention, and does not constitute a limitation to the present invention. In the attached picture:
图1为本发明的一种道路车流三维重建方法的流程图。FIG. 1 is a flow chart of a method for three-dimensional reconstruction of road traffic flow according to the present invention.
图2为本发明一个实施例中道路车流三维重建方法的样例中车辆采集数据处理的流程示意图。FIG. 2 is a schematic flow chart of vehicle acquisition data processing in an example of a method for three-dimensional reconstruction of road traffic flow in an embodiment of the present invention.
图3为本发明一个实施例中道路车流三维重建方法中估计车辆在道路上的实际位置信息和车辆型号的流程图。Fig. 3 is a flow chart of estimating actual location information and vehicle model of a vehicle on a road in a method for 3D reconstruction of road traffic flow in an embodiment of the present invention.
图4为本发明一个实施例中道路车流三维重建方法中第一卷积神经网络架构示意图。Fig. 4 is a schematic diagram of the architecture of the first convolutional neural network in the method for 3D reconstruction of road traffic flow in an embodiment of the present invention.
图5为图4中本发明一个实施例中道路车流三维重建方法中第一卷积神经网络架构的各个模块的组成示意图。FIG. 5 is a schematic diagram of the composition of each module of the first convolutional neural network architecture in the method for 3D reconstruction of road traffic flow in an embodiment of the present invention in FIG. 4 .
图6为本发明一个实施例中道路车流三维重建方法中第二卷积神经网络结构示意图。Fig. 6 is a schematic diagram of the structure of the second convolutional neural network in the method for 3D reconstruction of road traffic flow in an embodiment of the present invention.
图7为本发明一个实施例中服务端数据处理并与客户端进行数据通信的示意图。FIG. 7 is a schematic diagram of data processing and data communication between the server and the client in an embodiment of the present invention.
图8为本发明一个实施例中客户端根据服务端的信息进行道路车流三维重建的示意图。Fig. 8 is a schematic diagram of the three-dimensional reconstruction of road traffic flow performed by the client according to the information of the server in an embodiment of the present invention.
图9为本发明一个实施例中一个客户端利用本发明的道路车流三维重建方法实现的车流实时展示终端的示意图。FIG. 9 is a schematic diagram of a real-time display terminal of traffic flow realized by a client using the three-dimensional road traffic flow reconstruction method of the present invention in an embodiment of the present invention.
图10为图9本发明一个实施例中车流实时展示终端的另一视角示意图。FIG. 10 is a schematic diagram of another perspective of the real-time display terminal of traffic flow in one embodiment of the present invention shown in FIG. 9 .
图11为图9本发明一个实施例中车流实时展示终端的另一视角示意图。FIG. 11 is a schematic diagram of another perspective of the real-time display terminal of traffic flow in one embodiment of the present invention shown in FIG. 9 .
具体实施方式detailed description
为了使本发明的目的、技术方案及优点更加清楚明白,以下结合附图及实施例,对本发明进行进一步详细说明。应当理解,此处所描述的具体实施例仅用以解释本发明,并不用于限定本发明。In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.
下面将结合本发明示例性实施例中的附图,对本发明示例性实施例中的技术方案进行清楚、完整地描述,显然,所描述的示例性实施例仅仅是本发明一部分实施例,而不是全部的实施例。基于本发明中的实施例,本领域技术人员在没有作出创造性劳动前提下所获得的所有其他实施例,都属于本发明保护的范围。The technical solutions in the exemplary embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the exemplary embodiments of the present invention. Obviously, the described exemplary embodiments are only part of the embodiments of the present invention, rather than Full examples. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts fall within the protection scope of the present invention.
本发明提供的一种道路车流三维重建方法,在服务端利用神经网络从摄像头与毫米波雷达的数据中估计车辆的实际位置和车辆型号信息,并将信息编码为字符串,利用网络传输协议传输给多个客户端,客户端建立车辆三维模型库,解码服务端发送的数据获得车辆的实际位置和型号信息,在车辆模型库匹配车辆模型,并建立道路模型,将车辆模型映射到道路模型上进行显示。解决目前由于多个客户端之间共享道路车流视频图像所造成的巨大通信开销的问题。并且,本发明使得道路上的车流能够被客户端利用所接收的信息重新建立三维模型,有利于智能交通技术的发展,解决目前的道路检测系统的检测信息展示不够直观的问题。A method for three-dimensional reconstruction of road traffic flow provided by the present invention uses the neural network to estimate the actual position and vehicle model information of the vehicle from the data of the camera and the millimeter-wave radar at the server end, encodes the information into a character string, and transmits it using the network transmission protocol For multiple clients, the client establishes a vehicle 3D model library, decodes the data sent by the server to obtain the actual location and model information of the vehicle, matches the vehicle model in the vehicle model library, and establishes a road model, and maps the vehicle model to the road model to display. Solve the current problem of huge communication overhead caused by sharing road traffic video images between multiple clients. Moreover, the present invention enables the client to reconstruct a three-dimensional model of the traffic flow on the road using the received information, which is beneficial to the development of intelligent transportation technology and solves the problem that the detection information display of the current road detection system is not intuitive enough.
下面结合具体实施例对本发明的技术方案作进一步的说明:The technical scheme of the present invention will be further described below in conjunction with specific embodiments:
参阅图1所示,图1为本发明提供的一种道路车流三维重建方法的流程图。Referring to FIG. 1 , FIG. 1 is a flow chart of a method for three-dimensional reconstruction of road traffic flow provided by the present invention.
本发明的一个实施例提供了一种道路车流三维重建方法,为了解决目前交通检测系统多客户端共享视频流造成的巨大通信开销的问题,该方法包括以下步骤:An embodiment of the present invention provides a method for three-dimensional reconstruction of road traffic flow. In order to solve the problem of huge communication overhead caused by multi-client sharing of video streams in the current traffic detection system, the method includes the following steps:
S1、在道路测速龙门架上安装毫米波雷达、摄像头传感器,并分别将摄像头传感器、毫米波雷达与数据处理服务器连接,在服务器上部署基于神经网络的车辆信息检测器,检测车辆的实际位置与车辆型号信息。S1. Install the millimeter-wave radar and camera sensor on the road speed measuring gantry, and respectively connect the camera sensor and the millimeter-wave radar to the data processing server, and deploy a neural network-based vehicle information detector on the server to detect the actual position and location of the vehicle. Vehicle model information.
在本实施例中,服务器获取毫米波雷达与摄像头的数据。其中,数据采集所采用的设备为安装在道路龙门架上的毫米波雷达与摄像头。在车辆途经该设备安装路段时,由毫米波雷达采集车辆的雷达点云数据,由摄像头采集车辆的摄像头图像数据。服务端部署神经网络检测器,用于从毫米波雷达与摄像头数据中估计车辆在现实世界的实际位置信息和车辆型号。In this embodiment, the server acquires the data of the millimeter wave radar and the camera. Among them, the equipment used for data collection is the millimeter-wave radar and camera installed on the road gantry. When the vehicle passes through the road section where the equipment is installed, the radar point cloud data of the vehicle is collected by the millimeter-wave radar, and the camera image data of the vehicle is collected by the camera. The server deploys a neural network detector to estimate the actual location information and vehicle model of the vehicle in the real world from millimeter-wave radar and camera data.
在道路监测站测速龙门架上安装毫米波雷达、摄像头传感器,并分别通过摄像头传感器、毫米波雷达测量获得毫米波雷达数据和摄像头图像数据。Install millimeter-wave radar and camera sensors on the speed-measuring gantry of the road monitoring station, and obtain millimeter-wave radar data and camera image data through camera sensor and millimeter-wave radar measurements respectively.
其中,采集后的毫米波雷达数据通过千兆以太网接口进行数据传输,采集后的摄像头图像数据通过通用串行总线(Universal Serial Bus) 进行数据传输。Among them, the collected millimeter-wave radar data is transmitted through the Gigabit Ethernet interface, and the collected camera image data is transmitted through the Universal Serial Bus (Universal Serial Bus).
需要特别说明的是,参见图2所示,本实施例中是用于处理毫米波雷达点云数据和摄像头图像数据,从数据中运用神经网络算法估计出车辆在道路上的实际位置信息和车辆的型号。具体的处理方法为采用第一卷积神经网络和第二卷积神经网络以进行数据处理。It should be noted that, as shown in Figure 2, this embodiment is used to process millimeter-wave radar point cloud data and camera image data, and use neural network algorithms to estimate the actual position information and vehicle location information of the vehicle on the road from the data. model. A specific processing method is to use the first convolutional neural network and the second convolutional neural network for data processing.
所述摄像头图像数据采用第一卷积神经网络估计车辆的三维包围框信息和车辆型号信息。The camera image data uses the first convolutional neural network to estimate the three-dimensional bounding box information and vehicle model information of the vehicle.
所述毫米波雷达数据采用第二卷积神经网络结合车辆的三维包围框信息估计出车辆三维中心在道路上的具体位置信息。参见图3所示,估计车辆在道路上的实际位置信息的方法包括:The millimeter-wave radar data uses the second convolutional neural network combined with the three-dimensional bounding box information of the vehicle to estimate the specific position information of the three-dimensional center of the vehicle on the road. Referring to Fig. 3, the method for estimating the actual position information of the vehicle on the road includes:
S11、第一卷积神经网络估计车辆的三维包围框信息和车辆型号信息;S11. The first convolutional neural network estimates the three-dimensional bounding box information and vehicle model information of the vehicle;
S12、利用三维包围框在图像中的位置对点云图进行筛选;S12, using the position of the three-dimensional bounding box in the image to filter the point cloud image;
S13、将筛选之后的毫米波雷达点云图与车辆的三维包围框信息输入到第二卷积神经网络;S13. Input the filtered millimeter-wave radar point cloud image and the three-dimensional bounding box information of the vehicle into the second convolutional neural network;
S14、第二卷积神经网络估计车辆的实际位置信息。S14. The second convolutional neural network estimates the actual position information of the vehicle.
在本实施例中,估计车辆在道路上的实际位置信息的方法具体为:根据获取的毫米波雷达数据产生毫米波雷达数据点云图;利用第一卷积神经网络从图像数据产生车辆的三维包围框,将车辆三维包围框投影到二维图像平面,形成二维包围框,利用车辆二维包围框对毫米波雷达点云进行筛选,筛选出同一车辆的雷达点云;将筛选出的雷达点云与车辆的三维包围框信息输入到第二卷积神经网络,估计出车辆三维中心在道路上的具体位置信息。将检测器的输出编码为JSON格式的字符串,通过网络传输协议WebSocket发送数据到客户端。In this embodiment, the method for estimating the actual position information of the vehicle on the road is specifically: generating a point cloud image of the millimeter-wave radar data according to the acquired millimeter-wave radar data; using the first convolutional neural network to generate a three-dimensional surrounding of the vehicle from the image data Frame, project the vehicle's three-dimensional bounding box onto the two-dimensional image plane to form a two-dimensional bounding box, use the vehicle's two-dimensional bounding box to filter the millimeter-wave radar point cloud, and filter out the radar point cloud of the same vehicle; the screened radar point cloud The three-dimensional bounding box information of the cloud and the vehicle is input to the second convolutional neural network to estimate the specific position information of the three-dimensional center of the vehicle on the road. Encode the output of the detector into a string in JSON format, and send the data to the client through the network transmission protocol WebSocket.
在本实施例中,第一卷积神经网络结合了深层聚合特征提取网络架构,第一卷积神经网络的总体结构如图4所示。In this embodiment, the first convolutional neural network is combined with a deep aggregation feature extraction network architecture, and the overall structure of the first convolutional neural network is shown in FIG. 4 .
图4中各个模块的组成如图5所示,包含已知的神经网络模块:The composition of each module in Figure 4 is shown in Figure 5, including known neural network modules:
Conv : 卷积层;Conv : convolutional layer;
BN : 批标准化/规范化(Batch Normalization);BN : Batch Normalization/Normalization (Batch Normalization);
Relu: 线性整流函数(Rectified Linear Unit);Relu: linear rectification function (Rectified Linear Unit);
Concat:张量拼接;Concat: tensor splicing;
Pooling:池化;Pooling: Pooling;
Full connection:全连接层 ;Full connection: full connection layer;
Flatten:全连接层;Flatten: fully connected layer;
Sigmoid: S型的神经网络的激活函数。Sigmoid: Sigmoid activation function for neural networks.
在本实施例中,第二卷积神经网络的结构如图6所示。In this embodiment, the structure of the second convolutional neural network is shown in FIG. 6 .
S2、将检测器的输出编码为JSON格式的字符串,通过网络传输协议WebSocket发送数据到客户端。S2. Encode the output of the detector into a string in JSON format, and send the data to the client through the network transmission protocol WebSocket.
本实施例中,服务端神经网络所估计的车辆实际位置与车辆型号信息被编码为JSON格式的字符串,然后通过WebSocket网络协议将车辆位置信息和型号信息发送给多个客户端, 参见图7所示。In this embodiment, the actual vehicle position and vehicle model information estimated by the server-side neural network are encoded into a string in JSON format, and then the vehicle position information and model information are sent to multiple clients through the WebSocket network protocol, see Figure 7 shown.
在本实施例中,WebSocket基于TCP协议,能够给客户端与服务端提供可靠的连接。In this embodiment, the WebSocket is based on the TCP protocol and can provide a reliable connection between the client and the server.
S3、建立多个客户端,客户端接收服务端发送的数据并解码,得到所有车辆的型号和位置信息。S3. Multiple clients are established, and the clients receive and decode the data sent by the server to obtain the model and location information of all vehicles.
在本实施例中,基于WebSocket传输协议,服务端的数据可以同时与多个客户端共享所检测到的车辆数据,每个客户端都可以得到车辆的位置与型号信息。其中,服务器将所有车辆在道路上的实际位置和车辆型号信息编码成JSON字符串格式的数据,通过网络传输协议将编码后的数据推送给多个客户端。In this embodiment, based on the WebSocket transmission protocol, the server data can share the detected vehicle data with multiple clients at the same time, and each client can obtain the location and model information of the vehicle. Among them, the server encodes the actual position and vehicle model information of all vehicles on the road into data in JSON string format, and pushes the encoded data to multiple clients through the network transmission protocol.
S4、在客户端建立车辆类型的三维模型数据库,根据得到的车辆型号和位置信息,将模型在相应的位置上显示出来。S4. Establish a 3D model database of the vehicle type on the client terminal, and display the model at a corresponding position according to the obtained vehicle model and position information.
在本实施例中,客户端建立车辆三维模型数据库,参见图8所示,客户端在接收到JSON字符串信息之后,解码数据包获得车辆的型号和位置信息,在三维模型数据库中寻找合适的车辆模型,并且在虚拟的三维空间建立道路模型,根据车辆的实际位置信息将车辆的模型映射到道路模型上进行显示,重新建立道路上车流的实际状况,并全方位地进行展示。In this embodiment, the client establishes a vehicle three-dimensional model database, as shown in Figure 8, after receiving the JSON string information, the client decodes the data packet to obtain the model and location information of the vehicle, and searches for a suitable model in the three-dimensional model database. Vehicle models, and build road models in the virtual three-dimensional space, map the vehicle models to the road model for display according to the actual position information of the vehicles, re-establish the actual conditions of traffic flow on the road, and display them in an all-round way.
本发明实施例中使用了高精度的车辆三维模型以达到更好的可视化效果,用户可以自定义各种类型的车辆模型样式,并且根据客户端硬件性能,用户可以更改模型的精细程度,使得显示的帧率达到理想水平。In the embodiment of the present invention, a high-precision three-dimensional vehicle model is used to achieve a better visualization effect. Users can customize various types of vehicle model styles, and according to the performance of the client hardware, the user can change the fineness of the model so that the display The frame rate reached the ideal level.
其中车辆模型的加载利用了WebGL技术,使得客户端只需要访问特定的网页就可以观看车辆三维重建的场景。The loading of the vehicle model utilizes WebGL technology, so that the client only needs to visit a specific webpage to watch the scene of the 3D reconstruction of the vehicle.
其中,客户端根据解码出来的数据,统计货车、卡车和轿车的数量并展示;客户端利用WebGL,重建道路上车流的三维场景并进行展示,展示内容包括车辆的模型、车辆的位置信息和车辆型号。Among them, the client counts and displays the number of trucks, trucks, and cars based on the decoded data; the client uses WebGL to reconstruct and display the 3D scene of traffic flow on the road. The display content includes vehicle models, vehicle location information and vehicle model.
S5、客户端整合车辆数量统计功能、服务器连接状态显示功能和三维场景显示功能,形成一个完整的车流实时展示终端,参见图9所示。S5. The client terminal integrates the vehicle quantity statistics function, the server connection status display function and the 3D scene display function to form a complete real-time display terminal of traffic flow, as shown in FIG. 9 .
在本实施例中,车辆计数功能可以显示不同类型车辆的数量,在本发明的实施例中,能够显示轿车的数量、卡车的数量和货车的数量。In this embodiment, the vehicle counting function can display the number of different types of vehicles, and in the embodiment of the present invention, it can display the number of cars, trucks and trucks.
本发明实施例中三维场景的展示是360度的全方位展示,用户在客户端可以用过拖拽鼠标完成视角的切换,参见图10、图11所示。The display of the three-dimensional scene in the embodiment of the present invention is a 360-degree omni-directional display, and the user can switch the viewing angle by dragging the mouse on the client terminal, as shown in Fig. 10 and Fig. 11 .
需要特别说明的是,在车辆三维可视化展示时,根据处理后的输出数据,能够将车辆的具体位置、行驶速度、车辆型号和车牌等信息进行展示,其中,根据输出的车辆型号信息在一个汽车三维模型库里筛选出正确的模型。在本实施例中,模型可以分为轿车模型、摩托车模型、货车模型、卡车模型、客车模型。筛选出的模型会结合输出的位置信息,利用WebGL进行三维可视化。It should be noted that in the three-dimensional visual display of vehicles, according to the processed output data, the specific location, driving speed, vehicle model, license plate and other information of the vehicle can be displayed. Among them, according to the output vehicle model information in a car The correct model is selected from the 3D model library. In this embodiment, the models can be classified into car models, motorcycle models, truck models, truck models, and bus models. The screened models will be combined with the output position information and used for 3D visualization using WebGL.
在本发明的其他实施例中,在道路车流三维重建展示时,WebGL可以使用OpenGL进行代替,但是效果一样。In other embodiments of the present invention, WebGL can be replaced by OpenGL when displaying the three-dimensional reconstruction of road traffic flow, but the effect is the same.
本发明提供了一种道路车流三维重建方法,能够降低客户端之间直接共享道路车流视频信息带来的巨大通信开销。利用神经网络从摄像头与毫米波雷达的数据估计车辆的实际位置和车辆型号,以文本的方式在服务端存储下来,并被编码为JSON格式的字符串,通过WebSocket发送给多个客户端,客户端对信息进行解码,根据车辆型号信息从所建立的车辆三维模型库中匹配车辆模型,利用车辆的实际位置信息将车辆模型映射到虚拟道路模型上进行展示。本发明使道路上的车流能够在不共享视频流的情况下被客户端三维重建,降低了信号传输的开销,有利于交通信息的共享和交互,有利于智能交通技术的发展。The invention provides a method for three-dimensional reconstruction of road traffic flow, which can reduce the huge communication overhead caused by directly sharing video information of road traffic flow between clients. Use the neural network to estimate the actual position and vehicle model of the vehicle from the data of the camera and millimeter-wave radar, store it on the server side in the form of text, and encode it into a string in JSON format, and send it to multiple clients through WebSocket. The terminal decodes the information, matches the vehicle model from the established vehicle three-dimensional model library according to the vehicle model information, and uses the actual position information of the vehicle to map the vehicle model to the virtual road model for display. The invention enables the traffic flow on the road to be three-dimensionally reconstructed by the client without sharing the video stream, reduces the overhead of signal transmission, is beneficial to the sharing and interaction of traffic information, and is beneficial to the development of intelligent traffic technology.
应该理解的是,上述虽然是按照某一顺序描述的,但是这些步骤并不是必然按照上述顺序依次执行。除非本文中有明确的说明,这些步骤的执行并没有严格的顺序限制,这些步骤可以以其它的顺序执行。而且,本实施例的一部分步骤可以包括多个步骤或者多个阶段,这些步骤或者阶段并不必然是在同一时刻执行完成,而是可以在不同的时刻执行,这些步骤或者阶段的执行顺序也不必然是依次进行,而是可以与其它步骤或者其它步骤中的步骤或者阶段的至少一部分轮流或者交替地执行。It should be understood that although the above description is in a certain order, these steps are not necessarily executed in sequence in the above order. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, some of the steps in this embodiment may include multiple steps or stages, and these steps or stages are not necessarily executed at the same time, but may be executed at different times, and the order of execution of these steps or stages is also different. It must be performed sequentially, but may be performed alternately or alternately with other steps or at least a part of steps or stages in other steps.
在本发明的一个实施例中,在本发明的实施例中还提供了一种计算机设备,包括至少一个处理器,以及与所述至少一个处理器通信连接的存储器,所述存储器存储有可被所述至少一个处理器执行的指令,所述指令被所述至少一个处理器执行,以使所述至少一个处理器执行所述的道路车流三维重建方法,该处理器执行指令时实现上述方法。In one embodiment of the present invention, a computer device is also provided in the embodiment of the present invention, including at least one processor, and a memory connected to the at least one processor in communication, and the memory stores information that can be The instructions executed by the at least one processor, the instructions are executed by the at least one processor, so that the at least one processor executes the method for three-dimensional reconstruction of road traffic flow, and the above method is realized when the processor executes the instructions.
在本发明的一个实施例中,提供了一种计算机可读存储介质,计算机可读存储介质存储有计算机指令,所述计算机指令用于使所述计算机执行所述的道路车流三维重建方法。In one embodiment of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium stores computer instructions, and the computer instructions are used to make the computer execute the method for three-dimensional reconstruction of road traffic flow.
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程,是可以通过计算机指令表征的计算机程序来指令相关的硬件来完成,所述的计算机程序可存储于一非易失性计算机可读取存储介质中,该计算机程序在执行时,可包括如上述各方法的实施例的流程。其中,本申请所提供的各实施例中所使用的对存储器、存储、数据库或其它介质的任何引用,均可包括非易失性和易失性存储器中的至少一种。Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be realized by instructing related hardware through computer programs represented by computer instructions, and the computer programs can be stored in a non-volatile computer. In the readable storage medium, when the computer program is executed, it may include the processes of the embodiments of the above-mentioned methods. Wherein, any references to memory, storage, database or other media used in the various embodiments provided in the present application may include at least one of non-volatile memory and volatile memory.
非易失性存储器可包括只读存储器、磁带、软盘、闪存或光存储器等。易失性存储器可包括随机存取存储器或外部高速缓冲存储器。作为说明而非局限,RAM可以是多种形式,比如静态随机存取存储器或动态随机存取存储器等。Nonvolatile memory may include read only memory, magnetic tape, floppy disk, flash memory, or optical memory, among others. Volatile memory can include random access memory or external cache memory. By way of illustration and not limitation, RAM can take various forms, such as static random access memory or dynamic random access memory, among others.
综上所述,本发明提供的道路车流三维重建方法、计算机设备及存储介质,可以用于目前已知的应用领域有道路车流的检测与重建展示,以及现在的智能交通、自动驾驶、道路管控、交通规划领域。To sum up, the three-dimensional reconstruction method, computer equipment and storage medium of the road traffic flow provided by the present invention can be used in the detection and reconstruction display of the road traffic flow in the currently known application fields, as well as the current intelligent transportation, automatic driving, and road management and control. , Transportation planning field.
本发明相对于现在的技术而言,优点有两点。第一点是能够在降低车流监测站与用户端之间由于共享道路车辆视频流所造成的巨大通信开销。目前的车流监测站与客户端之间普遍采用直接共享视频数据流的方式来共享道路监测数据,客户端数量的增多会带来巨大的通信开销。本发明利用神经网络对摄像头与毫米波雷达的数据估计车辆的实际位置和车辆型号,并将车辆的实际位置和车辆型号编码为字符串格式的数据,利用网络传输协议WebSocket可靠地传输给客户端,减少了服务端与客户端之间传输数据所带来的通信开销。第二点是道路上的车流能够在客户端被三维重建并展示,这有利于智能交通的发展。目前的道路检测系统的检测信息只能以文本的方式进行一个二维的展示,所展示的不够直观。本发明利用第一点所估计的车辆的位置信息,整合车辆的型号,借助车辆三维模型库进行模型的筛选,根据车辆的实际位置将车辆模型映射到道路模型,可以直观地展示道路车辆状况。Compared with the present technology, the present invention has two advantages. The first point is to reduce the huge communication overhead caused by sharing the video stream of road vehicles between the traffic monitoring station and the user end. At present, the road monitoring data is generally shared between the traffic flow monitoring station and the client by directly sharing the video data stream. The increase in the number of clients will bring huge communication overhead. The present invention uses the neural network to estimate the actual position and model of the vehicle from the data of the camera and millimeter-wave radar, encodes the actual position and model of the vehicle into data in a string format, and uses the network transmission protocol WebSocket to reliably transmit the data to the client , reducing the communication overhead caused by data transmission between the server and the client. The second point is that the traffic flow on the road can be reconstructed and displayed in 3D on the client side, which is conducive to the development of intelligent transportation. The detection information of the current road detection system can only be displayed in two dimensions in the form of text, which is not intuitive enough. The present invention utilizes the position information of the vehicle estimated at the first point, integrates the model of the vehicle, screens the models with the help of the vehicle three-dimensional model library, maps the vehicle model to the road model according to the actual position of the vehicle, and can intuitively display the road vehicle status.
以上所述仅为本发明的较佳实施例而已,并不用以限制本发明,凡在本发明的精神和原则之内所作的任何修改、等同替换和改进等,均应包含在本发明的保护范围之内。The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention should be included in the protection of the present invention. within range.
Claims (8)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202111393942.7A CN114120642B (en) | 2021-11-23 | 2021-11-23 | Method for three-dimensional reconstruction of road traffic flow, computer equipment and storage medium |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202111393942.7A CN114120642B (en) | 2021-11-23 | 2021-11-23 | Method for three-dimensional reconstruction of road traffic flow, computer equipment and storage medium |
Publications (2)
Publication Number | Publication Date |
---|---|
CN114120642A CN114120642A (en) | 2022-03-01 |
CN114120642B true CN114120642B (en) | 2022-12-06 |
Family
ID=80440078
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202111393942.7A Active CN114120642B (en) | 2021-11-23 | 2021-11-23 | Method for three-dimensional reconstruction of road traffic flow, computer equipment and storage medium |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN114120642B (en) |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115082625A (en) * | 2022-06-30 | 2022-09-20 | 北京神目科技有限公司 | Millimeter wave image reconstruction method, image reconstruction model training method and device |
CN116048074A (en) * | 2022-12-28 | 2023-05-02 | 惠州市德赛西威汽车电子股份有限公司 | Obstacle avoidance method, device, equipment, domain controller, storage medium and system |
CN118311535A (en) * | 2022-12-31 | 2024-07-09 | 苏州思卡信息系统有限公司 | System and method for detecting fusion event based on multiple laser radars |
CN117492758A (en) * | 2024-01-03 | 2024-02-02 | 安徽中科星驰自动驾驶技术有限公司 | Automatic driving visualization method based on WebGL |
CN119402820A (en) * | 2024-12-31 | 2025-02-07 | 润芯微科技(江苏)有限公司 | A vehicle interaction system based on LBS and its interaction method |
Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH0981495A (en) * | 1995-07-11 | 1997-03-28 | Sony Corp | Method and system for sharing three-dimensional virtual reality space, medium and method for information recording, medium and method for information transmission, information processing method, and shared server terminal |
Family Cites Families (23)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8930844B2 (en) * | 2000-08-22 | 2015-01-06 | Bruce Carlin | Network repository of digitalized 3D object models, and networked generation of photorealistic images based upon these models |
US7305396B2 (en) * | 2002-12-31 | 2007-12-04 | Robert Bosch Gmbh | Hierarchical system and method for on-demand loading of data in a navigation system |
CN102364523A (en) * | 2011-05-11 | 2012-02-29 | 武汉理工大学 | Realization method of 3D virtual city system based on RIA architecture |
US10868890B2 (en) * | 2011-11-22 | 2020-12-15 | Trimble Navigation Limited | 3D modeling system distributed between a client device web browser and a server |
US9135754B2 (en) * | 2012-05-07 | 2015-09-15 | Honda Motor Co., Ltd. | Method to generate virtual display surfaces from video imagery of road based scenery |
CN102929969A (en) * | 2012-10-15 | 2013-02-13 | 北京师范大学 | Real-time searching and combining technology of mobile end three-dimensional city model based on Internet |
CN104573326A (en) * | 2014-12-16 | 2015-04-29 | 北京中交兴路车联网科技有限公司 | Statistical analysis system of vehicle running data |
CN105679079A (en) * | 2016-02-26 | 2016-06-15 | 同济大学 | Vehicle-mounted machine system facing cooperation of road network and vehicle network |
US10261747B2 (en) * | 2016-09-09 | 2019-04-16 | The Boeing Company | Synchronized side-by-side display of live video and corresponding virtual environment images |
CN106840178B (en) * | 2017-01-24 | 2019-05-03 | 中南大学 | An ArcGIS-based map creation and intelligent vehicle autonomous navigation method and system |
CN107959687A (en) * | 2017-12-13 | 2018-04-24 | 上海曼恒数字技术股份有限公司 | A kind of data processing method, system, equipment and storage medium |
CN109164764A (en) * | 2018-07-27 | 2019-01-08 | 中国兵器装备集团自动化研究所 | Workshop wholegrain degree three-dimensional visualization monitoring system and construction method |
CN110135453A (en) * | 2019-03-29 | 2019-08-16 | 初速度(苏州)科技有限公司 | A kind of laser point cloud data mask method and device |
CN110175576B (en) * | 2019-05-29 | 2022-09-16 | 电子科技大学 | A visual detection method of driving vehicles combined with laser point cloud data |
US10636209B1 (en) * | 2019-09-13 | 2020-04-28 | Bongfish GmbH | Reality-based three-dimensional infrastructure reconstruction |
CN111829535B (en) * | 2020-06-05 | 2022-05-03 | 阿波罗智能技术(北京)有限公司 | Method and device for generating offline map, electronic equipment and storage medium |
CN111934840A (en) * | 2020-06-29 | 2020-11-13 | 北京百度网讯科技有限公司 | Communication method of client and server, gateway, electronic equipment and storage medium |
CN112560580B (en) * | 2020-11-20 | 2022-01-28 | 腾讯科技(深圳)有限公司 | Obstacle recognition method, device, system, storage medium and electronic equipment |
CN112669614A (en) * | 2020-11-27 | 2021-04-16 | 图达通智能科技(苏州)有限公司 | Device and method for acquiring and publishing real-time road condition information |
CN113033412B (en) * | 2021-03-26 | 2023-08-22 | 平安国际智慧城市科技股份有限公司 | Traffic situation monitoring method, device, equipment and medium based on live-action fusion |
CN113240939B (en) * | 2021-03-31 | 2023-03-28 | 浙江吉利控股集团有限公司 | Vehicle early warning method, device, equipment and storage medium |
CN113192182A (en) * | 2021-04-29 | 2021-07-30 | 山东产研信息与人工智能融合研究院有限公司 | Multi-sensor-based live-action reconstruction method and system |
CN113421330B (en) * | 2021-06-21 | 2023-09-29 | 车路通科技(成都)有限公司 | Vehicle-road cooperative road three-dimensional scene construction method, device, equipment and medium |
-
2021
- 2021-11-23 CN CN202111393942.7A patent/CN114120642B/en active Active
Patent Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH0981495A (en) * | 1995-07-11 | 1997-03-28 | Sony Corp | Method and system for sharing three-dimensional virtual reality space, medium and method for information recording, medium and method for information transmission, information processing method, and shared server terminal |
Non-Patent Citations (1)
Title |
---|
车载移动执法中违规车辆智能检测研究;陈刚等;《电子科技大学学报》;20180530(第03期);全文 * |
Also Published As
Publication number | Publication date |
---|---|
CN114120642A (en) | 2022-03-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN114120642B (en) | Method for three-dimensional reconstruction of road traffic flow, computer equipment and storage medium | |
CN110164135B (en) | Positioning method, positioning device and positioning system | |
WO2019191142A1 (en) | Smart area monitoring with artificial intelligence | |
CN115187742B (en) | Method, system and related device for generating automatic driving simulation test scene | |
US11392733B2 (en) | Multi-dimensional event model generation | |
CN113032249B (en) | Test method, device and equipment of traffic flow monitoring system | |
CN114818230A (en) | Simulation test method, device and system | |
US20230097373A1 (en) | Traffic monitoring, analysis, and prediction | |
CN110880235A (en) | Road side equipment in road condition information processing system, processing method and device | |
EP3754449A1 (en) | Vehicle control method, related device, and computer storage medium | |
CN110175533A (en) | Overpass traffic condition method of real-time, device, terminal and storage medium | |
CN112530021A (en) | Method, apparatus, device and storage medium for processing data | |
CN116958763A (en) | Feature-result-level-fused vehicle-road collaborative sensing method, medium and electronic equipment | |
CN112991735B (en) | Test method, device and equipment of traffic flow monitoring system | |
Ahmed et al. | A joint perception scheme for connected vehicles | |
CN115620520A (en) | Method, device, device, medium and product for testing the number of perceived objects | |
CN117542042B (en) | Three-dimensional object detection method and device, electronic equipment and storage medium | |
CN116109698B (en) | Method, device and storage medium for determining coordinate value of target virtual parking space | |
CN113780247B (en) | Traffic light detection method and device, electronic equipment and computer readable medium | |
CN116486059A (en) | Road information collection method and device, storage medium and electronic equipment | |
CN111050153B (en) | Vehicle, vehicle equipment and three-dimensional realization method of vehicle equipment | |
CN116320466A (en) | Video encoding method, device, equipment, storage medium and automatic driving vehicle | |
Huang et al. | Technical and economic feasibility assessment of a cloud-enabled traffic video analysis framework | |
CN115421122A (en) | Target object detection method and device, electronic equipment and readable storage medium | |
Bai et al. | Dynamic feature sharing for cooperative perception from point clouds |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |