TWI712012B - Artificial intelligence traffic detection system - Google Patents

Abstract

The invention relates to an artificial intelligence traffic detection system, which continuously images an intersection image by a fisheye camera installed at an intersection, and analyzes the image by a processor with an artificial intelligence algorithm to obtain traffic information. After the traffic information is transmitted to a server, the server generates a time plan for controlling the traffic sign of the intersection based on the traffic information. The invention can provide traffic information in an instant and continuously without interruption, and helps to instantly generate a time plan which is most suitable for traffic signs of various intersections, thereby improving traffic congestion.

Description

Artificial Intelligence Traffic Detection System

The present invention is a traffic detection system, which refers to a system that uses artificial intelligence to detect traffic conditions.

Traffic congestion is often a difficult problem in metropolitan areas. In addition to the high volume of traffic, traffic congestion also plays a key role in controlling the length of time for vehicles to move and stop. At present, most traffic signs are controlled by traffic controllers set up at intersections. Most of them run on a preset timing plan, and most of the preset time schedules are based on personnel counting on-site traffic flow. After further calculation, the preset time system plan is obtained.

However, it is time-consuming and laborious to count the traffic flow by manpower, and there are errors due to the negligence of the personnel, and it is also difficult to carry out long-term uninterrupted statistics. Furthermore, because it is impossible to deploy manpower for on-site monitoring at any time, it is impossible to adjust the time system of traffic signs in real time with the intersection conditions. Therefore, the prior art does have its drawbacks.

In view of this, the present invention is an improved research and development based on the shortcomings of the prior art, which can detect the traffic conditions at the intersection in real time, and help to adjust the time system of traffic signs in real time.

In order to achieve the above-mentioned object of the invention, the technical means adopted by the present invention is to provide an artificial intelligence traffic detection system, which includes: a fisheye camera for shooting multiple consecutive images of intersections; a processing with artificial intelligence algorithms The processor receives the multiple continuous images, and the processor analyzes the multiple continuous images by the artificial intelligence algorithm to obtain traffic information. The traffic information includes traffic information in different directions such as going straight and turning. The number of various vehicles; and a communication device coupled to the processor for transmitting the traffic information.

The advantage of the present invention is that it can realize long-term monitoring and collect accurate traffic flow information, so as to realize the control of real-time adjustment of traffic signs and improve the traffic jam.

The following describes the technical means adopted by the present invention to achieve the intended purpose of the invention in conjunction with the drawings and the embodiments of the present invention.

Please refer to FIG. 1, the artificial intelligence traffic detection system P of the present invention includes a fisheye camera 10, a processor 20 with artificial intelligence algorithm, and a communication device 30. The artificial intelligence traffic detection system P is used to detect the traffic conditions at the intersection and generate traffic information. The artificial intelligence traffic detection system P transmits the traffic information to the remote server 40. The server 40 generates a timing plan for the traffic signal 60 based on the traffic information, and can transmit the timing plan Give the traffic signal controller 50 that controls the operation of the traffic signal 60.

The fisheye camera 10 has a wide-angle lens for shooting multiple continuous images. The fish-eye camera 10 can be set above the intersection to shoot overhead images. In the embodiment of FIG. 2, the fisheye camera 10 is installed on the light stand 70 of the traffic sign 60 at the intersection, but it is not limited to this. The processor 20 and the communication device 30 are placed in a case 200, and the shell of the case 200 can protect the processor 20 and the communication device 30 from rain or man-made damage. In the embodiment of FIG. 2, the case 200 is arranged on the light stand 70, but it is not limited to this. In one embodiment, the fish-eye camera 10 is a 360-degree panoramic fish-eye camera with 5 million or higher pixels, and can photograph various vehicles turning left, going straight or turning right at the intersection, such as motorcycles, tricycles, Vehicles with more than two wheels, such as small passenger cars, buses, large trucks, coupling cars, etc. In other embodiments, the fisheye camera 10 may also be set on other suitable brackets at the intersection, such as a bracket for road signs, and the chassis 200 containing the processor 20 and the communication device 30 is set on the bracket.

The processor 20 may be a JETSON TX2 platform developed by NVIDIA or other artificial intelligence platforms. Jetson TX2 is an artificial intelligence supercomputer module that adopts the NVIDIA Pascal architecture and supports artificial intelligence computing processing. The processor 20 is coupled to the fisheye camera 10 and has artificial intelligence for receiving multiple continuous images captured by the fisheye camera 10. In one embodiment, the fisheye camera 10 is connected to the processor 20 via a network cable (for example, RJ45), and the fisheye camera 10 transmits images to the processor 20 via the network cable. The processor 20 uses artificial intelligence algorithms to identify different types of vehicles and their traveling directions from the multiple continuous images to obtain traffic information. The traffic information may include the number of various vehicles going straight and turning. For example, the schematic diagram in Fig. 3 includes 8 intersections 1 to 8. The traffic information generated by the processor 20 is a table marked for each intersection, and each set of traffic information provides the type and number of vehicles in the three directions of travel. . 3 to A representative of buses, trucks and other large vehicles to B on behalf of SUVs, small and medium-sized buses and other vehicles to locomotives and other small vehicles on behalf of C, such as traffic information T ₁ show, from crossing from south to north 1 A total of 51 vehicles passed. Among them, 1 large vehicle left and entered junction 5, 4 medium vehicles, 10 small vehicles, 3 large vehicles leaving and entering junction 3, and 20 medium vehicles. There are 6 vehicles and small vehicles, 2 large vehicles, 5 medium vehicles, and 0 small vehicles turning left and entering junction 8. The traffic information T ₂ shows that at junction 3 from south to north, A total of 47 vehicles passed through. Among them, there are 0 large vehicles entering from Junction 7, 8 medium vehicles, 0 small vehicles, 3 large vehicles entering straight from Junction 1, 20 medium vehicles, and small vehicles. There are 6 vehicles. There are 0 large vehicles, 2 medium vehicles, and 8 small vehicles entering right from junction 6, and the traffic information of the junction can be obtained by analogy. In other embodiments, the vehicles recognized by the processor 20 from the fisheye camera 10 include vehicles with more than two wheels, such as motorcycles, tricycles, minibuses, buses, trucks, connected cars, etc.

The fisheye camera 10 and the processor 20 can be installed at every intersection, or only at the intersections 1, 4, 6, and 7. As shown in the figure, for example, the traffic information T _{2 of} intersection 3 can actually be obtained based on the traffic information of intersections 7, 1, and 6. Therefore, it is also feasible to omit the fisheye camera 10 and the processor 20 of intersection 3. By analogy, the fisheye camera 10 at intersections 2, 5, and 8 can also be omitted. If it is a small cross or T-shaped intersection, setting up a 360-degree panoramic fisheye camera 10 and a processor 20 may be sufficient to capture traffic images of all intersections and generate traffic information in all directions. In one embodiment, multiple processors 20 can share a communication device to save cost. In other embodiments, the traffic information generated by the processor 20 may include more content, such as vehicle speed, traffic volume per unit time (for example, hour or minute), and so on.

In the embodiment shown in FIG. 1, the processor 20 includes a Real Time Streaming Protocol (RTSP) image decoder 21, an artificial intelligence judgment unit 22 and a storage device 23. The real-time streaming protocol video decoder 21 is coupled to the storage device 23 and the artificial intelligence judgment unit 22. The input terminal of the real-time streaming protocol image decoder 21 is coupled to the output of the fisheye camera 10 to receive and decode multiple continuous images taken by the fisheye camera 10, and transmit the decoded image data to the artificial intelligence judgment unit twenty two.

The artificial intelligence judgment unit 22 includes an artificial intelligence algorithm and a vehicle database for analyzing multiple consecutive images to generate traffic information. The vehicle database is the result of machine learning and includes characteristic information of various vehicles. The artificial intelligence algorithm compares the vehicle objects in the image with the characteristic information of the vehicle database to identify the type of vehicle. In one embodiment, each type of vehicle corresponds to multiple pieces of feature information, and the multiple pieces of feature information respectively correspond to vehicle images with different degrees of fisheye distortion. For example, in order to perform machine learning, it is necessary to shoot a large number of continuous intersection images at various intersections (such as three-way intersections, crossroads) and various weather situations (such as day, night, clear sky, cloudy, rainy). The intersection image is a fish-eye image taken by the aforementioned fish-eye camera 10. From these fisheye images, images of various vehicles at various locations are used for machine learning to extract feature information. In one embodiment, the process of machine learning using these fisheye images does not restore or correct the distortion of the fisheye images or the distorted vehicle images in the fisheye images. Therefore, in these fisheye images, all fisheye images are caused by fisheye images. The distorted and deformed vehicle images will be used in machine learning to extract feature information. Each type of vehicle corresponds to multiple pieces of characteristic information, and the vehicle image corresponding to each piece of characteristic information has different degrees of distortion. The feature information is extracted from these distorted vehicle images, which will help to quickly identify the vehicle directly from the captured fisheye images in the future.

After receiving the image from the fisheye camera 10, the artificial intelligence judgment unit 22 uses an artificial intelligence algorithm to find all car-like objects in the image, and compares it with the characteristic information in the vehicle database to identify All vehicles in the image. In one embodiment, when the artificial intelligence judgment unit recognizes a vehicle from the fisheye image, it gives the vehicle a mark. For example, a recognition frame 80 shown in FIG. 4 includes the vehicle. With the aid of the mark and multiple continuous fisheye images, the artificial intelligence judgment unit 22 can track whether the vehicle is going straight, turning left or turning right.

The artificial intelligence judgment unit 22 uses artificial intelligence algorithms to identify the types of vehicles in the image (such as motorcycles, tricycles, minibuses, buses, trucks, connected cars, etc.) and identify the direction of travel of various vehicles (such as turning left and right). Turn and go straight). According to the recognition result, the artificial intelligence judgment unit 22 separately counts the number of vehicles in each direction of travel of various vehicles to generate traffic information. In other embodiments, according to the frame rate of the fisheye camera 10 (a few shots per second), the artificial intelligence judgment unit 22 can further calculate the speed of the vehicle, the traffic volume per unit time, the occupancy rate of the intersection, etc. Data and integrated into the traffic information.

For example, the calculation method of the vehicle speed may be to preset a distance measurement range in the fisheye image taken by the fisheye camera 10, the length of the distance measurement range is M pixels, and the length of the M pixels corresponds to The actual road distance N. By tracking the vehicle object and calculating how many consecutive fisheye images have passed, the vehicle object’s position in the image can be calculated through the range of the distance measurement range, for example, in 60 continuous images In the fisheye image of, the position of a vehicle object has moved the distance measurement range, the actual road distance corresponding to the distance measurement range is 20 meters, and the frame rate of the fisheye camera 10 is 30 images per second , In other words, the time for a vehicle object to move the ranging range is 2 seconds (60/30). Divide the distance of 20 meters by the travel time of 2 seconds to get the speed of the vehicle object as 10 meters per second. The traffic volume per unit time can be calculated based on the processor's statistics of the total number of vehicles passing through the intersection every 20 seconds. Intersection occupancy rate (Occupancy) is a piece of information used to evaluate whether there is a traffic jam. The calculation of intersection occupancy rate can be to preset a surveillance area in the intersection image taken by the fisheye camera 10, and the surveillance area can correspond to, for example, the center of the intersection. Area. By calculating the number of images in one minute of images, how many images include vehicle objects in the surveillance area, the intersection occupancy rate can be calculated. For example, the frame rate of the fisheye camera 10 is 30 images per second, and 1 minute is 1800 images. If there are 180 images in the 1800 continuous images, the surveillance area includes vehicle objects. , Then the intersection occupancy rate in the surveillance area is 180 divided by the number of images including vehicle objects 180 divided by the number of images produced by the fisheye camera 10 in 1 minute, which is equal to 10/100. The intersection occupancy rate can be used by the server 40 as one of the information for evaluating traffic congestion. For example, when the occupancy rate of an intersection reaches a threshold value (for example, 100/100), it means that there are continuous cars passing through the central area of the intersection, and traffic jams may have occurred, and the server 40 can use the intersection occupancy rate Immediately adjust the time control of each traffic signal at the intersection to maintain smooth traffic in all directions.

The storage device 23 is used to temporarily store the traffic information generated by the artificial intelligence judgment unit 22. In one embodiment, the storage device 23 further stores necessary images and historical traffic information. The storage device 23 may be a storage medium such as a hard disk or an SSD hard disk.

The communication device 30 is coupled to the processor 20 for transmitting traffic information generated by the processor 20. In one embodiment, the processor 20 includes a network interface (not shown in the figure) coupled to the communication device 30, and the communication device 30 is connected to the network interface by a network cable (for example, an RJ45 network cable). The traffic information temporarily stored in the storage device 23 is transmitted to the communication device 30 via the network interface. The communication device 30 can communicate with the server 40 via a wired network or a wireless network to transmit the traffic information. In one embodiment, the communication device 30 is a router.

The server 40 receives the traffic information from the communication device 30 and generates a timing plan for the traffic sign 60 based on the traffic information. The server 40 may be an existing or newly installed system of the traffic unit. The traffic signal 60 is controlled by the traffic signal controller 50. The server 40 can transmit the time schedule to the traffic signal controller 50 via a wired network or a wireless network, and the traffic signal controller 50 receives the time schedule. After drawing, control traffic sign 60 according to the time system plan. In one embodiment, the time schedule system includes time control of the traffic signal 60, such as the number of seconds for the green light to go straight, the number of seconds for the green light to turn right, the number of seconds for the green light to turn left, the seconds for the red light, and the yellow light. The number of seconds and so on.

It can be understood from the above description that the artificial intelligence traffic detection system P of the present invention uses the fisheye camera 10 to monitor the image of the intersection and the artificial intelligence image analysis, which can produce big data and can judge the direction of rotation (such as turning left or right) ) Traffic flow and identify the locomotive (this is a function that cannot be done with radar detection at present).

An advantage of the above embodiment is that the image captured by the fisheye camera 10 is analyzed by the processor 10 located at the near end of the fisheye camera 10, and the communication device 30 only transmits the traffic information generated by the processor 20 to the remote servo The traffic information has a small amount of data, so the bandwidth requirements for the network between the servers 40 of the communication device 30 are lower, which helps to reduce the cost of building the network and improve the reliability of data transmission And the speed is conducive to real-time adjustment of the time system plan. If the image of the fisheye camera 10 is transmitted to a server in the cloud for processing, a considerable network bandwidth will be required. In addition to increasing the cost of building the network, it may also cause image transmission failure due to network congestion.

In the schematic diagram of FIG. 5, processors 20 at multiple intersections share a communication device 30. In FIG. 5, intersections A, B, and C are respectively provided with a traffic sign 60, a fisheye camera 10, and a processor 30. The intersections A, B, and C may be three intersections on a road, and the processors 30 at these three intersections are all connected to a communication device 30. The traffic information generated by the three processors 20 is transmitted by the communication device 30 to the existing or newly installed server 40 of the remote traffic unit. The server 40 plans the traffic number of each intersection based on the obtained traffic information of each intersection The best time system plan of 60 is recorded, and the time system plan can be transmitted to the traffic signal controller 50 to control the traffic signal 60 of each intersection A, B, and C. In order to improve the traffic of a region or a whole road, the traffic flow of multiple intersections must be considered comprehensively. The architecture shown in Figure 4 is based on the traffic flow of multiple related intersections to plan the traffic signal timing system of each intersection. Optimize the overall traffic speed.

From the above description, it can be understood that the artificial intelligence traffic detection system P according to the present invention can instantly reflect the traffic situation of different traffic periods, which is helpful to optimize the control of the existing traffic signs 60 of the traffic unit and optimize the traffic flow speed. For example, when the traffic information output by the processor 20 shows that the traffic flow on the east-west road is low and the traffic flow is high in the north-south direction, the server 40 can automatically shorten the green light time of the east-west direction and extend the green light time of the north-south road to relieve the problem. Traffic jam problem. On the other hand, the long-term collection of traffic information can analyze the causes of traffic congestion, which is helpful for traffic management and dispatching and related decision-making, and establish an overall macroscopic traffic plan.

One of the characteristics of the present invention is that it uses artificial intelligence to identify vehicles, which not only has a high accuracy rate, but also can accurately identify two-wheeled vehicles with small volumes such as locomotives. If the vehicle object is found from the intersection image by means of computer image analysis, it is easy to reduce the accuracy of vehicle recognition due to weather and light, and the effect is obviously worse than that of the present invention.

The above are only the embodiments of the present invention and do not limit the present invention in any form. Although the present invention has been disclosed as above in the embodiments, it is not intended to limit the present invention. Anyone with ordinary knowledge in the relevant technical field, Without departing from the scope of the technical solution of the present invention, when the technical content disclosed above can be used to make slight changes or modification into equivalent embodiments with equivalent changes, but any content that does not depart from the technical solution of the present invention is based on the technical essence of the present invention Any simple modifications, equivalent changes and modifications made to the above embodiments still fall within the scope of the technical solutions of the present invention.

P‧‧‧Artificial intelligence traffic detection system 10‧‧‧Fisheye camera 20‧‧‧Processor 21‧‧‧Real-time streaming protocol video decoder 22‧‧‧Artificial intelligence judgment unit 23‧‧‧Storage device 30‧ ‧‧Communication device 40‧‧‧Server 50‧‧‧Traffic signal controller 60‧‧‧Traffic signal 70‧‧‧Light stand 80‧‧‧Identification frame 100‧‧‧Image 200‧‧‧Chassis 1, 2, 3, 4, 5, 6, 7, 8‧‧‧Intersection T ₁ , T ₂ ‧‧‧Traffic information

Figure 1 illustrates an embodiment of the invention. Fig. 2 is a schematic diagram illustrating the installation positions of various components of the present invention. Figure 3 is a schematic diagram of the traffic information of the present invention. Figure 4 is a schematic diagram illustrating that the present invention can mark vehicles in images. Figure 5 is a schematic diagram of the present invention applied to multiple intersections.

P‧‧‧Artificial Intelligence Traffic Detection System

10‧‧‧Fisheye Camera

20‧‧‧Processor

21‧‧‧Real-time streaming protocol video decoder

22‧‧‧Artificial Intelligence Judgment Unit

23‧‧‧Storage device

30‧‧‧Communication device

40‧‧‧Server

50‧‧‧Traffic signal controller

60‧‧‧Traffic Sign

Claims (15)

An artificial intelligence traffic detection system, which includes: a fish-eye camera for shooting multiple consecutive images of an intersection; a processor with an artificial intelligence algorithm that receives the multiple consecutive images, and the processor uses The artificial intelligence algorithm recognizes different types of vehicles and their traveling directions from the multiple consecutive images to obtain traffic information, the traffic information including the number of vehicles traveling straight and the number of vehicles turning; and A communication device, coupled to the processor, for transmitting the traffic information; wherein the number of the various vehicles traveling straight and the number of the various vehicles turning are not the number obtained by identifying vehicles on lanes in multiple directions . The artificial intelligence traffic detection system according to claim 1, wherein the fish-eye camera is a 360-degree panoramic fish-eye camera. The artificial intelligence traffic detection system according to claim 2, wherein the fish-eye camera is set above an intersection to shoot a top view image. The artificial intelligence traffic detection system according to claim 1, wherein the processor includes: a real-time streaming protocol image decoder that receives and decodes the multiple continuous images; and an artificial intelligence judgment unit coupled to the real-time Streaming protocol image decoder, the artificial intelligence judgment unit receives the decoded multiple continuous images, and uses the artificial intelligence algorithm to analyze the decoded multiple continuous images to identify a variety of different vehicles and their The direction of travel, and count the number of different vehicles in different directions to generate the traffic information. The artificial intelligence traffic detection system according to claim 4, wherein the processor includes a storage device for temporarily storing the traffic information. The artificial intelligence traffic detection system according to claim 1, wherein the fisheye camera is installed on a light stand of a traffic sign, or on other supports at the intersection. The artificial intelligence traffic detection system according to claim 6, wherein the processor and the communication device are arranged in a case on the light stand or the bracket. The artificial intelligence traffic detection system according to claim 1, wherein the communication device transmits the traffic information to a server via a wireless or wired network. The artificial intelligence traffic detection system according to claim 1, wherein the processor further marks the recognized vehicle with a recognition frame. The artificial intelligence traffic detection system according to claim 1, wherein the traffic information further includes at least one of vehicle speed, traffic volume per unit time, and intersection occupancy rate. The artificial intelligence traffic detection system according to claim 1, wherein the vehicle identified by the processor from the plurality of consecutive images includes a motorcycle or a tricycle. The artificial intelligence traffic detection system according to claim 1, wherein the processor performs vehicle identification based on the fisheye image captured by the fisheye camera, and does not recognize the fisheye image or the fisheye image Deformation correction of vehicle objects. The artificial intelligence traffic detection system according to claim 1, wherein the fisheye camera is connected to the processor through a network cable. The artificial intelligence traffic detection system according to claim 4, wherein the artificial intelligence judgment unit includes the artificial intelligence algorithm and a vehicle database, and the vehicle database includes characteristic information of various vehicles for the artificial intelligence algorithm to compare For the vehicle object in the image. The artificial intelligence traffic detection system according to claim 14, wherein in the vehicle database, each type of vehicle corresponds to multiple pieces of the characteristic information, and the multiple pieces of the characteristic information correspond to vehicles with different degrees of fisheye deformation. image.

Images