KR102097673B1 - System and method for obtaining information of vehicle using side image thereof - Google Patents

Abstract

The preset invention relates to a vehicle information obtaining system which comprises a camera for photographing a side image of a vehicle passing the road and a controller for obtaining information on the side length, the number of axes, and a wheelbase of the vehicle using the side image of the vehicle photographed by the camera. According to the present invention, by using the side image photographed by the camera, the information on the length, the number of axes, and a wheelbase of the vehicle can be obtained so that installation costs and maintenance costs can be reduced compared to the conventional method of embedding a sensor.

Description

Vehicle information acquisition system and method using vehicle side image {System and method for obtaining information of vehicle using side image thereof}

The present invention relates to a vehicle classifying apparatus, and more particularly, to a technology for obtaining vehicle information including side length, number of axes, and wheelbase of a vehicle.

2. Description of the Related Art In general, a vehicle classifier is installed on roads such as national roads, local roads and highways, and toll roads including highways for the purpose of investigating traffic or collecting tolls for each vehicle type.

Vehicle detection for ITS (Intelligent Transportation Systems) and automatic vehicle classification (AVC) for road traffic survey are installed in a buried or non-burden type on the road, and the traffic volume, speed, occupancy, vehicle length, etc. of passing vehicles The data is collected in real time and transmitted to the traffic control center, and the traffic control center guides the traffic volume for each section in real time through electronic signs and various service media based on the transmitted data.

These road traffic survey devices are installed and operated on toll gates, highways, and national highways.

Conventional vehicle classifiers have been developed and used in various ways, including contact and non-contact methods, but can be generally divided into buried and non-buried types.

1 is a view showing a conventional buried vehicle classifier.

Referring to FIG. 1, a conventional buried vehicle classifier uses a combination of a plurality of sensors using a loop sensor and a piezo sensor to detect the passage of the vehicle 10, classify the vehicle model, and ensure accuracy. In the example of FIG. 1, two piezoelectric sensors 20 and a loop sensor 30 are embedded.

In the conventional buried vehicle classifier, a plurality of buried sensors 20 and 30 are used to detect the entry and exit of a vehicle, to calculate the speed of a passing vehicle, and of the tire contacting the buried sensors 20 and 30. The vehicle is classified using the number of wheels and wheelbase detected by pressure, and wheel width and wheelbase information.

In the case of such a vehicle type classification device, a construction cost is incurred due to the embedding of multiple sensors on the road, problems such as road damage caused by construction are occurring, and the entire system is required because a plurality of sensors are required for accurate classification. There is a disadvantage of increasing the complexity.

In addition, due to the continuous contact of the tire to the buried sensor, the durability of the sensor is consumed, maintenance is required, and inconvenience such as having to cut off traffic on the road and perform civil engineering work on the road for maintenance and replacement of the sensor. It causes a problem that there is an expensive cost for maintenance.

Republic of Korea Patent Publication 10-2014-0027661

The present invention has been devised to solve the above problems, and an object thereof is to provide a system and method for acquiring side length, number of axes, and wheelbase information of a vehicle using a side image of the vehicle.

The objects of the present invention are not limited to the above-mentioned objects, and other objects not mentioned will be clearly understood by those skilled in the art from the following description.

The vehicle information acquisition system of the present invention for achieving the above object is a camera for photographing a side image of a vehicle passing through the road and a side length, number of axles, and wheelbase information of the vehicle using a side image of the vehicle captured by the camera It includes a controller for obtaining.

The camera photographs the side of the vehicle passing through the road, and generates a vehicle side image including the entire view of the side of the vehicle, and the controller uses the vehicle side image to obtain side length, number of axes, and wheelbase information of the vehicle. can do.

The controller may detect a vehicle area through image recognition using machine learning from the vehicle side image, and calculate side length, number of axes, and wheelbase information from the detected vehicle region.

The controller matches 1: 1 the actual coordinates calculated based on the actual information on the real area including the road section photographed by the camera and the virtual coordinates of the vehicle side image area captured by the camera, and machine learning. The vehicle area may be detected from the vehicle side image through the image recognition technique using, and the coordinates of the detected vehicle region may be calculated, and the vehicle length may be calculated using the front and rear coordinates of the vehicle in the vehicle region. .

The controller detects a tire object area from the vehicle side image using an image recognition technique using machine learning, calculates coordinates of the detected tire object area, recognizes the number of detected tire object areas as the number of axes, and ends Wheelbase information may be calculated using coordinates of each tire object area located at.

The controller may be implemented to generate a virtual trigger line on a screen transmitted by the camera, recognize that the vehicle has entered when the vehicle reaches the trigger line, and capture a side image of the vehicle through the camera.

In a vehicle information acquisition method in a vehicle information acquisition system including a camera and a controller of the present invention, the side image of a vehicle passing through the road is photographed by the camera, and the side image of the vehicle photographed by the camera is captured by the controller. And obtaining side length, number of axes, and wheelbase information of the vehicle.

The camera photographs the side of the vehicle passing through the road, and generates a vehicle side image including the entire view of the side of the vehicle, and the controller uses the vehicle side image to obtain side length, number of axes, and wheelbase information of the vehicle. can do.

The controller may detect a vehicle area through image recognition using machine learning from the vehicle side image, and calculate side length, number of axes, and wheelbase information from the detected vehicle region.

The controller, the 1: 1 coordinates of the actual coordinates calculated based on the actual information about the real area including the road section photographed by the camera, and the virtual coordinates of the vehicle side image area photographed by the camera, Detecting a vehicle area from the side image of the vehicle through image recognition using machine learning, calculating coordinates of the detected vehicle region, and calculating vehicle length using the front and rear coordinates of the vehicle in the vehicle region Steps can be performed.

The controller detects a tire object area in the vehicle side image using an image recognition technique using machine learning, calculates coordinates of the detected tire object area, and recognizes the detected number of tire object areas as the number of axes And calculating wheelbase information using coordinates of each tire object area located at both ends.

The controller may be implemented to generate a virtual trigger line on a screen transmitted by the camera, recognize that the vehicle has entered when the vehicle reaches the trigger line, and capture a side image of the vehicle through the camera.

According to the present invention, by using the side image of the vehicle photographed by the camera, it is possible to obtain the length, number of axes, and wheelbase information of the vehicle, so that it is possible to reduce installation and maintenance costs compared to a conventional sensor embedding method. There is an effect.

In addition, according to the present invention, it is possible to detect a variable axis of a vehicle that cannot be detected in an existing system, and there is an effect of improving accuracy in classifying a vehicle.

In addition, in the present invention, by acquiring a side image of a vehicle passing through a road, it can be used as a traffic data, and furthermore, it can be used in a traffic measurement and vehicle classification system, which is essential in ITS, to contribute to the demand forecast and road management of new roads. Is expected.

1 is a view showing a conventional buried vehicle classifier.
Figure 2 shows the configuration of a vehicle information acquisition system according to an embodiment of the present invention.
3 is a view showing a camera installation in the vehicle information acquisition system according to an embodiment of the present invention.
4 is a flowchart illustrating a method for obtaining vehicle information according to an embodiment of the present invention.
5 is a view for explaining a process of matching the coordinates of the real area and the coordinates of the virtual area in the vehicle information acquisition system according to an embodiment of the present invention.
6 is a view for explaining a process of recognizing a vehicle and a tire object area and calculating coordinates in a vehicle information acquisition system according to an embodiment of the present invention.
7 shows homography.
8 is a diagram illustrating an image area used for learning for tire object recognition in a vehicle information acquisition system according to an embodiment of the present invention.

The present invention can be applied to various changes and can have various embodiments, and specific embodiments will be illustrated in the drawings and described in detail. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention.

The terms used in this application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, terms such as “include” or “have” are intended to indicate that a feature, number, step, operation, component, part, or combination thereof described on the specification exists, and that one or more other features are present. It should be understood that the existence or addition possibilities of fields or numbers, steps, operations, components, parts or combinations thereof are not excluded in advance.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person skilled in the art to which the present invention pertains. Terms, such as those defined in a commonly used dictionary, should be interpreted as having meanings consistent with meanings in the context of related technologies, and should not be interpreted as ideal or excessively formal meanings unless explicitly defined in the present application. Does not.

In addition, in the description with reference to the accompanying drawings, the same components are given the same reference numerals regardless of the reference numerals, and redundant descriptions thereof will be omitted. In the description of the present invention, when it is determined that detailed descriptions of related known technologies may unnecessarily obscure the subject matter of the present invention, detailed descriptions thereof will be omitted.

Figure 2 shows the configuration of a vehicle information acquisition system according to an embodiment of the present invention.

2, the vehicle information acquisition system of the present invention includes a camera 110, the controller 120.

The camera 110 serves to photograph a side image of a vehicle passing through the road.

The controller 120 acquires side length, number of axes, and wheelbase information of the vehicle using the side image of the vehicle photographed by the camera 110. Here, the number of axes means the number of axes of the vehicle, and the wheelbase means the distance between the axes of the vehicle.

In the present invention, the camera 110 photographs a side of a vehicle passing through the road, and generates a vehicle side image including the entire shape of the side of the vehicle. In the present invention, the vehicle side image generated by the camera 110 may be a 2D (dimension) image.

Then, the controller 120 acquires side length, number of axes, and wheelbase information of the vehicle using the vehicle side image.

The controller 120 may detect a vehicle region through an image recognition technique using machine learning from a vehicle side image, and calculate side length, number of axes, and wheelbase information from the detected vehicle region. For example, a vehicle area may be detected through an image recognition technique using a convolution neural network (CNN) technique.

The controller 120 matches the actual coordinates calculated based on the actual information about the real area including the road section photographed by the camera 110 and the virtual coordinates of the vehicle side image area photographed by the camera on a 1: 1 basis. . Then, through the image recognition technique using machine learning, the vehicle area is detected from the vehicle side image, the coordinates of the detected vehicle area are calculated, and the vehicle length is calculated using the front and rear coordinates of the vehicle in the vehicle area. You can.

Then, the controller 120 detects the tire object area from the vehicle side image using the image recognition technique using machine learning, calculates the coordinates of the detected tire object area, and recognizes the detected number of the tire object areas as the number of axes. Then, wheelbase information may be calculated using coordinates of each tire object area located at both ends.

As shown in FIG. 2, the controller 120 generates a virtual trigger line 130 on a screen transmitted by the camera 110, and when the vehicle 10 reaches the trigger line 130, the vehicle enters It can be recognized and controlled to take a side image of a vehicle through the camera 110. That is, when a traffic vehicle enters the road, the virtual trigger line 130 set in advance in the image of the camera 110 is reached, and the controller 120 monitors the image histogram change amount of the virtual trigger area in real time. The entry is detected.

3 is a view showing a camera installation in the vehicle information acquisition system according to an embodiment of the present invention.

3 is a view seen from the front of the vehicle 10, the camera 110 is to take a side image of the vehicle from the side of the vehicle 10.

4 is a flowchart illustrating a method for obtaining vehicle information according to an embodiment of the present invention.

Referring to FIG. 4, in the method for acquiring vehicle information of the present invention, first, the camera 110 photographs a side image of a vehicle passing through the road, and detects the entry of the vehicle using a virtual trigger line 130 ( S401, S403).

Then, the camera 110 photographs the side of the vehicle passing through the road, and generates a vehicle side image including the entire shape of the vehicle side (S405).

Then, the controller 120 detects the vehicle area by using machine learning on the vehicle side image (S407).

Then, the controller 120 calculates the side length of the vehicle by using the coordinate values of the beginning and end of the detected vehicle area (S409).

Then, the controller 120 recognizes the tire object from the vehicle side image by using machine learning (S411).

Then, the controller 120 calculates the number of wheels and the wheelbase of the vehicle using the coordinate values of the tire object (S413).

In the present invention, the controller 120 displays the actual coordinates calculated based on the actual field information including the road section photographed by the camera 110 and the virtual coordinates of the vehicle side image area photographed by the camera 110. Matching 1: 1 and detecting the vehicle area from the side image of the vehicle through image recognition using machine learning, calculating coordinates of the detected vehicle area, and using the front and rear coordinates of the vehicle in the vehicle area By performing the step of calculating the length of the vehicle.

The controller 120 detects the tire object area from the vehicle side image using an image recognition technique using machine learning, calculates coordinates of the detected tire object area, and uses the detected number of tire object areas as the number of axes. A step of recognizing and calculating wheelbase information using coordinates of each tire object area located at both ends may be performed.

In the present invention, the process of calculating the length of a vehicle using machine learning and the process of obtaining the number of wheels and the wheelbase by recognizing a tire object are as follows.

5 is a view for explaining a process of matching the coordinates of the real area and the coordinates of the virtual area in the vehicle information acquisition system according to an embodiment of the present invention.

Referring to FIG. 5, in the present invention, a section area 510 of a road to be photographed by the camera 110 is measured in advance to obtain a real coordinate system. Then, the previously measured coordinates are matched 1: 1 with the coordinates 520 of the virtual area in the image captured by the camera 110 to perform the task of matching the coordinate system in the image with the actual road coordinate system. do.

6 is a view for explaining a process of recognizing a vehicle and a tire object area and calculating coordinates in a vehicle information acquisition system according to an embodiment of the present invention.

Referring to FIG. 6, a side image of a vehicle photographed by the camera 110 is illustrated, and an XY coordinate system is applied to the image, from (0,0) to (x, y) using the lower left vertex as the origin. Is transformed into the coordinate system.

Then, the vehicle area 620 is acquired from the vehicle side image photographed by the camera 110 using machine learning, and coordinates for the vehicle area 620 are obtained.

In FIG. 6, (x1, y1) is the coordinates of the front of the vehicle, and (x2, y2) is the coordinates of the rear of the vehicle. Therefore, x2-x1 becomes the side length of the vehicle, and it can be converted to actual coordinates to obtain the actual length.

In addition, in FIG. 6, the controller 120 may recognize each tire object area 630 and 640 by using machine learning, and obtain coordinates thereof. At this time, it is possible to recognize the number of axes by the number of tire object areas obtained in the vehicle area 620. In the embodiment of Fig. 6, the number of axes is two.

Among the coordinates obtained in each tire object area 630 and 640, a wheelbase that is an inter-axis distance may be obtained using (x3, y3) coordinates and (x5, y5) coordinates. That is, the value of x5-x3 becomes the wheelbase.

The method of obtaining the side length, the number of axes, and the wheelbase from the side image of the vehicle using the machine learning technique in the present invention will be described in more detail as follows.

In the present invention, calculating the side length of the vehicle from the side image of the vehicle uses a method used in image geometry, which is described as follows.

Using the camera 110 as a reference point, pixel coordinates in an image are substituted for coordinates (actual length) of the ground (road) previously measured. This is a technique using homography, and the homography is as follows.

7 shows homography.

Referring to FIG. 7, when a plane is projected to another plane by using homography, a certain transformation relationship is established between the projected corresponding points, and this transformation relation is called homography.

Homography is represented by a 3x3 matrix and is a transformational relationship established for the homogeneous coordinate representation of corresponding points. That is, points on one plane (x1, y1), (x2, y2), ... are projected onto points on another plane (x1 ', y1'), (x2 ', y2'), ... respectively. If possible, there is always a 3x3 homography matrix H satisfying the following relation between these corresponding points, and there is only one.

Homography is a method that is established when applied to a planar object in a 2D image. So, it can be said to be a suitable technique for measuring the length of one side of the vehicle of the present invention.

The essential content of using homography in the present invention is to obtain the corresponding ground coordinates for some sampled image coordinates through measurement, and then apply the general conversion of the obtained image coordinates to the ground coordinates in other cases.

In the present invention, after measuring the ground coordinates of four points on the actual ground in accordance with the criteria (eg, camera position, etc.) the user has caught, the image coordinates (pixel coordinates in the camera image) of the corresponding points are obtained.

For the vehicle entering the preset section, the entire image of the vehicle is photographed.

Then, the area (pixel coordinates) of the vehicle is obtained by image recognition using machine learning from the acquired image.

Then, as illustrated in FIG. 6, the side length of the vehicle is calculated from the first coordinates (x1, y1) and the last coordinates (x2, y2) of the vehicle area.

x2-x1 (length of vehicle area) = total length of vehicle

Then, the length obtained by the virtual coordinate system is substituted for the actual coordinates to obtain the length of the actual vehicle.

The method of recognizing a tire as an object in a vehicle image obtained in the present invention uses a machine learning technique using neural network learning.

1) First, after obtaining the digital images of the side of various vehicles, define the area of the tire to be learned in each image as coordinates and save it as a digital file in xml format. At this time, not only learning the entire vehicle image, but learning only using a portion of the vehicle image (the area including the tire) is the characteristic of the algorithm and the key to improving recognition accuracy.

8 is a diagram illustrating an image area used for learning for tire object recognition in a vehicle information acquisition system according to an embodiment of the present invention.

Referring to FIG. 8, in the present invention, the area including the tire is x2-x1 in length, and the height H is defined as 'diameter + α of the largest tire + α'.

2) Neural network learning (deep learning) is performed using a side image of a vehicle and an xml file predefined in 1) on a device that can perform neural network learning.

3) The neural network learning used here utilizes the CNN technique used for image recognition, and among the CNNs, a single shot multibox detector (SSD) model is used to train various types of tire side images, and the learned weight is calculated. Output.

4) CNN, a kind of neural network learning, can learn by extracting the feature points of the shape of the tires specified in the image input for learning, and then infer the region (coordinate) of the shape having similar feature points in other images.

5) By taking advantage of the sufficiently learned result weight, entering the detection area and applying it to the side image of the photographed vehicle, comparing the feature points in the input image, inferring the pixel coordinates of the tire area, inferring the bounding box of the area (bounding box).

6) All tires expressed in the photographed vehicle image area are recognized as objects (Bounding box), the respective coordinate values are obtained, the total number of tires is calculated, and the number of axes is calculated.

7) As shown in FIG. 6, coordinates of each recognized tire area are obtained as (x3, y3), (x5, y5).

8) The pixel coordinates of the tire are substituted into the previously measured image coordinate system used to obtain the length of the vehicle.

9) Calculate the distance between the coordinates of each tire area.

x5-x3 = distance between tires

10) The calculated distance becomes the wheelbase between each axis.

The data of the number of axes, wheelbase and vehicle length obtained in the present invention can be used for classifying the vehicle.

As described above, the vehicle information acquisition system using the vehicle side image of the present invention can perform vehicle classification using only the image acquired through the camera without installing a separate sensor. In addition, through machine learning using neural network learning (deep learning), tires can be recognized as objects in side images of a traffic vehicle, and the number of wheels and wheelbases can be calculated.

Through this, it is possible to recognize a variable axis (axle that can be lifted and lowered) that was not recognized by the existing contact sensor, thereby improving the accuracy of vehicle classification compared to the existing system.

In addition, it can be used as additional traffic data by acquiring the vehicle side image, and after installation, it is possible to further improve the accuracy of vehicle classification by just updating the software, or to use the image acquired by the camera in addition to the vehicle classification to add other information. Can be extracted.

Although the present invention has been described using several preferred embodiments, these embodiments are illustrative and not limiting. Those skilled in the art to which the present invention pertains will understand that various changes and modifications can be made without departing from the spirit of the present invention and the scope of the rights set forth in the appended claims.

110 Camera 120 Controller

Claims (12)

A camera for photographing a side image of a vehicle passing through the road; And
A controller for acquiring side length, number of axes and wheelbase information of a vehicle using a side image of a vehicle photographed by the camera,
The camera photographs the side of the vehicle passing through the road, and generates a vehicle side image including the entire view of the vehicle side,
The controller detects a vehicle area through an image recognition technique using machine learning from the vehicle side image, calculates side length, number of axes, and wheelbase information from the detected vehicle region,
The controller matches 1: 1 the actual coordinates calculated based on the actual information about the real area including the road section photographed by the camera, and the virtual coordinates of the vehicle side image area photographed by the camera, and performs machine learning. The vehicle area is detected from the vehicle side image through the image recognition technique used, the coordinates of the detected vehicle region are calculated, and the vehicle side length is calculated using the front and rear coordinates of the vehicle in the vehicle region,
The controller detects a tire object area from the side image of the vehicle using an image recognition technique using machine learning, calculates coordinates of the detected tire object area, recognizes the number of detected tire object areas as the number of axes, and ends Wheelbase information is calculated using the coordinates of each tire object area located at,
The controller generates a virtual trigger line on the screen transmitted by the camera, recognizes that the vehicle has entered when the vehicle reaches the trigger line, and attempts to capture a side image of the vehicle through the camera. The trigger line arrival is detected through monitoring the amount of change in the image histogram of the virtual trigger line area,
The controller defines a part of the vehicle side image captured by the camera as a coordinate and stores it as a digital file in xml format, wherein the part of the vehicle is calculated using the front and rear coordinates of the vehicle. This is an area that has a height that is longer than the side length and tire diameter, and deep learning is performed using a single shot multibox detector (SSD) model for the vehicle side image and the digital file, and the result value learned through deep learning ( weight), compare the feature points using the result from the input side image of the vehicle, infer pixel coordinates in the tire area accordingly, calculate a bounding box in the tire area, and calculate The bounding box is recognized as a tire object, coordinate values for each tire object are obtained, and the entire tire object Vehicle information acquisition system to obtain the variable calculating the bearing information, and calculates the wheelbase information in a manner of calculating the distance between each tire object coordinates characterized.
delete delete delete delete delete In a vehicle information acquisition method in a vehicle information acquisition system including a camera and a controller,
Photographing a side image of a vehicle passing through the road in the camera; And
The controller comprises using the side image of the vehicle photographed by the camera to obtain the side length, number of axes and wheelbase information of the vehicle,
The camera photographs the side of the vehicle passing through the road, and generates a vehicle side image including the entire view of the vehicle side,
The controller detects a vehicle area through an image recognition technique using machine learning from the vehicle side image, calculates side length, number of axes, and wheelbase information from the detected vehicle region,
The controller may match the actual coordinates calculated on the basis of the actual information on the real area including the road section photographed by the camera, and the virtual coordinates of the vehicle side image area photographed by the camera on a 1: 1 basis. Detecting a vehicle area from the vehicle side image through an image recognition technique using running, calculating coordinates of the detected vehicle region, and calculating a side length of the vehicle using the front and rear coordinates of the vehicle in the vehicle region Follow the steps,
The controller detects a tire object area in the vehicle side image using an image recognition technique using machine learning, calculates coordinates of the detected tire object area, and recognizes the detected number of tire object areas as the number of axes And calculating wheelbase information using coordinates of each tire object area located at both ends,
The controller generates a virtual trigger line on the screen transmitted by the camera, recognizes that the vehicle has entered when the vehicle reaches the trigger line, and attempts to capture a side image of the vehicle through the camera. The trigger line arrival is detected through monitoring the amount of change in the image histogram of the virtual trigger line area,
The controller defines a part of the vehicle side image captured by the camera as a coordinate and stores it as a digital file in xml format, wherein the part of the vehicle is calculated using the front and rear coordinates of the vehicle. This is an area that has a height that is longer than the side length and tire diameter, and deep learning is performed using a single shot multibox detector (SSD) model for the vehicle side image and the digital file, and the result value learned through deep learning ( weight), compare the feature points using the result from the input side image of the vehicle, infer pixel coordinates in the tire area accordingly, calculate a bounding box in the tire area, and calculate The bounding box is recognized as a tire object, coordinate values for each tire object are obtained, and the entire tire object Calculating bearing information, obtain the amount and the vehicle information acquisition method characterized in that for calculating the wheelbase information in a manner of calculating the distance between each tire object coordinates.
delete delete delete delete delete

Images