CN103116748A - Method and system for identifying illegal driving behavior based on road signs - Google Patents
Method and system for identifying illegal driving behavior based on road signs Download PDFInfo
- Publication number
- CN103116748A CN103116748A CN2013100770488A CN201310077048A CN103116748A CN 103116748 A CN103116748 A CN 103116748A CN 2013100770488 A CN2013100770488 A CN 2013100770488A CN 201310077048 A CN201310077048 A CN 201310077048A CN 103116748 A CN103116748 A CN 103116748A
- Authority
- CN
- China
- Prior art keywords
- road
- image
- sign
- guiding
- driving behavior
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Traffic Control Systems (AREA)
- Image Processing (AREA)
Abstract
Description
技术领域technical field
本发明涉及驾驶行为的识别技术领域,特别涉及一种基于路面标识识别违规驾驶行为的方法及系统。The invention relates to the technical field of driving behavior identification, in particular to a method and system for identifying illegal driving behavior based on road signs.
背景技术Background technique
随着车辆数量的急速增加,给道路安全带来了诸多问题,例如,交通拥堵、交通事故频发等。在交通事故中很大一部分是由驾驶人的不规范驾驶而引起的。With the rapid increase in the number of vehicles, many problems have been brought to road safety, such as traffic congestion and frequent traffic accidents. A large part of traffic accidents is caused by the driver's irregular driving.
路面标识识别的问题有以下几个方面:The problem of pavement sign recognition has the following aspects:
路面情况复杂:行驶路面情况主要可以分为两类,一是在市区行驶时的路面情况,此时前方车辆较多,前方路面可能被遮挡,行驶情况多变复杂。二是在高速公路行驶时的路面情况,此时路面空旷,车辆比较少,利于车道线的检测和标识的识别。Complicated road conditions: The road conditions can be divided into two categories. One is the road conditions when driving in urban areas. At this time, there are many vehicles ahead, and the road ahead may be blocked. The driving conditions are changeable and complicated. The second is the road condition when driving on the expressway. At this time, the road surface is empty and there are fewer vehicles, which is conducive to the detection of lane lines and identification of signs.
光照影响:由于车辆行驶时光照变化对于采集到的路面图像影响较大。主要表现为夜晚采集图像不清晰、白天采集图像可能出现各个角度的光照不均、隧道及道路两旁树木对于采集图像的影响等。因此需要设计对于光照变化鲁棒的算法。Illumination influence: As the illumination changes when the vehicle is driving, it has a great influence on the collected road surface images. The main manifestations are that the images collected at night are not clear, the images collected during the day may have uneven illumination from various angles, and the influence of trees on both sides of tunnels and roads on the collected images. Therefore, it is necessary to design an algorithm that is robust to illumination changes.
算法运行速度:车辆行驶时速度较快,并且需要进行车道线检测和标识识别。因此,我们研究的车道线以及标识的识别算法必须能够满足车辆高速行驶时的实时性要求,以提供安全的行驶保障。Algorithm running speed: The speed of the vehicle is relatively fast, and lane line detection and sign recognition are required. Therefore, the recognition algorithm of the lane lines and signs we research must be able to meet the real-time requirements of the vehicle at high speed, so as to provide safe driving guarantee.
目前普遍采用的方法是检测路边标识牌以及车辆状态信息的方法。该方法通过对路边标识牌的检测与识别确定当前车道对车辆行驶参数的制约情况,车辆状态信息通过车载传感器获得。检测路边标识牌结合车辆状态信息判断当前车辆行驶情况是否为违规行为。The method generally adopted at present is the method of detecting roadside signs and vehicle state information. The method determines the constraints of the current lane on the driving parameters of the vehicle through the detection and recognition of the roadside signs, and the vehicle state information is obtained through the vehicle sensor. Detect roadside signs and combine vehicle status information to judge whether the current vehicle driving situation is a violation.
现有技术的缺点:Disadvantages of existing technology:
(1)路边标识牌常被车辆及树木等遮挡,检测精度不高,没有检测路面标识稳定可靠。(1) The roadside signs are often blocked by vehicles and trees, the detection accuracy is not high, and there is no stable and reliable detection of road signs.
(2)基于路边标识牌检测与车辆状态信息判断违规驾驶行为的判断逻辑不够完善。(2) The judgment logic for judging illegal driving behavior based on roadside sign detection and vehicle status information is not perfect.
(3)检测算法受光照条件的影响较大。(3) The detection algorithm is greatly affected by the lighting conditions.
发明内容Contents of the invention
本发明的目的旨在至少解决上述的技术缺陷之一。The object of the present invention is to solve at least one of the above-mentioned technical drawbacks.
为达到上述目的,本发明一方面的实施例提出一种基于路面标识识别违规驾驶行为的方法,包括以下步骤:获取车辆行驶的道路图像,并对所述道路图像进行预处理以生成所述道路图像的二值化图像;根据所述二值化图像中的白像素点确定道路的边界以生成道路边界线;从所述道路图像中检测与所述道路边界线对应的道路线之间是否有引导标识,其中,所述引导标识包括公交专用标识和方向标识;以及当所述引导标识为方向标识时,根据所述方向标识和车辆状态信息判断所述车辆是否违规行驶。In order to achieve the above object, an embodiment of the present invention proposes a method for identifying illegal driving behavior based on road signs, including the following steps: acquiring a road image on which a vehicle is driving, and performing preprocessing on the road image to generate the road The binarized image of the image; determine the boundary of the road according to the white pixels in the binarized image to generate the road boundary line; detect whether there is between the road lines corresponding to the road boundary line from the road image A guide sign, wherein the guide sign includes a bus-specific sign and a direction sign; and when the guide sign is a direction sign, it is judged whether the vehicle is driving illegally according to the direction sign and vehicle state information.
根据本发明实施例的方法,通过生成道路图像的二值化图像,并在该二值化图像中判别引导标识识别驾驶是否规范,提前预防了交通事故的发生,进而提高了驾驶的安全性。According to the method of the embodiment of the present invention, by generating the binarized image of the road image, and judging whether the guide sign recognizes the driving standard in the binarized image, the occurrence of traffic accidents can be prevented in advance, and the driving safety can be improved.
本发明的一个实施例中,当引导标识为公交车专用车道标识时,向驾驶员发出提示信息。In one embodiment of the present invention, when the guide sign is a bus lane sign, prompt information is sent to the driver.
本发明的一个实施例中,所述引导标识和引导标识模板的特征相似度小于阈值时,则所述引导标识与相对应的引导标识模板相同。In an embodiment of the present invention, when the feature similarity between the guide logo and the guide logo template is less than a threshold, the guide logo is the same as the corresponding guide logo template.
本发明的一个实施例中,所述预处理包括线性滤波和帧间叠加。In an embodiment of the present invention, the preprocessing includes linear filtering and inter-frame superposition.
为达到上述目的,本发明的实施例另一方面提出一种基于路面标识识别违规驾驶行为的系统,包括:预处理模块,用于获取车辆行驶的道路图像,并对所述道路图像进行预处理以生成所述道路图像的二值化图像;生成模块,用于根据所述二值化图像中的白像素点确定道路的边界以生成道路边界线;检测模块,用于从所述道路图像中检测与所述道路边界线对应的道路线之间是否有引导标识,其中,所述引导标识包括公交专用标识和方向标识;以及处理模块,用于当所述引导标识为方向标识时,根据所述方向标识和车辆状态信息判断所述车辆是否违规行驶。In order to achieve the above purpose, another embodiment of the present invention proposes a system for identifying illegal driving behavior based on road signs, including: a preprocessing module, used to obtain road images of vehicles driving, and preprocess the road images to generate a binarized image of the road image; a generation module, used to determine the boundary of the road according to the white pixels in the binarized image to generate a road boundary line; a detection module, used to obtain a road boundary line from the road image Detecting whether there is a guide sign between the road lines corresponding to the road boundary line, wherein the guide sign includes a bus-specific sign and a direction sign; The direction sign and the vehicle status information are used to determine whether the vehicle is driving in violation of regulations.
根据本发明实施例的系统,通过生成道路图像的二值化图像,并在该二值化图像中判别引导标识识别驾驶是否规范,提前预防了交通事故的发生,进而提高了驾驶的安全性。According to the system of the embodiment of the present invention, by generating the binarized image of the road image, and judging whether the guide sign recognizes the driving standard in the binarized image, the occurrence of traffic accidents can be prevented in advance, and the driving safety can be improved.
本发明的一个实施例中,所述处理模块还用于当引导标识为公交车专用车道标识时,向驾驶员发出提示信息。In an embodiment of the present invention, the processing module is further configured to send prompt information to the driver when the guide sign is a bus lane sign.
本发明的一个实施例中,所述引导标识和引导标识模板的特征相似度小于阈值时,则所述引导标识与相对应的引导标识模板相同。In an embodiment of the present invention, when the feature similarity between the guide logo and the guide logo template is less than a threshold, the guide logo is the same as the corresponding guide logo template.
本发明的一个实施例中,所述预处理包括线性滤波和帧间叠加。In an embodiment of the present invention, the preprocessing includes linear filtering and inter-frame superposition.
本发明附加的方面和优点将在下面的描述中部分给出,部分将从下面的描述中变得明显,或通过本发明的实践了解到。Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
附图说明Description of drawings
本发明上述的和/或附加的方面和优点从下面结合附图对实施例的描述中将变得明显和容易理解,其中:The above and/or additional aspects and advantages of the present invention will become apparent and easy to understand from the following description of the embodiments in conjunction with the accompanying drawings, wherein:
图1为根据本发明一个实施例的基于路面标识识别违规驾驶行为的方法的流程图;1 is a flowchart of a method for identifying illegal driving behavior based on road signs according to an embodiment of the present invention;
图2为根据本发明一个实施例的实际道路图像;Fig. 2 is an actual road image according to an embodiment of the present invention;
图3为根据本发明一个实施例的礼帽运算图像;Fig. 3 is a top hat computing image according to one embodiment of the present invention;
图4为根据本发明一个实施例的二值化图像;Fig. 4 is a binarized image according to one embodiment of the present invention;
图5为根据本发明一个实施例的进行帧间叠加的二值化图像;FIG. 5 is a binarized image for inter-frame superposition according to an embodiment of the present invention;
图6为根据本发明一个实施例的道路边界线的搜索示意图;Fig. 6 is a schematic diagram of searching for road boundary lines according to an embodiment of the present invention;
图7为根据本发明一个实施例的引导标识模板图;Fig. 7 is a template diagram of a guide logo according to an embodiment of the present invention;
图8为根据本发明一个实施例的直行线上左、右转弯的示意图;Fig. 8 is a schematic diagram of left and right turns on a straight line according to an embodiment of the present invention;
图9为根据本发明一个实施例的公交车道占用示意图;Fig. 9 is a schematic diagram of bus lane occupancy according to an embodiment of the present invention;
图10为根据本发明一个实施例的左、右转弯车道上直行的示意图;以及Figure 10 is a schematic diagram of going straight on left and right turning lanes according to an embodiment of the present invention; and
图11为本发明实施例的基于路面标识识别违规驾驶行为的系统的结构框图。Fig. 11 is a structural block diagram of a system for identifying illegal driving behavior based on road signs according to an embodiment of the present invention.
具体实施方式Detailed ways
下面详细描述本发明的实施例,实施例的示例在附图中示出,其中自始至终相同或类似的标号表示相同或类似的元件或具有相同或类似功能的元件。下面通过参考附图描述的实施例是示例性的,仅用于解释本发明,而不能解释为对本发明的限制。Embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary only for explaining the present invention and should not be construed as limiting the present invention.
图1为本发明实施例的基于路面标识识别违规驾驶行为的方法的流程图。如图1所示,根据本发明实施例的基于路面标识识别违规驾驶行为的方法,包括以下步骤:FIG. 1 is a flowchart of a method for identifying illegal driving behavior based on road signs according to an embodiment of the present invention. As shown in Figure 1, the method for identifying illegal driving behavior based on road signs according to an embodiment of the present invention includes the following steps:
步骤S101,获取车辆行驶的道路图像,并对道路图像进行预处理以生成道路图像的二值化图像。其中,预处理包括线性滤波和帧间叠加。Step S101 , acquiring a road image on which the vehicle is driving, and performing preprocessing on the road image to generate a binarized image of the road image. Among them, preprocessing includes linear filtering and inter-frame superposition.
具体地,本项目采用所谓的“礼帽”运算进行形态学滤波。“礼帽”运算定义为该图像和其开运算之间的逐像素差分,而开运算定义为对图像先进行腐蚀,再进行膨胀。这里的腐蚀和膨胀运算都是实数域上的运算。Specifically, this project uses the so-called "top hat" operation for morphological filtering. The "top hat" operation is defined as the pixel-by-pixel difference between the image and its opening operation, and the opening operation is defined as first corroding the image and then dilating it. The erosion and dilation operations here are both operations on the real number field.
腐蚀:dst(x,y)=min(x',y');element(x',y')≠0src(x+x',y+y'),其中,src为源图像,dst为目标图像,element为结构元素,x',y'和x,y分别表示结构元素与图像像素的坐标。Corrosion: dst(x,y)=min(x',y'); element(x',y')≠0src(x+x',y+y'), where src is the source image and dst is the target Image, element is the structure element, x', y' and x, y represent the coordinates of the structure element and the image pixel respectively.
膨胀:dst(x,y)=max(x',y');element(x',y')≠0src(x+x',y+y'),其中,src为源图像,dst为目标图像,element为结构元素,x',y'和x,y分别表示结构元素与图像像素的坐标。Expansion: dst(x,y)=max(x',y'); element(x',y')≠0src(x+x',y+y'), where src is the source image and dst is the target Image, element is the structure element, x', y' and x, y represent the coordinates of the structure element and the image pixel respectively.
于是“礼帽”运算可以定义为:dst=src-open(src,element)=src-dilate(erode(src,element)),其中,open(src,dlement)表示开运算,dilate(erode(src,element))表示先进行腐蚀,再进行膨胀。“礼帽”运算突出了比周围像素更亮的点,图2和图3分别为实际道路图像和礼帽运算图像。从图3中可以看出礼帽运算的结果不受全局光照的影响,换言之,结果只与像素与周边像素的大小关系决定,这一优点源于运算中的差分环节,这一优点与图片的梯度特征是一致的,但梯度特征对于中心倒影带来的严重干扰更加敏感。So the "top hat" operation can be defined as: dst=src-open(src,element)=src-dilate(erode(src,element)), where open(src,dlement) means open operation, dilate(erode(src, element)) means to corrode first and then expand. The "top hat" operation highlights the points that are brighter than the surrounding pixels. Figure 2 and Figure 3 are the actual road image and the top hat operation image respectively. It can be seen from Figure 3 that the result of the top hat calculation is not affected by global illumination. In other words, the result is only determined by the size relationship between the pixel and the surrounding pixels. This advantage comes from the difference link in the operation. This advantage is related to the gradient of the image The features are consistent, but the gradient feature is more sensitive to the severe interference caused by the central reflection.
由“礼帽”运算直接进行阈值化已经能够得到较好的结果,但结果依赖于阈值的选择,因此利用模板滤波来实现。注意到道路线的特征不仅是具有明显的边缘,同时道路线也具有明显的宽度,因此考虑如表1所示的模板对图像进行卷积,之后再对卷积图像进行阈值化,采取固定的阈值(灰度值220),得到二值化图像,如图4所示。但是,其道路线为不连续的线条,因此再进行帧间叠加使得道路线成为连续的线。Thresholding directly by the "top hat" operation has been able to get better results, but the result depends on the selection of the threshold, so template filtering is used to achieve it. Notice that the feature of the road line is not only the obvious edge, but also the obvious width of the road line, so consider the template shown in Table 1 to convolve the image, and then threshold the convolved image, and take a fixed Threshold (gray value 220) to obtain a binarized image, as shown in Figure 4. However, the road line is a discontinuous line, so inter-frame superposition is performed to make the road line a continuous line.
表1Table 1
道路线不总是连续的,例如,图4所示,道路线左侧连续,而右侧为间断的虚线。而公路上常见的虚线类型又有长虚线(如图4)和短虚线两种,可以对三种虚线类型分别处理,但对帧间信息的帧间叠加可以使原本间断的道路线成为连续的直线,从而得到最终的二值化图像,如图5所示。The road lines are not always continuous. For example, as shown in FIG. 4, the road lines are continuous on the left and intermittent dotted lines on the right. There are two types of dashed lines commonly seen on highways: long dashed lines (as shown in Figure 4) and short dashed lines. The three types of dashed lines can be processed separately, but the inter-frame superposition of inter-frame information can make the originally intermittent road lines become continuous. straight line, so as to obtain the final binarized image, as shown in Figure 5.
步骤S102,根据二值化图像中的白像素点确定道路的边界以生成道路边界线。Step S102, determining the boundary of the road according to the white pixels in the binarized image to generate a road boundary line.
具体地,对于道路的数学模型有多种选择,直线模型的拟合最为简单,也有包含更多参数的高阶多项式模型及双曲线模型,本系统选择了直线模型,其主要原因除了计算简单的考虑外还在于:对于路面符号的检测范围仅限于车前一小段距离,即使是曲率很大的路面(类似于立交桥上的环形弯道),在车辆前方的一小段距离内也可以大致近似成直线,而曲线模型带来的关于道路在远方的趋势信息对于近处的路面符号检测没有帮助。Specifically, there are many choices for the mathematical model of the road. The straight line model is the easiest to fit, and there are also high-order polynomial models and hyperbolic models that contain more parameters. This system chooses the straight line model. Another consideration is that the detection range of road symbols is limited to a short distance in front of the vehicle. Even a road with a large curvature (similar to a circular curve on an overpass) can be roughly approximated to a small distance in front of the vehicle. Straight lines, and the trend information about the road in the distance brought by the curve model is not helpful for the detection of nearby road signs.
在本发明的一个实施例中,在得到二值化图像的基础上,需要得到道路线的准确参数,即确定道路的边界线。通过采用基于经验规则的逐行搜索方法,这种方法具有计算快速的优点,虽然这种方法容易受到噪声以及路面符号的干扰,但如果适当利用帧间信息,上述干扰的影响可以被大幅消减。In one embodiment of the present invention, on the basis of obtaining the binarized image, it is necessary to obtain accurate parameters of the road line, that is, to determine the boundary line of the road. By adopting a line-by-line search method based on empirical rules, this method has the advantage of fast calculation. Although this method is susceptible to interference from noise and road symbols, if the inter-frame information is properly used, the impact of the above interference can be greatly reduced.
在本发明的一个实施例中,在二值化图像中按行进行搜素,从中线向左右两侧开始搜索,每行存储搜索得到的第一个白色点,将这个点当作可能的道路线点,每向道路点集合中加入一个新的点,下一次的搜索范围就在上一个点的附近。直到连续出现多次搜索失败或达到手工设定的边界时认为搜索结束,对已有的结果进行直线拟合,若相关系数大于某个阈值(例如0.99)则接受搜索结果,否则就从集合中的第一个点的附近选取一新起始点,继续进行搜索过程。In one embodiment of the present invention, search is performed row by row in the binarized image, starting from the center line to the left and right sides, each row stores the first white point obtained from the search, and takes this point as a possible path Route point, every time a new point is added to the way point set, the next search range is near the previous point. The search is considered to be over until multiple consecutive search failures occur or the manually set boundary is reached, and the existing results are fitted with a straight line. If the correlation coefficient is greater than a certain threshold (for example, 0.99), the search result is accepted, otherwise, it is selected from the collection. Select a new starting point near the first point of , and continue the search process.
典型情况下成功的搜索如图6所示,圆圈代表搜索的点的轨迹,搜索左侧道路线时,首先从中线开始向左搜索,将遇到的第一个白像素加入道路线集合,之后每次的搜索都在之前的上一行,而水平方向上的搜索范围在之前点的附近,如此很大程度上地减少了搜索量。在搜索过程中可能会遇到并非道路线上的像素但其灰度值为255,由于以这样的像素点为开始的后续的搜索无法找到在同一直线上的多个白色像素,这样的错误初始点会被抛弃。通过这种方式可得到一个类似直线型的白像素点集合。A typical successful search is shown in Figure 6. The circle represents the trajectory of the searched point. When searching for the left road line, first search from the center line to the left, add the first white pixel encountered to the road line set, and then Each search is on the previous line, and the search range in the horizontal direction is near the previous point, which greatly reduces the amount of search. During the search process, pixels that are not on the road line may be encountered but their gray value is 255. Since the subsequent search starting with such a pixel point cannot find multiple white pixels on the same line, such an initial error points will be discarded. In this way, a set of white pixels similar to a straight line can be obtained.
从所得到的白像素点集合中各点位于道路内侧边缘上的位置之后,采用最小二乘拟合生成道路的边界线。最小二乘拟合是一种直线优化技术,已知一组点(xi,yi)位于直线x=ky+b上,已知对于yi的测量是准确的,全部的误差来自于x坐标(即列的位置是有误差的),最小二乘方法给出了最小化误差平方k、b的估计, 其中,Sx=∑xi为白像素点x坐标之和,xi为白像素点x坐标,Sy=∑yi为白像素点y坐标之和,yi为白像素点y坐标,Sxy=∑xiyi, 为白像素点坐标的相应运算,为参数k的估计值,为参数b的估计值。From the position of each point in the obtained white pixel point set on the inner edge of the road, the least squares fitting is used to generate the boundary line of the road. Least squares fitting is a straight line optimization technique. It is known that a set of points ( xi , y i ) are located on the straight line x=ky+b, and the measurement of y i is known to be accurate, and all errors come from x coordinates (i.e. the position of the column is in error), the least squares method gives an estimate of the minimized error square k, b, Wherein, S x =∑x i is the sum of the x coordinates of the white pixels, x i is the x coordinates of the white pixels, S y =∑y i is the sum of the y coordinates of the white pixels, and y i is the y coordinates of the white pixels, S xy =∑x i y i , is the corresponding operation of the white pixel coordinates, is the estimated value of parameter k, is the estimated value of parameter b.
在本发明的一个实施例中,道路线搜索算法中确实包含了对于噪声的抵抗能力,但仅限于不在同一直线上的噪声,而道路间的符号包含了数量众多的白像素点,很容易找出在同一直线上的若干个像素,从而造成搜索算法不能正常工作。假设所得到的上一帧图像是正确的,这个信息就为消除道路中间的符号提供了重要的依据,因为只要将两条斜边延长交于消失点,两条斜线与水平的底线组成了三角形的道路区,道路区外是道路线,而道路区内是可能造成干扰的道路间的符号,因此去除干扰的方法就是将道路区内的像素都置成黑色,由此可准确的得出道路的边界线。In one embodiment of the present invention, the road line search algorithm does include the ability to resist noise, but it is only limited to noise that is not on the same straight line, and the symbols between roads contain a large number of white pixels, which are easy to find Several pixels that appear on the same straight line cause the search algorithm to not work properly. Assuming that the previous frame image obtained is correct, this information provides an important basis for eliminating the sign in the middle of the road, because as long as the two hypotenuses are extended and intersected at the vanishing point, the two oblique lines and the horizontal bottom line form a In the triangular road area, road lines are outside the road area, and road signs that may cause interference are inside the road area. Therefore, the method to remove interference is to set all pixels in the road area to black, and it can be accurately obtained The boundary line of the road.
步骤S103,从道路图像中检测与道路边界线对应的道路线之间是否有引导标识,其中,引导标识包括公交专用标识和方向标识。In step S103, it is detected from the road image whether there is a guide sign between the road lines corresponding to the road boundary line, wherein the guide sign includes a bus-only sign and a direction sign.
具体地,引导标识有引导标识模板,由于其结构简单可通过引导标识模板与处理所获得的引导标识进行比较,从而确定是何种引导标识。引导标识包括公交专用标识和方向标识,如图7所示。Specifically, the guide sign has a guide sign template. Due to its simple structure, the guide sign template can be compared with the guide sign obtained through processing to determine what kind of guide sign it is. Guidance signs include bus-specific signs and direction signs, as shown in Figure 7.
步骤S104,当引导标识为方向标识时,根据方向标识和车辆状态信息判断车辆是否违规行驶。Step S104, when the guide sign is a direction sign, judge whether the vehicle is driving illegally according to the direction sign and the vehicle state information.
在本发明的一个实施例中,确定道路线后,为了更加稳定地进行模式分类,需要将摄像机的平视视图变换为俯视视图,这样的变换对于匹配非常重要,因为在平视视图中摄像机的水平位移会导致符号的形状发生变化,而俯视视图中,只要正确地确定了道路线,符号的形状就是大致固定的。经过转换后,俯视块中有很大可能存在着标识,下面通过模板匹配的方法决定标识的种类。即定义图片与模板间的相关系数C(I,Ii),其中Ii是引导标识i的模板,图像的分类按照如下方式确定:x=argmaxiC(I,Ii),其中,x表示最终确定的引导标识类别。In one embodiment of the present invention, after the road line is determined, in order to perform pattern classification more stably, it is necessary to transform the head-up view of the camera into a top-down view. Such transformation is very important for matching, because the horizontal displacement of the camera in the head-up view would cause the shape of the symbol to change, whereas in an overhead view, the shape of the symbol is roughly fixed as long as the road lines are correctly identified. After conversion, it is very likely that there is a logo in the top-view block, and the type of logo is determined by the method of template matching. That is to define the correlation coefficient C(I,I i ) between the picture and the template, where I i is the template of the guide logo i, and the classification of the image is determined as follows: x=argmax i C(I,I i ), where, x Indicates the finalized bootstrap identity category.
也就是在所有模板中寻找与图片相关度最高的一幅。同时为了处理没有包含在模板库中的标识并且增强对噪声的抵抗能力,当最大相关系数仍旧较小时认为识别失败,放弃对这一符号的识别,于是分类依据可以完整地表述为:
相关系数的选择对于分类的效果有着重要的影响,就这一问题研究者提出了多种相关系数的计算方法,需要在不同的场合择优选取。The choice of correlation coefficient has an important impact on the effect of classification. On this issue, researchers have proposed a variety of calculation methods for correlation coefficient, which need to be selected on different occasions.
在本发明的一个实施例中,最常用的实向量的相似性测度包括内积、余弦相关系数、Pearson相关系数和汉明测度。In one embodiment of the present invention, the most commonly used similarity measures of real vectors include inner product, cosine correlation coefficient, Pearson correlation coefficient and Hamming measure.
(1)内积(1) inner product
定义为C(x,y)=xTy=∑xiyi,其中,xi与yi分别表示向量x与y的坐标。在大多数情况下,当向量x和y被归一化而具有相同的长度a时,C的上限和下限分别为+a2和-a2。It is defined as C(x,y)=x T y=∑x i y i , where x i and y i represent the coordinates of vectors x and y respectively. In most cases, when the vectors x and y are normalized to have the same length a, the upper and lower bounds of C are +a 2 and -a 2 , respectively.
(2)余弦相关系数(2) Cosine correlation coefficient
余弦相关系数与内积相关较大,定义为:其中,x和y分别表示向量。这一相关系数有着向量夹角余弦的几何意义,也是归一化于-1到+1之间的相关系数,而且是旋转不变的。The cosine correlation coefficient has a large correlation with the inner product and is defined as: where x and y represent vectors, respectively. This correlation coefficient has the geometric meaning of the cosine of the vector angle, and is also a correlation coefficient normalized between -1 and +1, and is invariant to rotation.
(3)Pearson相关系数。其表达式是:其中,xd=(x1-mx,x2-mx,…,xn-mx)表示向量x与均值的差值坐标,yd=(y1-my,y2-my,…,yn-my)表示向量y与均值的差值坐标,表示向量x的均值,表示向量y的均值,xi与yi分别为向量x与y的坐标。(3) Pearson correlation coefficient. Its expression is: Among them, x d =(x 1 -m x ,x 2 -m x ,…,x n -m x ) represents the difference coordinates between the vector x and the mean value, y d =(y 1 -m y ,y 2 -m y ,…,y n -m y ) represent the coordinates of the difference between the vector y and the mean, represents the mean of the vector x, Indicates the mean value of vector y, and x i and y i are the coordinates of vector x and y respectively.
(4)汉明测度,设
根据C(x,y)获得引导标识和引导标识模板的特征相似度,当相似度即相关系数小于阈值时,则引导标识与相对应的引导标识模板相同,从而确定引导标识的类别。According to C(x, y), the feature similarity between the guide logo and the guide logo template is obtained. When the similarity, that is, the correlation coefficient is less than the threshold, the guide logo is the same as the corresponding guide logo template, thereby determining the category of the guide logo.
在本发明的一个实施例中,由于余弦相关系数能够取得较好的分类效果,因此采用余弦相关系数作为引导标识和引导标识模板之间的相似性度量方法。In one embodiment of the present invention, since the cosine correlation coefficient can achieve a better classification effect, the cosine correlation coefficient is used as a similarity measurement method between the guide logo and the guide logo template.
本发明所要识别的违规驾驶包括直行线左、右转,转弯未开转向灯,占用公交车道和左、右转弯车道直行。The illegal driving to be identified by the present invention includes turning left and right on the straight line, turning without turning on the turn signal, occupying the bus lane and going straight on the left and right turning lanes.
图8为根据本发明一个实施例的直行线上左、右转弯的示意图。如图8所示,车辆在直行车道上,不应该存在左、右转弯的驾驶行为,若驾驶人想左、右转弯应该变更至左右转弯车道上。在直行线上转弯,即使开启了转向车灯,后方驾驶员也会误以为前方车辆为换线行为,造成后方驾驶人未能有效解读出前方驾驶人的操作意图,没有一定的心理及行为上的准备,造成安全隐患。Fig. 8 is a schematic diagram of left and right turns on a straight line according to an embodiment of the present invention. As shown in Figure 8, when the vehicle is on the straight lane, there should be no left or right turning driving behavior. If the driver wants to turn left or right, he should change to the left and right turning lanes. Turning on a straight line, even if the turn lights are turned on, the driver behind will mistakenly think that the vehicle in front is changing lanes, causing the driver behind to fail to effectively interpret the operation intention of the driver in front, without certain psychological and behavioral preparations, causing safety hazards.
路面标识检测到直行标识时,检测方向盘转角信号,若方向盘转角在一定阈值内变化,说明当前状态下,车辆没有转弯行为;若方向盘转角超过了一定的阈值,说明车辆有转弯行为。若检测到车辆有转弯行为,则违规程序输出预警信息。When the road sign detects the straight-going sign, the steering wheel angle signal is detected. If the steering wheel angle changes within a certain threshold, it means that the vehicle has no turning behavior in the current state; if the steering wheel angle exceeds a certain threshold, it means that the vehicle has turning behavior. If a turning behavior of the vehicle is detected, the violation program outputs early warning information.
在本发明的一个实施例中,车辆转弯行驶时,一般伴随着降低车速的驾驶行为,若此时未开启转向灯,跟随车辆未能有效辨识出前车的驾驶意图,依旧直线快速行驶,此时便会造成严重的交通事故。转弯未开转向灯行为,危害巨大。In one embodiment of the present invention, when the vehicle is turning, it is generally accompanied by the driving behavior of reducing the speed of the vehicle. If the turn signal is not turned on at this time, the following vehicle cannot effectively recognize the driving intention of the vehicle in front and is still driving in a straight line. It will cause serious traffic accidents. The act of turning without turning on the turn signal is extremely harmful.
路面标识检测到左转或右转标识时,说明车辆行驶在转向车道上。此时检测方向盘转角信号,若方向盘转角在一定的转角阈值内,驾驶人员开启转向灯,说明驾驶人已经产生了转向意识,且开启了转向车灯。若超过一定的转角阈值,驾驶人员仍未开启转向车灯,则违规程序输出预警信息,通知驾驶人应该开启转向灯。道路设专用车道的,在专用车道内,只准许规定的车辆通行,其他车辆不得进入专用车道内行驶。When the road marking detects a left-turn or right-turn sign, it means that the vehicle is driving in the turning lane. At this time, the steering wheel angle signal is detected. If the steering wheel angle is within a certain threshold value, the driver will turn on the turn signal, indicating that the driver has become aware of the steering and has turned on the turn signal. If a certain corner threshold is exceeded and the driver has not turned on the turn signal, the violation program will output an early warning message to inform the driver that the turn signal should be turned on. If a road has a special lane, only the specified vehicles are allowed to pass in the special lane, and other vehicles are not allowed to enter the special lane.
图9为根据本发明一个实施例的公交车道占用示意图。如图9所示,在道路资源严重稀缺的现实中,设置公交专用道是为了解决大多数人的出行问题。公交专用道的设置,提升了市民乘坐公交的热情,公交专用车道用于提醒市民应改变出行方式,减少驾驶私家车,选择公共交通,从而缓解交通拥堵。在交通运输高峰时期,占用公交车道,势必造成交通拥堵,影响交通运力。路面标识检测到“公”、“交”、“车”、“道”标识时,说明当前情况下,车辆行驶在公交专用车道上,占用了公交车道,违规程序发出预警提示信息,通知驾驶人变更车道,从当前车道变更至其他车道。Fig. 9 is a schematic diagram of bus lane occupancy according to an embodiment of the present invention. As shown in Figure 9, in the reality of severe scarcity of road resources, the purpose of setting up bus lanes is to solve the travel problem of most people. The setting of bus-only lanes has enhanced citizens' enthusiasm for taking buses. Bus-only lanes are used to remind citizens to change their travel methods, reduce driving private cars, and choose public transportation, thereby alleviating traffic congestion. During the peak period of traffic transportation, occupying bus lanes will inevitably cause traffic congestion and affect traffic capacity. When the signs of "public", "traffic", "vehicle" and "road" are detected on the road surface markings, it means that under the current situation, the vehicle is driving on the bus lane and occupies the bus lane. A person changes lanes from the current lane to another lane.
图10为根据本发明一个实施例的左、右转弯车道上直行的示意图。如图10所示,设立左、右转车道是为了便于车辆转弯行驶。车辆转弯行驶时,一般伴随着减速行为。若在左、右转车道上,前方车辆有转弯行为,则其开始减速,并开启转向车灯,若此时自车仍高速行驶在左、右转车道上,未能有效意识到前车的转弯行为,则很有可能与前车发生碰撞,造成严重的交通安全事故。Fig. 10 is a schematic diagram of going straight on left and right turning lanes according to an embodiment of the present invention. As shown in Figure 10, the purpose of setting up left and right turn lanes is to facilitate the turning of vehicles. When a vehicle is turning, it is generally accompanied by deceleration behavior. If the vehicle in front turns on the left or right turn lane, it will start to slow down and turn on the turn lights. Turning behavior, it is very likely to collide with the vehicle in front, causing serious traffic safety accidents.
路面标识检测到左转或右转标识时,说明车辆行驶在转向车道上。此时检测车速信号,将车速对时间积分,若在一定的距离范围内,驾驶人员开启转向灯,说明驾驶人的驾驶意图为转向行驶,且已经意识到自车行驶在转向车道上,开启了转向车灯准备换道。若超过一定的距离阈值,驾驶人员仍未开启转向车灯,依旧直线行驶,则违规程序输出预警信息。When the road marking detects a left-turn or right-turn sign, it means that the vehicle is driving in the turning lane. At this time, the vehicle speed signal is detected, and the vehicle speed is integrated with time. If the driver turns on the turn signal within a certain distance range, it means that the driver's driving intention is to turn, and he has realized that the car is driving on the turn lane. The turn signal is ready to change lanes. If a certain distance threshold is exceeded and the driver still drives straight without turning on the turn lights, the violation program will output an early warning message.
根据本发明实施例的方法,通过生成道路图像的二值化图像,并在该二值化图像中判别引导标识识别驾驶是否规范,提前预防了交通事故的发生,进而提高了驾驶的安全性。According to the method of the embodiment of the present invention, by generating the binarized image of the road image, and judging whether the guide sign recognizes the driving standard in the binarized image, the occurrence of traffic accidents can be prevented in advance, and the driving safety can be improved.
图11为本发明实施例的基于路面标识识别违规驾驶行为的系统的结构框图,如图11所示,根据本发明实施例的基于路面标识识别违规驾驶行为的系统状态评估系统包括预处理模块100、生成模块200、检测模块300和处理模块400。11 is a structural block diagram of a system for identifying illegal driving behavior based on road signs according to an embodiment of the present invention. As shown in FIG. , a generation module 200 , a detection module 300 and a processing module 400 .
预处理模块100用于获取车辆行驶的道路图像,并对道路图像进行预处理以生成道路图像的二值化图像。The pre-processing module 100 is used to acquire the road image of the vehicle, and perform pre-processing on the road image to generate a binarized image of the road image.
具体地,本项目采用所谓的“礼帽”运算进行形态学滤波。“礼帽”运算定义为该图像和其开运算之间的逐像素差分,而开运算定义为对图像先进行腐蚀,再进行膨胀。这里的腐蚀和膨胀运算都是实数域上的运算。Specifically, this project uses the so-called "top hat" operation for morphological filtering. The "top hat" operation is defined as the pixel-by-pixel difference between the image and its opening operation, and the opening operation is defined as first corroding the image and then dilating it. The erosion and dilation operations here are both operations on the real number field.
腐蚀:dst(x,y)=min(x',y');element(x',y')≠0src(x+x',y+y'),其中,src为源图像,dst为目标图像,element为结构元素,x',y'和x,y分别表示结构元素与图像像素的坐标。Corrosion: dst(x,y)=min(x',y'); element(x',y')≠0src(x+x',y+y'), where src is the source image and dst is the target Image, element is the structure element, x', y' and x, y represent the coordinates of the structure element and the image pixel respectively.
膨胀:dst(x,y)=max(x',y');element(x',y')≠0src(x+x',y+y'),其中,src为源图像,dst为目标图像,element为结构元素,x',y'和x,y分别表示结构元素与图像像素的坐标。Expansion: dst(x,y)=max(x',y'); element(x',y')≠0src(x+x',y+y'), where src is the source image and dst is the target Image, element is the structure element, x', y' and x, y represent the coordinates of the structure element and the image pixel respectively.
于是“礼帽”运算可以定义为:dst=src-open(src,element)=src-dilate(erode(src,element)),其中,open(src,dlement)表示开运算,dilate(erode(src,element))表示先进行腐蚀,再进行膨胀。“礼帽”运算突出了比周围像素更亮的点,图2和图3分别为实际道路图像和礼帽运算图像。从图3中可以看出礼帽运算的结果不受全局光照的影响,换言之,结果只与像素与周边像素的大小关系决定,这一优点源于运算中的差分环节,这一优点与图片的梯度特征是一致的,但梯度特征对于中心倒影带来的严重干扰更加敏感。So the "top hat" operation can be defined as: dst=src-open(src,element)=src-dilate(erode(src,element)), where open(src,dlement) means open operation, dilate(erode(src, element)) means to corrode first and then expand. The "top hat" operation highlights the points that are brighter than the surrounding pixels. Figure 2 and Figure 3 are the actual road image and the top hat operation image respectively. It can be seen from Figure 3 that the result of the top hat calculation is not affected by global illumination. In other words, the result is only determined by the size relationship between the pixel and the surrounding pixels. This advantage comes from the difference link in the operation. This advantage is related to the gradient of the image. The features are consistent, but the gradient feature is more sensitive to the severe interference caused by the central reflection.
由“礼帽”运算直接进行阈值化已经能够得到较好的结果,但结果依赖于阈值的选择,因此利用模板滤波来实现。注意到道路线的特征不仅是具有明显的边缘,同时道路线也具有明显的宽度,因此考虑如表1所示的模板对图像进行卷积,之后再对卷积图像进行阈值化,采取固定的阈值(灰度值220),得到二值化图像,如图4所示。但是,其道路线为不连续的线条,因此再进行帧间叠加使得道路线成为连续的线。Thresholding directly by the "top hat" operation has been able to get better results, but the result depends on the selection of the threshold, so template filtering is used to achieve it. Notice that the feature of the road line is not only the obvious edge, but also the obvious width of the road line, so consider the template shown in Table 1 to convolve the image, and then threshold the convolved image, and take a fixed Threshold (gray value 220) to obtain a binarized image, as shown in Figure 4. However, the road line is a discontinuous line, so inter-frame superposition is performed to make the road line a continuous line.
表1Table 1
道路线不总是连续的,例如,图4所示,道路线左侧连续,而右侧为间断的虚线。而公路上常见的虚线类型又有长虚线(如图4)和短虚线两种,可以对三种虚线类型分别处理,但对帧间信息的帧间叠加可以使原本间断的道路线成为连续的直线,从而得到最终的二值化图像,如图5所示。The road lines are not always continuous. For example, as shown in FIG. 4, the road lines are continuous on the left and intermittent dotted lines on the right. There are two types of dashed lines commonly seen on highways: long dashed lines (as shown in Figure 4) and short dashed lines. The three types of dashed lines can be processed separately, but the inter-frame superposition of inter-frame information can make the originally intermittent road lines become continuous. straight line, so as to obtain the final binarized image, as shown in Figure 5.
生成模块200用于根据二值化图像中的白像素点确定道路的边界以生成道路边界线。The generation module 200 is used to determine the boundary of the road according to the white pixels in the binarized image to generate the road boundary line.
具体地,对于道路的数学模型有多种选择,直线模型的拟合最为简单,也有包含更多参数的高阶多项式模型及双曲线模型,本系统选择了直线模型,其主要原因除了计算简单的考虑外还在于:对于路面符号的检测范围仅限于车前一小段距离,即使是曲率很大的路面(类似于立交桥上的环形弯道),在车辆前方的一小段距离内也可以大致近似成直线,而曲线模型带来的关于道路在远方的趋势信息对于近处的路面符号检测没有帮助。Specifically, there are many choices for the mathematical model of the road. The straight line model is the easiest to fit, and there are also high-order polynomial models and hyperbolic models that contain more parameters. This system chooses the straight line model. Another consideration is that the detection range of road symbols is limited to a short distance in front of the vehicle. Even a road with a large curvature (similar to a circular curve on an overpass) can be roughly approximated to a small distance in front of the vehicle. Straight lines, and the trend information about the road in the distance brought by the curve model is not helpful for the detection of nearby road signs.
在本发明的一个实施例中,在得到二值化图像的基础上,需要得到道路线的准确参数,即确定道路的边界线。通过采用基于经验规则的逐行搜索方法,这种方法具有计算快速的优点,虽然这种方法容易受到噪声以及路面符号的干扰,但如果适当利用帧间信息,上述干扰的影响可以被大幅消减。In one embodiment of the present invention, on the basis of obtaining the binarized image, it is necessary to obtain accurate parameters of the road line, that is, to determine the boundary line of the road. By adopting a line-by-line search method based on empirical rules, this method has the advantage of fast calculation. Although this method is susceptible to interference from noise and road symbols, if the inter-frame information is properly used, the impact of the above interference can be greatly reduced.
在本发明的一个实施例中,在二值化图像中按行进行搜素,从中线向左右两侧开始搜索,每行存储搜索得到的第一个白色点,将这个点当作可能的道路线点,每向道路点集合中加入一个新的点,下一次的搜索范围就在上一个点的附近。直到连续出现多次搜索失败或达到手工设定的边界时认为搜索结束,对已有的结果进行直线拟合,若相关系数大于某个阈值(例如0.99)则接受搜索结果,否则就从集合中的第一个点的附近选取一新起始点,继续进行搜索过程。In one embodiment of the present invention, the binary image is searched row by row, starting from the center line to the left and right sides, each row stores the first white point obtained from the search, and takes this point as a possible path Route point, every time a new point is added to the way point set, the next search range is near the previous point. The search is considered to be over until multiple consecutive search failures occur or the manually set boundary is reached, and the existing results are fitted with a straight line. If the correlation coefficient is greater than a certain threshold (for example, 0.99), the search result is accepted, otherwise, it is selected from the collection. Select a new starting point near the first point of , and continue the search process.
典型情况下成功的搜索如图6所示,圆圈代表搜索的点的轨迹,搜索左侧道路线时,首先从中线开始向左搜索,将遇到的第一个白像素加入道路线集合,之后每次的搜索都在之前的上一行,而水平方向上的搜索范围在之前点的附近,如此很大程度上地减少了搜索量。在搜索过程中可能会遇到并非道路线上的像素但其灰度值为255,由于以这样的像素点为开始的后续的搜索无法找到在同一直线上的多个白色像素,这样的错误初始点会被抛弃。通过这种方式可得到一个类似直线型的白像素点集合。A typical successful search is shown in Figure 6. The circle represents the trajectory of the searched point. When searching for the left road line, first search from the center line to the left, add the first white pixel encountered to the road line set, and then Each search is on the previous line, and the search range in the horizontal direction is near the previous point, which greatly reduces the amount of search. During the search process, pixels that are not on the road line may be encountered but their gray value is 255. Since the subsequent search starting with such a pixel point cannot find multiple white pixels on the same line, such an initial error points will be discarded. In this way, a set of white pixels similar to a straight line can be obtained.
从所得到的白像素点集合中各点位于道路内侧边缘上的位置之后,采用最小二乘拟合生成道路的边界线。最小二乘拟合是一种直线优化技术,已知一组点(xi,yi)位于直线x=ky+b上,已知对于yi的测量是准确的,全部的误差来自于x坐标(即列的位置是有误差的),最小二乘方法给出了最小化误差平方k、b的估计, 其中,Sx=∑xi为白像素点x坐标之和,xi为白像素点x坐标,Sy=∑yi为白像素点y坐标之和,yi为白像素点y坐标,Sxy=∑xiyi, 为白像素点坐标的相应运算,为参数k的估计值,为参数b的估计值。From the position of each point in the obtained white pixel point set on the inner edge of the road, the least squares fitting is used to generate the boundary line of the road. Least squares fitting is a straight line optimization technique. It is known that a set of points ( xi , y i ) are located on the straight line x=ky+b, and the measurement of y i is known to be accurate, and all errors come from x coordinates (i.e. the position of the column is in error), the least squares method gives an estimate of the minimized error square k, b, Wherein, S x =∑x i is the sum of the x coordinates of the white pixels, x i is the x coordinates of the white pixels, S y =∑y i is the sum of the y coordinates of the white pixels, and y i is the y coordinates of the white pixels, S xy =∑x i y i , is the corresponding operation of white pixel coordinates, is the estimated value of parameter k, is the estimated value of parameter b.
在本发明的一个实施例中,道路线搜索算法中确实包含了对于噪声的抵抗能力,但仅限于不在同一直线上的噪声,而道路间的符号包含了数量众多的白像素点,很容易找出在同一直线上的若干个像素,从而造成搜索算法不能正常工作。假设所得到的上一帧图像是正确的,这个信息就为消除道路中间的符号提供了重要的依据,因为只要将两条斜边延长交于消失点,两条斜线与水平的底线组成了三角形的道路区,道路区外是道路线,而道路区内是可能造成干扰的道路间的符号,因此去除干扰的方法就是将道路区内的像素都置成黑色,由此可准确的得出道路的边界线。In one embodiment of the present invention, the road line search algorithm does include the ability to resist noise, but it is only limited to noise that is not on the same straight line, and the symbols between roads contain a large number of white pixels, which are easy to find Several pixels that appear on the same straight line cause the search algorithm to not work properly. Assuming that the previous frame image obtained is correct, this information provides an important basis for eliminating the sign in the middle of the road, because as long as the two hypotenuses are extended and intersected at the vanishing point, the two oblique lines and the horizontal bottom line form a In the triangular road area, road lines are outside the road area, and road signs that may cause interference are inside the road area. Therefore, the method to remove interference is to set all pixels in the road area to black, and it can be accurately obtained The boundary line of the road.
检测模块300用于从道路图像中检测与道路边界线对应的道路线之间是否有引导标识,其中,引导标识包括公交专用标识和方向标识。The detection module 300 is used to detect from the road image whether there is a guide sign between the road lines corresponding to the road boundary line, wherein the guide sign includes a bus-only sign and a direction sign.
具体地,引导标识有引导标识模板,由于其结构简单可通过引导标识模板与处理所获得的引导标识进行比较,从而确定是何种引导标识。引导标识包括公交专用标识和方向标识,如图7所示。Specifically, the guide sign has a guide sign template. Due to its simple structure, the guide sign template can be compared with the guide sign obtained through processing to determine what kind of guide sign it is. Guidance signs include bus-specific signs and direction signs, as shown in Figure 7.
处理模块400用于当引导标识为方向标识时,根据方向标识和车辆状态信息信息判断车辆是否违规行驶。The processing module 400 is used for judging whether the vehicle is driving illegally according to the direction sign and the vehicle state information when the guide sign is a direction sign.
在本发明的一个实施例中,确定道路线后,为了更加稳定地进行模式分类,需要将摄像机的平视视图变换为俯视视图,这样的变换对于匹配非常重要,因为在平视视图中摄像机的水平位移会导致符号的形状发生变化,而俯视视图中,只要正确地确定了道路线,符号的形状就是大致固定的。经过转换后,俯视块中有很大可能存在着标识,下面通过模板匹配的方法决定标识的种类。即定义图片与模板间的相关系数C(I,Ii),其中Ii是引导标识i的模板,图像的分类按照如下方式确定:x=argmaxiC(I,Ii),其中,x表示最终确定的引导标识类别。In one embodiment of the present invention, after the road line is determined, in order to perform pattern classification more stably, it is necessary to transform the head-up view of the camera into a top-down view. Such transformation is very important for matching, because the horizontal displacement of the camera in the head-up view would cause the shape of the symbol to change, whereas in an overhead view, the shape of the symbol is roughly fixed as long as the road lines are correctly identified. After conversion, it is very likely that there is a logo in the top-view block, and the type of logo is determined by the method of template matching. That is to define the correlation coefficient C(I,I i ) between the picture and the template, where I i is the template of the guide logo i, and the classification of the image is determined as follows: x=argmax i C(I,I i ), where, x Indicates the finalized bootstrap identity category.
也就是在所有模板中寻找与图片相关度最高的一幅。同时为了处理没有包含在模板库中的标识并且增强对噪声的抵抗能力,当最大相关系数仍旧较小时认为识别失败,放弃对这一符号的识别,于是分类依据可以完整地表述为:
相关系数的选择对于分类的效果有着重要的影响,就这一问题研究者提出了多种相关系数的计算方法,需要在不同的场合择优选取。The choice of correlation coefficient has an important impact on the effect of classification. On this issue, researchers have proposed a variety of calculation methods for correlation coefficient, which need to be selected on different occasions.
在本发明的一个实施例中,最常用的实向量的相似性测度包括内积、余弦相关系数、Pearson相关系数和汉明测度。In one embodiment of the present invention, the most commonly used similarity measures of real vectors include inner product, cosine correlation coefficient, Pearson correlation coefficient and Hamming measure.
(1)内积(1) inner product
定义为C(x,y)=xTy=∑xiyi,其中,xi与yi分别表示向量x与y的坐标。在大多数情况下,当向量x和y被归一化而具有相同的长度a时,C的上限和下限分别为+a2和-a2。It is defined as C(x,y)=x T y=∑x i y i , where x i and y i represent the coordinates of vectors x and y respectively. In most cases, when the vectors x and y are normalized to have the same length a, the upper and lower bounds of C are +a 2 and -a 2 , respectively.
(2)余弦相关系数(2) Cosine correlation coefficient
余弦相关系数与内积相关较大,定义为:其中,x和y分别表示向量。这一相关系数有着向量夹角余弦的几何意义,也是归一化于-1到+1之间的相关系数,而且是旋转不变的。The cosine correlation coefficient has a large correlation with the inner product and is defined as: where x and y represent vectors, respectively. This correlation coefficient has the geometric meaning of the cosine of the vector angle, and is also a correlation coefficient normalized between -1 and +1, and is invariant to rotation.
(3)Pearson相关系数。其表达式是:其中,xd=(x1-mx,x2-mx,…,xn-mx)表示向量x与均值的差值坐标,yd=(y1-my,y2-my,…,yn-my)表示向量y与均值的差值坐标,表示向量x的均值,表示向量y的均值,xi与yi分别为向量x与y的坐标。(3) Pearson correlation coefficient. Its expression is: Among them, x d =(x 1 -m x ,x 2 -m x ,…,x n -m x ) represents the difference coordinates between the vector x and the mean value, y d =(y 1 -m y ,y 2 -m y ,…,y n -m y ) represent the coordinates of the difference between the vector y and the mean, represents the mean of the vector x, Indicates the mean value of vector y, and x i and y i are the coordinates of vector x and y respectively.
(4)汉明测度,设
根据C(x,y)获得引导标识和引导标识模板的特征相似度,当相似度即相关系数小于阈值时,则引导标识与相对应的引导标识模板相同,从而确定引导标识的类别。在本发明的一个实施例中,由于余弦相关系数能够取得较好的分类效果,因此采用余弦相关系数作为引导标识和引导标识模板之间的相似性度量方法。According to C(x, y), the feature similarity between the guide logo and the guide logo template is obtained. When the similarity, that is, the correlation coefficient is less than the threshold, the guide logo is the same as the corresponding guide logo template, thereby determining the category of the guide logo. In one embodiment of the present invention, since the cosine correlation coefficient can achieve a better classification effect, the cosine correlation coefficient is used as a similarity measurement method between the guide logo and the guide logo template.
本发明所要识别的违规驾驶包括直行线左、右转,转弯未开转向灯,占用公交车道和左、右转弯车道直行。The illegal driving to be identified by the present invention includes turning left and right on the straight line, turning without turning on the turn signal, occupying the bus lane and going straight on the left and right turning lanes.
图8为根据本发明一个实施例的直行线上左、右转弯的示意图。如图8所示,车辆在直行车道上,不应该存在左、右转弯的驾驶行为,若驾驶人想左、右转弯应该变更至左右转弯车道上。在直行线上转弯,即使开启了转向车灯,后方驾驶员也会误以为前方车辆为换线行为,造成后方驾驶人未能有效解读出前方驾驶人的操作意图,没有一定的心理及行为上的准备,造成安全隐患。Fig. 8 is a schematic diagram of left and right turns on a straight line according to an embodiment of the present invention. As shown in Figure 8, when the vehicle is on the straight lane, there should be no left or right turning driving behavior. If the driver wants to turn left or right, he should change to the left and right turning lanes. Turning on a straight line, even if the turn lights are turned on, the driver behind will mistakenly think that the vehicle in front is changing lanes, causing the driver behind to fail to effectively interpret the operation intention of the driver in front, without certain psychological and behavioral preparations, causing safety hazards.
路面标识检测到直行标识时,检测方向盘转角信号,若方向盘转角在一定阈值内变化,说明当前状态下,车辆没有转弯行为;若方向盘转角超过了一定的阈值,说明车辆有转弯行为。若检测到车辆有转弯行为,则违规程序输出预警信息。When the road sign detects the straight-going sign, the steering wheel angle signal is detected. If the steering wheel angle changes within a certain threshold, it means that the vehicle has no turning behavior in the current state; if the steering wheel angle exceeds a certain threshold, it means that the vehicle has turning behavior. If a turning behavior of the vehicle is detected, the violation program outputs early warning information.
在本发明的一个实施例中,车辆转弯行驶时,一般伴随着降低车速的驾驶行为,若此时未开启转向灯,跟随车辆未能有效辨识出前车的驾驶意图,依旧直线快速行驶,此时便会造成严重的交通事故。转弯未开转向灯行为,危害巨大。In one embodiment of the present invention, when the vehicle is turning, it is generally accompanied by the driving behavior of reducing the speed of the vehicle. If the turn signal is not turned on at this time, the following vehicle cannot effectively recognize the driving intention of the vehicle in front and is still driving in a straight line. Serious traffic accidents will result. The behavior of turning without turning on the turn signal is extremely harmful.
路面标识检测到左转或右转标识时,说明车辆行驶在转向车道上。此时检测方向盘转角信号,若方向盘转角在一定的转角阈值内,驾驶人员开启转向灯,说明驾驶人已经产生了转向意识,且开启了转向车灯。若超过一定的转角阈值,驾驶人员仍未开启转向车灯,则违规程序输出预警信息,通知驾驶人应该开启转向灯。道路设专用车道的,在专用车道内,只准许规定的车辆通行,其他车辆不得进入专用车道内行驶。When the road marking detects a left-turn or right-turn sign, it means that the vehicle is driving in the turning lane. At this time, the steering wheel angle signal is detected. If the steering wheel angle is within a certain threshold value, the driver will turn on the turn signal, indicating that the driver has become aware of the steering and has turned on the turn signal. If a certain corner threshold is exceeded and the driver has not turned on the turn signal, the violation program will output an early warning message to inform the driver that the turn signal should be turned on. If a road has a special lane, only the specified vehicles are allowed to pass in the special lane, and other vehicles are not allowed to enter the special lane.
图9为根据本发明一个实施例的公交车道占用示意图。如图9所示,在道路资源严重稀缺的现实中,设置公交专用道是为了解决大多数人的出行问题。公交专用道的设置,提升了市民乘坐公交的热情,公交专用车道用于提醒市民应改变出行方式,减少驾驶私家车,选择公共交通,从而缓解交通拥堵。在交通运输高峰时期,占用公交车道,势必造成交通拥堵,影响交通运力。路面标识检测到“公”、“交”、“车”、“道”标识时,说明当前情况下,车辆行驶在公交专用车道上,占用了公交车道,违规程序发出预警提示信息,通知驾驶人变更车道,从当前车道变更至其他车道。Fig. 9 is a schematic diagram of bus lane occupancy according to an embodiment of the present invention. As shown in Figure 9, in the reality of severe scarcity of road resources, the purpose of setting up bus lanes is to solve the travel problem of most people. The setting of bus-only lanes has enhanced citizens' enthusiasm for taking buses. Bus-only lanes are used to remind citizens to change their travel methods, reduce driving private cars, and choose public transportation, thereby alleviating traffic congestion. During the peak period of traffic transportation, occupying bus lanes will inevitably cause traffic congestion and affect traffic capacity. When the signs of "public", "traffic", "vehicle" and "road" are detected on the road surface markings, it means that under the current situation, the vehicle is driving on the bus lane and occupies the bus lane. A person changes lanes from the current lane to another lane.
图10为根据本发明一个实施例的左、右转弯车道上直行的示意图。如图10所示,设立左、右转车道是为了便于车辆转弯行驶。车辆转弯行驶时,一般伴随着减速行为。若在左、右转车道上,前方车辆有转弯行为,则其开始减速,并开启转向车灯,若此时自车仍高速行驶在左、右转车道上,未能有效意识到前车的转弯行为,则很有可能与前车发生碰撞,造成严重的交通安全事故。Fig. 10 is a schematic diagram of going straight on left and right turning lanes according to an embodiment of the present invention. As shown in Figure 10, the purpose of setting up left and right turn lanes is to facilitate the turning of vehicles. When a vehicle is turning, it is generally accompanied by deceleration behavior. If the vehicle in front turns on the left or right turn lane, it will start to slow down and turn on the turn lights. Turning behavior, it is very likely to collide with the vehicle in front, causing serious traffic safety accidents.
路面标识检测到左转或右转标识时,说明车辆行驶在转向车道上。此时检测车速信号,将车速对时间积分,若在一定的距离范围内,驾驶人员开启转向灯,说明驾驶人的驾驶意图为转向行驶,且已经意识到自车行驶在转向车道上,开启了转向车灯准备换道。若超过一定的距离阈值,驾驶人员仍未开启转向车灯,依旧直线行驶,则违规程序输出预警信息。When the road marking detects a left-turn or right-turn sign, it means that the vehicle is driving in the turning lane. At this time, the vehicle speed signal is detected, and the vehicle speed is integrated with time. If the driver turns on the turn signal within a certain distance range, it means that the driver’s driving intention is to turn, and the driver has realized that the car is driving on the turn lane. The turn signal is ready to change lanes. If a certain distance threshold is exceeded and the driver still drives straight without turning on the turn lights, the violation program will output an early warning message.
根据本发明实施例的系统,通过生成道路图像的二值化图像,并在该二值化图像中判别引导标识识别驾驶是否规范,提前预防了交通事故的发生,进而提高了驾驶的安全性。According to the system of the embodiment of the present invention, by generating the binarized image of the road image, and judging whether the guide sign recognizes the driving standard in the binarized image, the occurrence of traffic accidents can be prevented in advance, and the driving safety can be improved.
尽管上面已经示出和描述了本发明的实施例,可以理解的是,上述实施例是示例性的,不能理解为对本发明的限制,本领域的普通技术人员在不脱离本发明的原理和宗旨的情况下在本发明的范围内可以对上述实施例进行变化、修改、替换和变型。Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be construed as limitations to the present invention. Variations, modifications, substitutions, and modifications to the above-described embodiments are possible within the scope of the present invention.
Claims (8)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201310077048.8A CN103116748B (en) | 2013-03-11 | 2013-03-11 | Based on the method and system of road surface identification identification violation driving behavior |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201310077048.8A CN103116748B (en) | 2013-03-11 | 2013-03-11 | Based on the method and system of road surface identification identification violation driving behavior |
Publications (2)
Publication Number | Publication Date |
---|---|
CN103116748A true CN103116748A (en) | 2013-05-22 |
CN103116748B CN103116748B (en) | 2016-03-23 |
Family
ID=48415120
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201310077048.8A Expired - Fee Related CN103116748B (en) | 2013-03-11 | 2013-03-11 | Based on the method and system of road surface identification identification violation driving behavior |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN103116748B (en) |
Cited By (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103886745A (en) * | 2013-11-25 | 2014-06-25 | 天津思博科科技发展有限公司 | Automobile-dedicated lane early warning system |
CN105679092A (en) * | 2016-01-29 | 2016-06-15 | 深圳市美好幸福生活安全系统有限公司 | Driving behavior analysis system and method |
CN105761525A (en) * | 2016-05-11 | 2016-07-13 | 南京信息职业技术学院 | System device for warning car to enter bus lane |
CN106004883A (en) * | 2016-05-24 | 2016-10-12 | 北京小米移动软件有限公司 | Vehicle violation reminding method and device |
CN109740424A (en) * | 2018-11-23 | 2019-05-10 | 深圳市华尊科技股份有限公司 | Traffic violations recognition methods and Related product |
CN109987090A (en) * | 2018-01-03 | 2019-07-09 | 奥迪股份公司 | Driving assistance system and method |
CN110455302A (en) * | 2018-05-08 | 2019-11-15 | 奥迪股份公司 | Navigation system control method, device, computer equipment and storage medium |
CN111409649A (en) * | 2019-01-04 | 2020-07-14 | 奥迪股份公司 | Early warning method and device for vehicle lane change, computer equipment and storage medium |
CN111815959A (en) * | 2020-06-19 | 2020-10-23 | 浙江大华技术股份有限公司 | Vehicle violation detection method and device and computer readable storage medium |
CN111833629A (en) * | 2019-04-23 | 2020-10-27 | 上海博泰悦臻网络技术服务有限公司 | Special lane prompting method and system and vehicle |
CN112699825A (en) * | 2021-01-05 | 2021-04-23 | 上海博泰悦臻网络技术服务有限公司 | Lane line identification method and device |
WO2021077760A1 (en) * | 2019-10-23 | 2021-04-29 | 江苏智通交通科技有限公司 | Abnormal driving early warning method on basis of reasonable driving range of vehicle at intersection |
CN112949470A (en) * | 2021-02-26 | 2021-06-11 | 上海商汤智能科技有限公司 | Method, device and equipment for identifying lane-changing steering lamp of vehicle and storage medium |
CN114485671A (en) * | 2022-01-24 | 2022-05-13 | 轮趣科技(东莞)有限公司 | Automatic turning method and device for mobile equipment |
CN114619950A (en) * | 2020-12-14 | 2022-06-14 | 宝能汽车集团有限公司 | Control method of vehicle steering lamp, storage medium, electronic device and vehicle |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN201149729Y (en) * | 2007-12-28 | 2008-11-12 | 四川川大智胜软件股份有限公司 | Video traffic sign alarm |
CN101470807A (en) * | 2007-12-26 | 2009-07-01 | 河海大学常州校区 | Accurate detection method for highroad lane marker line |
-
2013
- 2013-03-11 CN CN201310077048.8A patent/CN103116748B/en not_active Expired - Fee Related
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101470807A (en) * | 2007-12-26 | 2009-07-01 | 河海大学常州校区 | Accurate detection method for highroad lane marker line |
CN201149729Y (en) * | 2007-12-28 | 2008-11-12 | 四川川大智胜软件股份有限公司 | Video traffic sign alarm |
Cited By (20)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103886745A (en) * | 2013-11-25 | 2014-06-25 | 天津思博科科技发展有限公司 | Automobile-dedicated lane early warning system |
CN105679092A (en) * | 2016-01-29 | 2016-06-15 | 深圳市美好幸福生活安全系统有限公司 | Driving behavior analysis system and method |
CN105679092B (en) * | 2016-01-29 | 2018-05-04 | 深圳市美好幸福生活安全系统有限公司 | A kind of driving behavior analysis system and method |
CN105761525A (en) * | 2016-05-11 | 2016-07-13 | 南京信息职业技术学院 | System device for warning car to enter bus lane |
CN106004883A (en) * | 2016-05-24 | 2016-10-12 | 北京小米移动软件有限公司 | Vehicle violation reminding method and device |
CN106004883B (en) * | 2016-05-24 | 2018-12-11 | 北京小米移动软件有限公司 | The method and device that rule-breaking vehicle is reminded |
CN109987090A (en) * | 2018-01-03 | 2019-07-09 | 奥迪股份公司 | Driving assistance system and method |
CN110455302B (en) * | 2018-05-08 | 2023-10-20 | 奥迪股份公司 | Navigation system control method, device, computer equipment and storage medium |
CN110455302A (en) * | 2018-05-08 | 2019-11-15 | 奥迪股份公司 | Navigation system control method, device, computer equipment and storage medium |
CN109740424A (en) * | 2018-11-23 | 2019-05-10 | 深圳市华尊科技股份有限公司 | Traffic violations recognition methods and Related product |
CN111409649A (en) * | 2019-01-04 | 2020-07-14 | 奥迪股份公司 | Early warning method and device for vehicle lane change, computer equipment and storage medium |
CN111409649B (en) * | 2019-01-04 | 2023-10-20 | 奥迪股份公司 | Early warning method and device for lane change of vehicle, computer equipment and storage medium |
CN111833629A (en) * | 2019-04-23 | 2020-10-27 | 上海博泰悦臻网络技术服务有限公司 | Special lane prompting method and system and vehicle |
WO2021077760A1 (en) * | 2019-10-23 | 2021-04-29 | 江苏智通交通科技有限公司 | Abnormal driving early warning method on basis of reasonable driving range of vehicle at intersection |
CN111815959A (en) * | 2020-06-19 | 2020-10-23 | 浙江大华技术股份有限公司 | Vehicle violation detection method and device and computer readable storage medium |
CN114619950A (en) * | 2020-12-14 | 2022-06-14 | 宝能汽车集团有限公司 | Control method of vehicle steering lamp, storage medium, electronic device and vehicle |
CN112699825A (en) * | 2021-01-05 | 2021-04-23 | 上海博泰悦臻网络技术服务有限公司 | Lane line identification method and device |
CN112949470A (en) * | 2021-02-26 | 2021-06-11 | 上海商汤智能科技有限公司 | Method, device and equipment for identifying lane-changing steering lamp of vehicle and storage medium |
CN114485671A (en) * | 2022-01-24 | 2022-05-13 | 轮趣科技(东莞)有限公司 | Automatic turning method and device for mobile equipment |
CN114485671B (en) * | 2022-01-24 | 2024-06-07 | 轮趣科技(东莞)有限公司 | Automatic turning method and device for mobile equipment |
Also Published As
Publication number | Publication date |
---|---|
CN103116748B (en) | 2016-03-23 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN103116748B (en) | Based on the method and system of road surface identification identification violation driving behavior | |
CN103324930B (en) | A license plate character segmentation method based on gray histogram binarization | |
Satzoda et al. | Multipart vehicle detection using symmetry-derived analysis and active learning | |
CN104992145B (en) | A kind of square samples track tracking detection method | |
JP6237214B2 (en) | Reverse running detection device | |
Chiang et al. | Detecting and recognizing traffic lights by genetic approximate ellipse detection and spatial texture layouts | |
KR101517181B1 (en) | System and method for warning lane departure | |
JP6353448B2 (en) | Method for measuring driving lane transitions for automobiles | |
TWI493513B (en) | Lane departure warning system and lane identification apparatus and related method thereof | |
Park et al. | Robust range estimation with a monocular camera for vision‐based forward collision warning system | |
CN103984950B (en) | A kind of moving vehicle brake light status recognition methods for adapting to detection on daytime | |
WO2018059585A1 (en) | Vehicle identification method and device, and vehicle | |
CN107066986A (en) | A kind of lane line based on monocular vision and preceding object object detecting method | |
US8385601B2 (en) | In-vehicle white line recognition apparatus | |
JP2015534152A5 (en) | ||
KR20150102546A (en) | Apparatus and method for recognizing lane | |
CN104240515A (en) | Exclusive bus lane occupation snapshotting method based on image processing | |
Parvin et al. | Vision-based on-road nighttime vehicle detection and tracking using taillight and headlight features | |
JP5233696B2 (en) | Lane boundary detection device, boundary detection program, and departure warning device | |
CN110335467A (en) | A Method of Using Computer Vision to Realize Vehicle Behavior Detection on Expressway | |
JP7214329B2 (en) | Display content recognition device and vehicle control device | |
TWI549102B (en) | Lane departure warning system and lane identification apparatus and related method thereof | |
CN103295003B (en) | A kind of vehicle checking method based on multi-feature fusion | |
EP3522073A1 (en) | Method and apparatus for detecting road surface marking | |
Guo et al. | CADAS: A multimodal advanced driver assistance system for normal urban streets based on road context understanding |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
C14 | Grant of patent or utility model | ||
GR01 | Patent grant | ||
CF01 | Termination of patent right due to non-payment of annual fee | ||
CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20160323 |