BE1008236A3 - TRAFFIC MONITORING DEVICE. - Google Patents

Abstract

PCT No. PCT/BE95/00032 Sec. 371 Date Dec. 9, 1996 Sec. 102(e) Date Dec. 9, 1996 PCT Filed Apr. 7, 1995 PCT Pub. No. WO95/27962 PCT Pub. Date Oct. 19, 1995A traffic monitoring device comprising a picture recording unit, a traffic dectection zone determination unit and a picture analysis unit, the traffic detection zone determination unit of which being provided to determine as traffic detection zone a follower axis extending substantially in parallel with a traffic axis in said traffic road and situated thereon, and in that said picture analysis unit is further provided to realize said verification pointwise on predetermined points situated on said follower axis and upon detection of such an object to assign thereon an identification pattern and to check within subsequent pictures within said sequence if patterns corresponding with said identification pattern occur.

Description

Traffic monitoring device

The invention relates to a traffic monitoring device comprising: - an image recording unit for recording a sequence of consecutive traffic images of the same traffic route; a detection zone determining unit connected to the image recording unit and provided for determining a traffic detection zone in the traffic images from said sequence; - an image analysis unit provided to cooperate with the image recording unit and with the detection zone determination unit and to check from each offered traffic image within said traffic detection zone whether an object identifiable as a vehicle is present there;

Such a traffic monitoring device is known and is marketed by Wootton Jeffreys Consultants under the name Impacts. An image recording unit formed by a T.V. camera, records consecutive images of a traffic stream. Within each image, a detection zone is determined within which the traffic flow will be analyzed. The image analysis unit examines the image content of the detection zone to determine whether an object identifiable as a vehicle is present within this detection zone. This makes it possible to monitor traffic electronically and to check whether traffic jams are occurring, for example. The latter is done by checking how many objects identifiable as a vehicle are present within the recorded image.

In the known traffic monitoring device, a rectangle is always used as the detection zone, which is superimposed on the image of the road to be monitored. For example, the road is divided into a plurality of rectangles. The dimension of each rectangle is chosen so that it can easily accommodate one vehicle. If it is now determined by the image analysis unit that there is more than one vehicle within such a rectangle, this is indicated on an image display because the color with which the contours of the rectangle is indicated changes, for example from blue to red. This therefore indicates where there is a problem within the traffic flow.

A drawback of the known traffic monitoring device is that by using rectangles the image analysis unit has to discount a large number of pixels, more than 2,000 pixels are discounted per image, which requires a considerable computing capacity. In addition, because traffic situations can change quickly, it is necessary in order to obtain a reliable system to have the images from the sequence follow in quick succession. This in turn imposes high demands on the computing capacity of the device.

A further drawback of the known traffic monitoring device is that no correlation is placed between these images when analyzing the successive images.

The object of the invention is to realize a traffic monitoring device in which, without detracting from the reliability of the device, a relatively small computing capacity is sufficient for the image analyzes, and in which a correlation between the successive images can also be determined.

To that end, a traffic monitoring device according to the invention is characterized in that said traffic detection zone determining unit is provided as a traffic detection zone to determine a full parallel extending with a traffic axis in said traffic route and located thereon, and that said image analysis unit is further provided for checking said point-wise on predetermined points located on said full throttle and to assign an identification pattern thereto upon detection of such an object and to check from successive images within said sequence whether patterns corresponding to said identification pattern occur. The choice of a full throttle as a traffic detection zone significantly reduces the number of pixels considered. As a result, the reliability of the device can be sufficient with a small computing capacity, since the full-throttle is located on the traffic axis. The traffic will therefore move along that full throttle, so that the reliability of the detection is guaranteed. Furthermore, by assigning an identification pattern to an object identified as a vehicle and checking whether this identification pattern repeats over successive images, the displacement of the vehicle can be observed because a correlation is thus placed between successive images.

A first preferred embodiment of a traffic monitoring device according to the invention is characterized in that said traffic detection zone determining unit is provided for superimposing a predetermined line segment on said traffic images for said full throttle. Determining the full throttle is therefore easy and reliable to achieve.

A second preferred embodiment of a traffic monitoring device according to the invention is characterized in that said image analysis unit is provided to determine a gray value of the pixel during said checking and to check whether said gray value exceeds a predetermined threshold, and to identify said identification pattern when exceeded. can be determined with said gray value. Certainly when working with video images, the gray value of a pixel can simply be determined and converted into digital form. Moreover, calculating with digitized grayscale can be carried out quickly, making the comparison with the threshold value quick and easy. In addition, this provides a reliable way to determine the identification pattern.

It is advantageous that said image analysis unit is provided to determine said gray value from an average value of n (n> 1) neighboring pixels. As a result, noise filtering is performed on the image data.

A third preferred embodiment of a traffic monitoring device according to the invention is characterized in that said image analysis unit is provided for applying a Laplacian operation to said gray level and determining a Laplacian operator therefrom and testing its value against said threshold. Determining a Laplacian operator is electronically feasible with simple means.

A fourth preferred embodiment of a traffic monitoring device according to the invention is characterized in that said image analysis unit is provided for delimiting a first image window located around the pixel to which said gray value belongs for each gray value that exceeds said threshold, and to display said identification pattern within said first image window. determine. The demarcation of the first image window offers the possibility to determine the identification pattern within that first image window. The identification pattern thus extends over several pixels, so that the search for corresponding patterns can be carried out with a higher degree of reliability.

A fifth preferred embodiment of a traffic monitoring device according to the invention has the feature that said image analysis unit is provided to determine a second image window from said first image window and to shift said second image window in steps of said full throttle with a predetermined incrementation, and to step that said second image window is shifted is said to check whether to perform corresponding patterns on steps. Thus, the identification pattern is shifted along with the second display window and the search for corresponding patterns is limited to the search within that second display window.

A favorable way of determining the correlation between the first and the second image window is characterized in that said image analysis unit is provided to determine a further identification pattern at said step within the second image window and to determine a correlation value from said further identification pattern and said identification pattern, and to check whether said correlation value exceeds a further threshold value.

The invention will now be explained in more detail with reference to the drawing, in which:

Figure 1 schematically shows an exemplary embodiment of a traffic monitoring device according to the invention;

Figure 2 illustrates the term "full throttle".

Figures 3 a + b in the form of a flow chart illustrate the functioning of a traffic monitoring device according to the invention /

Figures 4 a + b illustrate the term "first and second image window".

Figure 5 shows in the form of a flow chart an alternative embodiment of the functioning of a traffic monitoring device according to the invention.

In the drawing, the same or an analogous element is assigned the same reference numeral.

A traffic monitoring device according to the invention, of which an exemplary embodiment is shown in Figure 1, comprises an image recording unit 1, for instance formed by a CCD camera. The image recording unit is intended to be positioned along a road in order to record the traffic on the road. The recorded images are delivered to an analog-digital converter 2 where they are digitized. The converter 2 is connected to a bus 3 intended, inter alia, for the transport of data and instructions. A data processing unit 4, for example formed by a microprocessor, as well as a read memory 5 (RAM) and a read / write memory 6 (RAM) are connected to the bus 3. Furthermore, an image generator 7 is connected to the bus 3, an output of which is connected to an image display unit 8.

The image recording unit and the other components are not necessarily located in the same location. For example, said other components will preferably be located at the traffic control center of the competent authority.

In the data processing unit 4 and / or in the memories 5, 6 data and instructions with which an analysis of the recorded traffic image can be carried out are stored. The data processing unit 4 and the memories 5, 6 as well as the bus 3 thus form an image analysis unit. However, before going into the operation of the image analysis unit in more detail, the term "full throttle" will first be explained.

Figure 2 shows a traffic image in which a part of a traffic road 9 is shown, on which there is a vehicle 10 which moves in the direction of the arrow 11. This direction is the one of the traffic axis along which the traffic normally moves. The full-throttle 12 is now defined in a direction extending almost parallel to the traffic axis. In the example, figure 2 is parallel to arrow 11. The full throttle is located in the image in such a way that it is located on that part of the road on which the likelihood of a vehicle moving there is greatest. For example, when the full throttle is placed in a carriageway of a two-lane road, it will be shifted slightly in the middle or relative to the center to the right edge of the road (for right-hand traffic). For example, in a road with two or more lanes, as shown in figure 2, the strip on the far right resp. far left the full throttle 12 resp. 12 'positioned so that it is shifted slightly (e.g. 15%) to the right or left relative to the center of the strip.

The full throttle 12 (or 12 ") is predetermined when installing the image capture unit, at least when it has a fixed arrangement. Once the image recording unit is set, a full throttle is superimposed on the recorded image which is predetermined and thus remains unchanged. However, it is also possible to have the image recording unit change position, for example a first position during the morning rush hour and a second position during the evening rush hour. In the latter configuration, two predetermined full throttles are then operated and switched between a first and second full throttle depending on the position occupied by the image recording unit.

Instead of working with a predetermined full throttle, it is also possible to build up a full throttle strip in function of the local traffic density. This is useful, for example, at an intersection where every road user does not necessarily follow the same route when turning in one direction or another. By building up the full throttle in strips in function of the local traffic density at different points of the road or the intersection, it is possible to "update" the full throttle regularly. In addition, it avoids disturbing monitoring of the vehicle's progress if a vehicle should leave the predetermined full throttle. The full-throttle build-up of the full throttle occurs by starting with an initial strip and then varying each successive strip relative to its previous one by an angle and determining the angle with the highest probability or clarity of a vehicle being detected. The angular variation α is, for example, -15 ° <a <15 °. The angle at which the highest probability is determined is therefore chosen.

The zone within which the presence of traffic will be investigated is thus defined by means of the full throttle. To perform this detection, a number of (N) points X, (l <i <N) are recorded on the full throttle. The gray value of these points will now always be determined, which gives an indication of the presence of a vehicle there. This gray value is preferably represented by an 8 bit word which gives 256 values.

However, this detection is not limited to only the pixels Xf. To limit the noise, a strip 13 with a width of n (for example 1 <n <10) pixels is actually retrieved from the recorded image. Preferably, these n pixels are selected in a direction substantially perpendicular to the full throttle. After all, a vehicle always has a certain width. When the vehicle moves along the full throttle, the same gray level will be present in the image in a line of n pixels almost perpendicular to the full throttle. The width n of the selected strip may vary depending on the position in the image. Preferably, in line-wise image construction, the n pixels are chosen within the same image line, which simplifies calculation. However, the n pixels can also be selected along the full-throttle itself or form an angle β, 0 </ 3 <135 ° with respect to a full-axis, depending on the curvature of the full-axis within the recorded image.

The images recorded by the image pick-up unit are digitized and only those pixels located in a strip of n pixels around the full throttle as described above are discounted. For this purpose, for example, the digitized pixels are written into the memory 6 and only those pixels located on said strip are subsequently read and processed by the data processing unit 4. This selective reading is effected, for example, by selectively addressing the memory 6 by means of an address generator which is programmed in function of the position of the full throttle in the image. This address generator then functions as a detection zone determining unit to determine the correct traffic detection zone in the image. Other embodiments, such as selective writing in the memory, are of course also possible.

The flowchart shown in Figure 3a illustrates the selection of relevant pixels in a traffic monitoring device according to the invention.

The different steps in this flowchart will now be explained.

20. Xj = Xj + 1: The pixel X to be treated; is selected from the full throttle of the same image. For example, a modulo N counter is used for this purpose, which functions as an address generator. The counter counts sequentially with one increment in order to address the different pixels.

21 ^ —BB. X | : The gray value Xj of the selected pixel Xj is retrieved.

22. RD nXj: The gray value x (j of each of the n pixels, nearest neighbors of Xj, as discussed above is also retrieved (l <j <n). The value of n can vary here in function of the camera angle and the position of Xj in the picture.

23. — AV Xj: The Mathematical Operation

is now performed in order to determine an average value Xj taken over its nearest neighbors for the gray value of the pixel Xf and thereby suppress image noise. The division by (n + 1) can possibly be omitted, since the relative value and not the absolute value is important.

24. DT Xj_ ^: In order to further limit the image noise, another low-pass filtering is applied to the image signal. Mathematically this is expressed in the following operation:

where, based on the average gray value associated with pixel Xt, a substitution value x ('is determined by counting at xx the value x', and x, ^, of its nearest neighbors.

25. DT Ai: To check whether the value x '; is relevant, i.e. indicates the presence of a vehicle, an identification operation is carried out thereon. This identification operation can take various forms. For example, it can simply be checked whether the gray value x'j exceeds a predetermined threshold value. However, in order to take into account various factors, such as the light intensity, the camera sensitivity, wet road surface, etc., which strongly influence the absolute gray value of the pixel, the relative gray value is not considered, but the relative gray value. In this exemplary embodiment, on the basis of the gray value x '; a Laplaian operator calculated; Δί = X 'i_2 - 2X'; + x'1 + 2.

This makes it possible to check whether the point Xj with respect to the points X, .2, X1 + 2 has a clearly different gray value. Instead of calculating a Laplacian operator, it is also possible to determine the evolution of x'j relative to the average of its nearest neighbors xi ± p '(l <p <3).

26.I Ai I> TH_LA? : Here it is checked whether the absolute value of the Laplacian Ai exceeds a predetermined threshold value TH_LA. If this is not the case, step 28 is switched over.

27. ST xi: When the Laplacian operator | Ai | if the value exceeds TH_LA, then the point X is considered relevant. After all, exceeding the value TH_LA indicates the possible presence of a vehicle. The point X | and the associated value x '| are temporarily stored in a memory.

28. X „? : It is checked whether the point X considered; the last in the series of N points. If not, another point Xj + is taken and steps 20 through 28 repeated.

^ RL? : It is examined whether relevant points X (are stored in the memory. If not, a sub-sequential image from the image sequence recorded by the image recording unit is processed.

RT: presence of relevant points activates another routine (shown in Figure 3b). This other routine preferably runs simultaneously with the routine of figure 3a.

A graphical illustration of the gray value x i as a function of the position X; the full throttle is shown in figure 4a. This example shows that if, for example, TA_LA = 150, then a vehicle is probably present at pixels 7, 10, 28, 30 and 35, since relevant information is found there from which the Laplacian operator | Δΐ |> TA_LA. These points were therefore repressed to check whether this identification pattern repeats itself in sub-sequential images, shifted or not in the direction of full throttle. After all, a shift along the full throttle means that there is movement in traffic, while a stationary or a slow shift indicates congestion.

To check whether corresponding patterns occur in sub-sequence images, for each pixel x; for which | Δϊ |> TH_LA delineating a first image window. This is done in step 31 DTW.i of the other routine shown in the flow chart of Figure 3b. The first image window has a width of L pixels, for example L = 6 and is centered around the pixel Xj. The gray values of the pixels within that first window W1 now form an identification pattern which will be attempted to be found in the following images.

Thus, in the example shown in Figure 4a, a first image window W1 is placed around the pixel 7. The points 4 through 10 now belong to that first window and the identification pattern contains points 7 and 10 which had a Laplacian greater than TH LA. A further first image window is placed around pixels 28, 30 and 35.

To now check whether corresponding patterns, such as those from the first image windows, occur in subsequent traffic images, the data from the first image windows is temporarily stored until the steps shown in Figure 3a are again processed for a subsequent image. Now to distinguish the gray value data from that next image from that (x \) from the previous image, that from the next image will be referred to as y j. The gray value y'i is determined in an analogous way as xV. Figure 4b shows an example of gray values y \. For example, at pixel 12 there is a gray value y'j with a Laplacian | Δί |> bij_LA)

After the gray value y '; (y'i?: 32 figure 3b) was determined, a second display window W2 (DT W2; 33 figure 3b) is determined. That second image window is the same size as the first W1, and is initially positioned in the same location as the first image window. Then, that second display window is shifted in the direction of the full throttle (SH W2; 34), each time with a predetermined increment of, for example, one position on the full throttle. A correlation factor is determined with each shift of the second image window (DT CF, 35). This correlation factor CF is determined, for example, by the following mathematical operation:

(L + i is the total number of points of the full throttle located within the second image window. The correlation factor is minimal when the patterns almost correspond and increases as the difference between the patterns occurs. For each shift of the second image window, the correlation factor temporarily stored, linked to the corresponding position of the second image window.

After the determination of the correlation factor, it is checked whether the second window has already reached its maximum shift As (SH = As? 36). After all, a vehicle in motion will have traveled a limited distance between two consecutive images. Therefore, the shift of the second image window can be limited by a predetermined number, for example, As-6 pixels. If this maximum has not yet been reached, the second window is shifted by one pixel and steps 34 and 35 are repeated.

When the second display window has reached its maximum offset from the first (35, Y), then (DT MN: 37) that position of the second display window is selected with CF being the smallest value. After all, it is expected there to obtain a pattern corresponding to the identification pattern. Then it is checked whether that smallest value is smaller than a threshold value TH_COR for the correlation value (MN <TH_COR ?; 38). If yes, (38: y) then a subsequent calculation (PT; 39) such as the displacement of the vehicle is determined. This routine is then completed and a new determination can be started for the next image.

The calculation of the displacement or speed of the vehicle is done by determining the difference in the position of the two image windows. If MN> TH_COR, this meant that no corresponding patterns were found.

In the example shown in Figure 4, when the second display window W2 is shifted 5 positions relative to the first, a correspondence is obtained between the contents of the first and the second display window. Thus, the identification pattern present in the first image window has shifted 5 positions in the subsequent image, indicating that the associated vehicle has moved in the meantime.

The traffic monitoring device according to the invention thus makes it possible to monitor the progress of traffic in a reliable and simple manner. When traffic is stopped, the identification pattern will also freeze in the image. Correspondence between the image content of the first and second image windows will then be found at substantially identical positions in the image. Detecting such correspondence at identical image positions, or determining that the speed at which the vehicle is moving (step 38, Figure 3b) then gives rise to a traffic jam warning signal. That traffic jam warning signal is also generated when the sensed speed drops below a predetermined lower limit. After all, the latter situation indicates slow-moving traffic, which is characteristic of traffic jams. The traffic jam warning signal is disabled when the speed of the traffic rises above the set value again.

The device according to the invention is preferably further provided for measuring the duration of a traffic jam. To this end, a counter is started when generating the traffic jam warning signal which is then stopped when that signal is turned off.

In file detection it is beneficial to choose the full throttle long enough and thus be insensitive to small movements in the traffic jam. In order not to unnecessarily generate a signal for every short-term file, it is advantageous to generate that signal only when the file has existed continuously for a certain time, for instance 3 minutes.

Figure 5 shows a flow chart showing an alternative embodiment. Some steps are analogous to the routine shown in Figure 3a and therefore carry the same reference. The routine shown in figure 5 is intended to also use the device according to the invention as a traffic frequency / density counter. To this end, an improvement is made in the routine in which the determination of a Laplacian operator AXj> TH_LA will lead to the assumption that such a determination will probably take place again in the following image. To this end, the average gray value of the identification pattern is determined and compared with a reference value that is regularly adjusted. When the average gray value is less than or equal to a first value P which represents a background gray value and that repeats over a number of images (e.g. 5), it is determined with certainty that no vehicle is present. In particular, this calculation serves to discount a change in light intensity.

The flow chart shown in Figure 5 now includes the following steps: 40 · DTM: An average value M is calculated from the values x (

This value is determined over all points M of the full throttle.

43 ·· SW Q: When | Δϊ | > TH_LA indicates the presence of a vehicle and a vehicle presence signal V is generated and the frequency counters T, and T2 (step 53; T, = 0, T2 = 0) are reset. The function of these frequency counters T1 and T2 will be explained below.

42. SW? : It is checked whether a vehicle presence signal V has already been generated.

43. T, = Ti + 1: If no signal V has been generated, counter T3 is incremented. The counter T3 serves to refresh the reference value regularly in order to discount light intensity variations. The counter T3 is only incremented when | ΔΪ | <TH_LA and the signal V is not active. The counter T3 counts numbers of images.

44. T? = MX 3? : It is checked whether counter T3 indicates a maximum value. For example, the maximum value is equal to 10 images.

45. REF = M: If counter T3 indicates a maximum position, the first reference value P is substituted by the value M calculated in step 40. This sets an updated background gray value then counter T3 is reset.

46. M Ref 1? : It is checked whether the value L = M / P is located within a first region Q which, for example, represents 4% of the maximum intensity.

47. T | = T | +1: When LeQ then the counter T, is incremented. The counter T counts the number of sub-sequence images after a signal V has been generated, and for which | Ai | <TH_LA. Such a situation occurs, for example, when in an image, as a result of a malfunction or a sudden intensity variation, an object was identified as a vehicle (| Δϊ |> TH_LA), and in the subsequent images the same object is no longer identified.

48. T] = MX 1? : It is checked whether counter T, indicates a maximum value. For example, the maximum value is two images.

.4.9. swo: When T, indicates a maximum value, the vehicle presentation signal V is turned off and the vehicle frequency counter is incremented by one unit. After all, the signal V was generated. The value P is set equal to M and the counters T |, T2 and T3 are reset.

50y — M — Ref — 2? : It is checked whether the value L = M / P lies within a second region Q 'which, for example, represents 30% of the maximum intensity.

5Ij_jr2 = T2 ± I: When LeQ 'then counter T2 is incremented. The counter T2 has an analogous function to that of counter T1, but T2 counts those images within which a reasonable (greater than 30%) intensity is still present. Such a situation occurs, for example, with the passage of a long truck with a uniform insufficiently distinguishable color, such as white. The intensity is insufficient because | Ai |> TH_LA but the vehicle is present.

52 —T2 = MX2? : It is checked whether counter T2 indicates a maximum value, if so, step 49 is switched on. For example, the maximum value is 5 images.

Claims (12)

Traffic monitoring device comprising: - an image recording unit for recording a sequence of consecutive traffic images of the same traffic route; a detection zone determining unit connected to the image recording unit and provided for determining a traffic detection zone in the traffic images from said sequence; - an image analysis unit provided to cooperate with the image recording unit and with the detection zone determination unit and to check from each offered traffic image within said traffic detection zone whether an object identifiable as a vehicle is present there; characterized in that said traffic detection zone determining unit is provided as a traffic detection zone to determine a full throttle extending parallel to it with a traffic axis in said traffic route and situated thereon, and that said image analysis unit is further provided to point said check point-wise at predetermined points located on said full throttle. and to assign an identification pattern thereto upon detection of such an object and to check from successive images within said sequence whether patterns corresponding to said identification pattern occur. Traffic monitoring device according to claim 1, characterized in that said traffic detection zone determining unit is provided for superimposing a predetermined line segment on said traffic images for said full throttle. Traffic monitoring device according to claim 1 or 2, characterized in that said image analysis unit is provided for determining, upon said checking, a gray value of the pixel and checking whether said gray value exceeds a predetermined threshold, and if said identification pattern with said grayscale. Traffic monitoring device according to claim 3, characterized in that said image analysis unit is provided to determine said gray value from an average value of n (n> 1) neighboring pixels. Traffic monitoring device according to claim 4, characterized in that said image analysis unit is provided to determine said average value from neighboring pixels located substantially perpendicular to said full throttle. Traffic monitoring device according to claim 4 or 5, characterized in that said image analysis unit is further provided to determine said average value from neighboring pixels located on said full throttle. Traffic monitoring device according to any one of claims 3-6, characterized in that said image analysis unit is provided for applying a Laplacian operation to said gray level and deriving a Laplacian operator therefrom and testing its value against said threshold. Traffic monitoring device according to claim 3 or 7, characterized in that said image analysis unit is provided for delimiting a first image window located around the pixel to which said gray value belongs for each gray value that exceeds said threshold, and to define said identification pattern within said first image window. to decide . Traffic monitoring device according to claim 8, characterized in that said image analysis unit is provided to determine a second image window from said first image window and to move said second image window in stepwise manner along said full-throttle with a predetermined incrementation, and to indicate at each step that said second image window shifted is said to check whether to perform corresponding patterns on steps. Traffic monitoring device according to claim 9, characterized in that said image analysis unit is provided to determine a further identification pattern at said step within the second display window and to determine a correlation value from said further identification pattern and said identification pattern, and to check whether said correlation value exceeds a further threshold. Traffic monitoring device according to any one of claims 1-10, characterized in that said image analysis unit is provided to generate a vehicle presence signal when an identification pattern is assigned. Traffic monitoring device according to claim 11, characterized in that said image analysis unit is provided with background gray level compensation means.