JP2005032074A - Image detection apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image detection apparatus capable of preventing a blurred photographic image of a detecting object, even if the vehicle mounting the camera and/or the detecting object moves. <P>SOLUTION: The image detection apparatus includes an image processing chip 2 for extracting a detecting object candidate from the photographic image obtained by a camera 1; a microcomputer 5 for calculating the number of times of luminance improvement integrals from the difference between the luminance value of the detecting object candidate, extracted by the image processing chip 2 and the luminance value of the background; a vehicle sensor 4 for measuring the behavior of the vehicle concerned; and an integration chip 6 for performing image integration by the number of times of the integrals calculated by the microcomputer 5. The microcomputer 5 calculates integration conditions from the detecting object candidate extracted by the image processing chip 2, the behavior of the vehicle concerned, and a movement model of the detecting object; determines the number of times of the integrals from the number of times of the luminance improvement integrals and the integral condition; and detects the detecting object using the integrated image calculated by the integration chip 6. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an image detection device mounted on a vehicle.

Conventionally, a technology has been developed in which a sensor is mounted on a vehicle and a detection target existing in the vicinity of the host vehicle is detected. In particular, in a detection apparatus using a visible camera as a sensor, the brightness (luminance value) of a detection target on an image changes depending on an imaging environment such as time, weather, and illumination state (by fixed illumination or movable illumination). For this reason, it is important that the detection target object is clearly imaged and reliably detected in any environment. Many researches have been made on methods for reliably detecting a detection target even when the imaging environment changes. As one of them, there is a method using image integration that accumulates images continuously input from a camera. This method is compared with the control of camera exposure as follows: 1. Can integrate locally. This is advantageous in terms of function and cost, such as eliminating the need for a complicated mechanism for exposure control. As a method for detecting a detection target using image integration in this way, there is an “in-vehicle image storage device and in-vehicle image recognition device” described in Patent Document 1 below.
In this conventional example, for example, the exposure time of the camera is controlled in accordance with a signal received from the exposure time control means. The exposure time control means calculates, for example, the maximum edge intensity and the minimum edge intensity for each scanning line of the edge image obtained by performing edge processing on the captured image, and calculates the exposure time from the values.

JP 2000-285224 A

However, in the conventional technology as described above, since the image is improved by controlling the exposure time of the camera, the exposure is performed when either the vehicle equipped with the camera or the detection target to be detected moves. There is a problem that the detection target object moves on the screen within the time, and the captured image of the detection target object is blurred.
An object of the present invention is to provide an image detection device that can prevent blur of a captured image of a detection target even if a vehicle or a detection target mounted with a camera moves.

  In order to solve the above-described problems, an image detection apparatus according to the present invention is mounted on a host vehicle and captures an image of the vicinity of the host vehicle, and a detection target candidate extraction unit that extracts a detection target candidate from the captured image. An own vehicle behavior measuring means, a detection target candidate extracted by the detection target candidate extraction means, a behavior of the own vehicle, an integration condition calculating means for calculating an integration condition from the detection target motion model, and the integration An image integration unit that performs image integration under conditions and a detection target detection unit that detects a detection target using the integrated image calculated by the image integration unit are provided.

  According to the present invention, even if the host vehicle or the detection target moves, an image with no image blur and a large luminance difference can be obtained.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a block diagram showing the configuration of the image detection apparatus according to the present embodiment. As shown in FIG. 1, the image detection apparatus of the present embodiment includes an electronic camera 1, an image processing chip (IC) 2, a detected image memory 3, a vehicle sensor 4, a microcomputer 5, an integration chip (addition processing chip). IC) 6, processing image memory 7, and monitor 8 are connected.
The camera 1 is provided at a predetermined position of the host vehicle, for example, at the front, and images the vicinity of the host vehicle, here in front. The image processing chip 2 performs a filtering process with a predetermined image size for each scanning line of the image captured by the camera 1 to extract a detection target candidate (detection target candidate region). The detected image memory 3 converts the filter image calculated by the image processing chip 2 into a digital value and holds it. The vehicle sensor 4 measures the behavior of the host vehicle, here the rotation of the wheels. The microcomputer 5 inputs the digital image held in the detected image memory 3, and calculates the luminance improvement integration number from the difference between the luminance value of the detection target candidate and the luminance value of the background. Further, the microcomputer 5 calculates an integration condition from the digital image held in the detected image memory 3, the wheel rotational speed input from the vehicle sensor 4, and the detected object motion model. Further, the microcomputer 5 calculates an integration parameter from the brightness improvement integration number and the integration condition calculated by the microcomputer 5. The integration chip 6 performs image integration using the integration parameters calculated by the microcomputer 5. The processed image memory 7 converts the integrated image calculated by the integrating chip 6 into a digital value and holds it. Moreover, the microcomputer 5 inputs an integral image and detects a detection target object by pattern matching processing. The monitor 8 displays the detected detection object.

2A to 2H are diagrams illustrating the basic principle of extracting a detection target candidate from the captured image of the camera 1 by the image processing chip 2 in the present embodiment. FIG. 2A shows a configuration of the image processing chip 2 which is a spatial filter for extracting an object (detection target) having a predetermined image size. FIGS. 2B to 2E show processing of the image processing chip 2. FIG. 2B shows a captured image when the object is small, FIG. 2C shows a filter image when the object is small, and FIG. 2D shows a large object. (E) is a filter image when the object is large. 2 (f) to 2 (h) show images when extracting detection target candidates, (f) is a captured image, (g) is a spatial filter image, and (h) is a binarized image.
In FIG. 2A, 21 is a central pixel, 22 is one pixel, 23 is a background area, Tgt is a target size, and Bak is a background size. The image processing chip 2 in FIG. 2A has a pixel size of one pixel in the vertical direction and a predetermined background size Bak in the horizontal direction, and the luminance of the central pixel 21 is determined from the target luminance value and the background size Bak. If the average luminance of the background area 23 excluding the target size Tgt is a background luminance value, the output value is (target luminance value) − (background luminance value). When the filter processing by the image processing chip 2 is performed, as shown in FIGS. 2B to 2E, when the size of the imaged object is smaller than the target size Tgt of the spatial filter, the luminance remains as it is. If the size of the imaged detection target is larger than the target size Tgt, the luminance value is output as zero. Therefore, when the spatial filter process is performed on the captured image obtained by capturing the building 10, the lane mark 11, and the traveling vehicle 12 as illustrated in FIG. 2F, only an object having a predetermined filter size is obtained. For emphasis, a spatial filter image as shown in FIG. 2G from which the large-sized building 10 has been removed is obtained, and binarization processing is performed with a predetermined threshold value, so that FIG. A binarized image having an area matching the filter size as shown in FIG. By extracting an object indicating the size of the detection target from the binarized image of FIG. 2H calculated in this way, the lane mark 11 is removed, and the detection target that only the traveling vehicle 12 should detect is detected. Can be extracted as an object candidate.

3 to 6 show the detection target candidates extracted by the image processing chip 2 in the present embodiment, the behavior of the host vehicle measured by the vehicle sensor 4, and the integration condition by the microcomputer 5 from the detection target motion model. The basic principle of calculating is shown. 3A to 3C show a method for calculating the spatial position of the detection target object from the detection target candidate. FIG. 4 (a) shows the position and orientation of the vehicle and the detection object viewed from above in space, and FIG. 4 (b) shows the position and orientation of the vehicle and the detection object viewed from the side in space. . 5A and 5B show the relationship between the spatial position viewed from above the detection target and the pixel position on the image. 6A and 6B show the relationship between the spatial position viewed from the side of the detection object and the pixel position on the image.
In FIG. 3A, 31 is a camera visual axis, 32 is a detection target object candidate region, Xr, Xl, Yb, and Yt are camera visual axis shifts, and in FIGS. 3B and 3C, θl and θr are viewing angles, θt, θb is a viewing angle, and in (c), hs is a camera installation height of the host vehicle, and 33 is a detection object height.
In FIG. 4A, 41 is the host vehicle, 42 is a detection target, dθx is a camera visual axis change, and in FIG. 4B, dθy is a camera visual axis change.
In FIG. 5A, θx is a deviation angle from the camera visual axis, Cx is a horizontal image sensor surface size, F is a focal length, Px is a deviation pixel from the camera visual axis, and Pix is a horizontal direction in (b). The image size.
In FIG. 6A, θy is a deviation angle from the camera viewing axis, Cy is a vertical image sensor surface size, Py is a displacement pixel from the camera viewing axis, and Piy is a vertical image size.

As shown in FIG. 3A, when the positional relationship of the detection object candidate region 32 in the captured image is at a position shifted by Xr, Xl, Yt, Yb pixels with respect to the camera visual axis 31, FIG. As shown in FIG. 6, the shift pixels Px (FIG. 5B) and Py (FIG. 6B) from the camera visual axis 31 on the image and the shift angle θx from the camera visual axis 31 in the space (FIG. a)) and θy (FIG. 6A) are determined by the known camera focal length F and imaging element surface size C.
θl = 2 tan ⁻¹ (Cx / 2 · F) / Pix · xl
θr = 2 tan ⁻¹ (Cx / 2 · F) / Pix · xr
θt = 2tan ⁻¹ (Cy / 2 · F) / Pyy · yt
θb = 2 tan ⁻¹ (Cy / 2 · F) / Pyy · yb (1)
It becomes. Note that xl is the leftmost position of the detection target object candidate area 32, xr is the rightmost position, yt is the uppermost position, and yb is the lowermost position.
Therefore, the camera installation height hs of the host vehicle 41 and the minimum value Ymin of the vertical position from the lowest position yb of the detection object candidate area 32 to the detection object candidate area 32, as well as the camera installation height hs and the detection object. The maximum value Ymax of the vertical position from the highest position yt of the candidate area 32 and the maximum height of the detection object motion model to the detection object candidate area 32 can be calculated geometrically. Further, the minimum value Ymin of the vertical position up to the detection object candidate area 32, the maximum value Ymax of the vertical position up to the detection object candidate area 32, and the rightmost position xr and the leftmost position xl of the detection object candidate area 32 The maximum value Xrmin and maximum value Xrmax of the right position of the detection target object candidate region 32 and the maximum value Xlmin and maximum value Xlmax of the left position of the detection target candidate region 32 can also be calculated geometrically. That is, the range of spatial positions where the detection target object candidate area 32 can exist can be calculated from the detection target object candidate area 32.

As shown in FIG. 4A, the positional relationship between the host vehicle 41 and the detection object 42 at time T0 is Yt0 in the vertical direction, Xt0 in the horizontal direction, and the positional relationship between the host vehicle 41 and the detection object 42 at time T1. Is Yt1 in the vertical direction and Xt1 in the horizontal direction, the direction θx0 of the detection object 42 viewed from the own vehicle 41 with reference to the camera visual axis at time T0 and the own vehicle with respect to the camera visual axis at time T1. The direction θx1 of the detection object 42 viewed from 41 is:
θx0 = tan ⁻¹ (Xt0 / Yt0) (2)
θx1 = tan ⁻¹ (Xt1 / Yt1) + dθx (3)
It becomes. Here, dθx is a change angle of the camera visual axis in the horizontal direction at time T0 and time T1.
Also, the change in viewing angle dθx01 between time T0 and time T1 is
dθx01 = θx1−θx0 = tan ⁻¹ (Xt1 / Yt1) + dθx−tan ⁻¹ (Xt0 / Yt0) (4)
It becomes.
4B, the camera installation height of the host vehicle 41 is hs, the height of the detection target region 32 is ht, and the positional relationship between the host vehicle 41 and the detection target region 32 is at time T0. When the positional relationship between the host vehicle 41 and the detection target region 32 is Yt1 in the vertical direction and Yt1 in the vertical direction, the detection target viewed from the host vehicle 41 with respect to the camera visual axis at the time T0. The direction θy0 of the region 32 and the direction θy1 of the detection target region 32 viewed from the host vehicle 41 with reference to the camera visual axis at time T1 is
θy0 = tan ⁻¹ ((hs−ht) / Yt0) (5)
θy1 = tan ⁻¹ ((hs−ht) / Yt1) + dθy (6)
It becomes. Here, dθy is a change angle of the camera visual axis in the height direction at time T0 and time T1. Also, the change in viewing angle dθy01 between time T0 and time T1 is
dθy01 = θy1−θy0 = tan ⁻¹ ((hs−ht) / Yt1) + dθy−tan ⁻¹ ((hs−ht) / Yt0) (7)
It becomes.
Here, looking at equation (4), Xt0 and Yt0 can be calculated from the detection object region 32 as shown in FIG. 3, and dθx can be calculated from the vehicle behavior. In addition, Xt1 and Yt1 can be calculated based on Xt0 and Yt0 from the maximum momentum of the detected object motion model in a range that can move between times T1 and T0. From the above, using the information at time T0, the maximum change amount dθx01max of the viewing angle up to time T1 can be calculated, and by using the viewing angle-pixel conversion formula of FIG.
The relationship between the deviation pixel from the visual axis and the deviation angle from the visual axis:
θx = 2 tan ⁻¹ (Cx / 2 · F) / Pix · Px (8)
Similarly, looking at equation (7), the camera installation height hs is a known value, Yt0 can be calculated from the detection object region 32 as shown in FIG. 3, and dθy can be calculated from the vehicle behavior. The height ht of the detection object region 32 can specify the range that can be taken from the detection object motion model, and Yt1 is the range that can move between the time T1 and T0 from the maximum momentum of the detection object motion model based on Yt0. It can be calculated. From the above, using the information at time T0, the maximum change amount dθy01max of the viewing angle until time T1 can be calculated, and the maximum pixel movement amount in the image can be obtained by using the viewing angle-pixel conversion equation of FIG.
The relationship between the deviation pixel from the visual axis and the deviation angle from the visual axis:
θy = 2tan ⁻¹ (Cy / 2 · F) / Pyy · Py (9)
From the above, since the maximum pixel movement amount between the imaging frames of the camera can be determined from the detection object candidate region 32, the own vehicle behavior, and the detection object motion model, that is, how many frames it takes to move one pixel does not blur the image. The number of times (integration conditions) that can be integrated can be calculated.

FIG. 7 shows the basic principle of performing integration processing in the integration chip 6 in this embodiment. 7A and 7B show the configuration of the offset filter that calculates the background luminance value, FIG. 7C shows the processing of the offset filter, and FIG. 7D shows the result of the integration processing. As shown in FIGS. 7A and 7B, the offset filters are prepared separately on the left and right sides of the detection object candidate region. The offset filter is composed of one pixel in the vertical direction and a predetermined background size in the horizontal direction. The average value of the background area 23 is the background luminance value, and the value obtained by subtracting the background luminance value from the luminance value of the central pixel 21 is the integrated luminance value. Output. The integrated luminance value and the background luminance value of the detection object candidate region are calculated by processing the filter while shifting the pixel by pixel. As shown in FIG. 7C, when the luminance value of the detection target object region is Lt and the background luminance value is Lb, when the offset filter processing is performed at the filter position 70 in the figure, the background luminance value is Lb and the integrated luminance value is When the offset filter processing is performed at the filter position 71, the background luminance value is Lb and the integrated luminance value is (Lt−Lb). In the integration process, the integrated luminance value of the image signal input from the camera 1 is accumulated by the number of integrations input from the microcomputer 5, and the average value of the background luminance values in all frame images used for accumulation is added. That is, if the background luminance of coordinates x, y is Imgb (x, y), the integral luminance is Imgd (x, y), and the number of integrations is n, the integral image is
ΣImgd (x, y) + ΣImgb (x, y) / n (10)
Thus, the luminance value of the background luminance Lb area as shown in FIG. 7D is not improved, and only the area of the detection object candidate luminance Lt having the luminance value is integrated. Images can be calculated.

FIG. 8 shows a processing flow for detecting a detection object having a small luminance difference from the background according to the basic principle described above.
S100 is a spatial filter process in which the image processing chip 2 extracts detection target candidates from the image signal output from the camera 1.
S101 is a binarization process that binarizes the spatial filter image calculated in S100 with a predetermined threshold.
S102 is a clustering process for labeling adjacent binarized pixels of the binarized image calculated in S101 as the same object.
S103 is a process of calculating the vertical and horizontal sizes of the areas clustered in S102 and extracting the detection target object candidate areas.
S104 is a process of reading the image of the detection target object candidate area extracted in S103.
In step S105, edge detection processing is performed on the image read in step S104 using a Sobel filter.
S106 is a process for calculating the minimum value of the edge intensity calculated in S105 and calculating the number of times of luminance improvement integration for emphasizing up to a predetermined luminance difference.
S107 is a process of reading a signal from the vehicle sensor 4.
S108 is a process of calculating the spatial position of the detection object candidate from the detection object candidate area extracted in S103.
S109 is the integration of the detection target candidate from the detection target motion model based on the detection target candidate calculated in S108, the signal read out by the vehicle sensor 4 read out in S107, and the spatial position of the detection target candidate. This is a process for calculating the upper limit (integration condition) of the number of times.
S110 is a process of calculating (determining) an integration parameter for image integration from the luminance improvement integration count calculated in S106 and the integration count upper limit value calculated in S109.
S111 is processing for calculating a background luminance value from the image read in S104.
S112 is processing for subtracting the background luminance value calculated in S111 in the integration chip 6 and calculating an offset-removed image.
S113 is a process of integrating the offset-removed image calculated in S112 in the integration chip 6 with the integration parameter calculated in S110, calculating an integrated image, and storing the result in the processed image memory 7.
S114 is a process of adding the background luminance value calculated in S111 to the integrated image calculated in S113 in the integration chip 6 to calculate a processed image.
S <b> 115 is processing for reading out the integrated image calculated by the integrating chip 6 and stored in the processed image memory 7 from the processed image memory 7.
S <b> 116 is a process of reading the image pattern of the detection target registered in the microcomputer 5.
S117 is a process of calculating the degree of correlation between the processed image (integrated image) read in S115 and the image pattern of the detection target read in S116.
S118 is processing for determining whether or not the detection target candidate is a detection target based on the correlation degree calculated in S117. When the degree of correlation is higher than a predetermined value, it is determined that the object is a detection target, and the process of S119 is performed.
S119 is a process of displaying on the monitor 8 an image indicating the spatial position of the detection object determined to be the detection object in S118.

As described above, the image detection apparatus according to the present embodiment is arranged at a predetermined position of the host vehicle, and detects the detection target candidate from the camera 1 that captures the vicinity of the host vehicle and the image captured by the camera 1. The image processing chip 2 for extracting the vehicle, the vehicle sensor 4 for measuring the behavior of the own vehicle, the detection object candidate extracted by the image processing chip 2, the behavior of the own vehicle measured by the vehicle sensor 4, and the detection target A microcomputer 5 that calculates integration conditions from an object motion model, an integration chip 6 that performs image integration under the integration conditions calculated by the microcomputer 5, and an object to be detected using the integration image calculated by the integration chip 6 (The number of microcomputers 5 is usually one). The camera 1 in FIG. 1 is the imaging means in the claims, the image processing chip 2 and S103 in FIG. 8 are the detection object candidate extraction means, the vehicle sensors 4 and S107 are the vehicle behavior measuring means, and the microcomputer. 5 and S109 correspond to integration condition calculation means, the integration chips 6 and S113 correspond to image integration means, and the microcomputers 5 and S118 correspond to detection object detection means.
According to such a configuration, the maximum pixel motion amount on the imaging screen of the extracted detection object candidate can be understood by considering the behavior of the host vehicle and the movement of the detection object, so that the image can be integrated many times. Can be calculated. That is, even if the host vehicle is moving or the detection target object is moving, there can be calculated an integral image having no image blur and having a large luminance difference.
In addition, the image detection apparatus according to the present embodiment is arranged at a predetermined position of the host vehicle, the camera 1 that captures the vicinity of the host vehicle, and an image that extracts detection target candidates from the image captured by the camera 1. A processing chip 2, a microcomputer 5 for calculating a luminance improvement integration number (luminance difference improvement value) from the difference between the luminance value of the detection target candidate extracted by the image processing chip 2 and the luminance value of the background; The microcomputer for calculating the integration condition from the vehicle sensor 4 for measuring the behavior, the detection object candidate extracted by the image processing chip 2, the behavior of the own vehicle measured by the vehicle sensor 4, and the detection object motion model. 5 and the microphone for determining the number of integrations based on the number of luminance improvement integrations calculated by the microcomputer 5 and the integration condition calculated by the microcomputer 5 A computer 5, an integration chip 6 that performs image integration with the number of integrations calculated by the microcomputer 5, and the microcomputer 5 that detects an object to be detected using the integration image calculated by the integration chip 6 (note that The microcomputer 5 is usually one). The microcomputers 5 and S106 correspond to the luminance improvement integration number calculation means, and the microcomputers 5 and S110 correspond to the integration parameter determination means.
According to such a configuration, the number of luminance improvement integrations for luminance improvement calculated from the luminance value of the detection target candidate and the luminance value of the background in the vicinity thereof, the integration considering the own vehicle behavior and the movement of the detection target Since the number of times of image integration is determined from the conditions, it is possible to calculate an integrated image that has no blur and has an optimum luminance difference for image processing.
Furthermore, the image detection apparatus according to the present embodiment uses the integration chip 6 to calculate a luminance difference obtained by subtracting the background luminance from the luminance of the detection object using the offset filter, and performs integration processing only for the luminance difference.
According to such a configuration, only the luminance difference obtained by subtracting the luminance value of the background luminance level from the luminance value of the detection target candidate is integrated, and then the background luminance is added. Therefore, the background luminance is not changed by the integration. That is, it is possible to calculate an integral image in which luminance values having no luminance difference are continuous between the integration region and the non-integration region (or different integration frequency region).
The embodiment described above is described in order to facilitate understanding of the present invention, and is not described in order to limit the present invention. Therefore, each element disclosed in the above embodiment includes all design changes and equivalents belonging to the technical scope of the present invention.

It is a block diagram which shows the structure of the image detection apparatus of embodiment of this invention. It is a figure which shows the basic principle of the image detection apparatus of embodiment of this invention. It is a figure which shows the basic principle of the image detection apparatus of embodiment of this invention. It is a figure which shows the basic principle of the image detection apparatus of embodiment of this invention. It is a figure which shows the basic principle of the image detection apparatus of embodiment of this invention. It is a figure which shows the basic principle of the image detection apparatus of embodiment of this invention. It is a figure which shows the basic principle of the image detection apparatus of embodiment of this invention. It is a flowchart which shows the process of the image detection apparatus of embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Camera 2 ... Image processing chip 3 ... Detection image memory 4 ... Vehicle sensor 5 ... Microcomputer 6 ... Integration processing chip 7 ... Processing image memory 8 ... Monitor

Claims (3)

An imaging means that is arranged at a predetermined position of the host vehicle and images the vicinity of the host vehicle;
A detection target candidate extraction unit that extracts a detection target candidate from an image captured by the imaging unit;
Own vehicle behavior measuring means for measuring the behavior of the own vehicle;
Integration condition calculation means for calculating an integration condition from the detection object candidate extracted by the detection object candidate extraction means, the behavior of the own vehicle measured by the own vehicle behavior measurement means, and a detection object motion model When,
Image integration means for performing image integration under the integration condition calculated by the integration condition calculation means;
An image detection apparatus comprising: a detection target detection unit that detects a detection target using the integrated image calculated by the image integration unit. An imaging means that is arranged at a predetermined position of the host vehicle and images the vicinity of the host vehicle;
A detection target candidate extraction unit that extracts a detection target candidate from an image captured by the imaging unit;
A luminance improvement integration number calculating means for calculating a luminance improvement integration number from a difference between a luminance value of the detection target candidate extracted by the detection target candidate extraction means and a background luminance value;
Own vehicle behavior measuring means for measuring the behavior of the own vehicle;
Integration condition calculation means for calculating an integration condition from the detection object candidate extracted by the detection object candidate extraction means, the behavior of the own vehicle measured by the own vehicle behavior measurement means, and a detection object motion model When,
An integration parameter determination unit that determines the number of integrations from the integration condition calculated by the integration condition calculation unit, and the luminance improvement integration number calculated by the luminance improvement integration number calculation unit;
Image integration means for performing image integration at the number of integrations calculated by the integration parameter determination means;
An image detection apparatus comprising: a detection target detection unit that detects a detection target using the integrated image calculated by the image integration unit. The image detection device includes:
An image detection apparatus characterized in that, in the image integration means, a luminance difference obtained by subtracting background luminance from luminance of the detection object is calculated by an offset filter, and integration processing is performed only for the luminance difference.

Images