JP4451161B2 - Obstacle recognition assist device - Google Patents

Description

  The present invention relates to an image recognition technique, and more particularly to a technique for appropriately capturing an image of a moving object within a screen.

A technique related to the present invention is disclosed in, for example, Patent Document 1. FIG. 4 shows the configuration of the apparatus according to this prior art. This device is a device that detects the distance between the vehicle and the preceding vehicle that is traveling ahead of the host vehicle.
In FIG. 4, lenses 1001 and 1002 are a pair of lenses that are separated from each other by a base line length L and are symmetrically arranged on the surface of each of image sensors 1003 and 1004 that are positioned rearward by a focal length f. Then, an image of the preceding vehicle 1014 positioned forward by the distance R is formed. Image signals that are output from each of the image sensors 1003 and 1004 and that are analog signals representing an image of the field of view including the preceding vehicle 1014 are digital data by the A / D converters 1005 and 1006, respectively. Converted to data.

  The memories 1007, 1008, and 1009 are memories that store the visual field image data output from the A / D converters 1005 and 1006, respectively. However, in the memory 1008, when new image data is stored in the memory 1007, the image data previously stored in the memory 1007, that is, the image data before one imaging timing is transferred from the memory 1007 and stored. The

  The tracking calculator 1010 performs a tracking calculation of the preceding vehicle 1014 based on the image data stored in the memory 1007 and the memory 1008. The distance calculator 1011 calculates the distance of the preceding vehicle 14 based on the image signals stored in the memory 1007 and the memory 1009. The CPU 12 calculates the position of the window including the preceding vehicle 1014 that is set for the image of the field of view represented by the image data stored in each of the memories 1007, 1008, and 1009, and calculates the position. The window information shown is output to the tracking calculator 1010 and the distance calculator 1011, and the image of the field of view set in the window is displayed on the display 1013.

The operation of the tracking calculator 1010 will be further described.
The tracking computing unit 1010 is image data of an image of a field of view in which a window position related to the preceding vehicle 1014 is set after being imaged one imaging timing before (time t = t ₀ : hereinafter referred to as “previous time”). (Hereinafter referred to as “previous image data”) is read from the memory 1008. Further, the tracking computing unit 1010 reads the field image data (hereinafter referred to as “current image data”) captured at the current time (t = t ₀ + Δt) and stored in the memory 7. Then, in the image of the field of view represented by the previous image data (hereinafter referred to as “previous image”), the same image as the preceding vehicle 1014 that is the tracking target for which the window is set is represented by the current image data. A search is performed from the image of the field of view shown (hereinafter referred to as “current image”). This search method uses a known pattern matching algorithm.

  At this time, the search range of the image of the preceding vehicle 1014 in the current image is set with reference to the position of the window set in the previous image. This is because the elapsed time Δt from the previous time to the current time is very small, so that the image of the preceding vehicle 1014 can be regarded as hardly moving from the position on the screen of the image of the field of view in which the window is set. .

  When the tracking computing unit 1010 searches the current image for the same image as the preceding vehicle 1014, the CPU 1012 calculates the position of a new window to be set for the current image and sets it to the current image (window setting means 1120). The position of this window is set at the address of the memory 1007 corresponding to the pixel arrangement of the image sensor 1003 on which the image of the preceding vehicle 1014 is formed. When the preceding vehicle 1014 is searched from the current image represented by the current image data stored in the memory 1007, the image data of the current image including the image of the preceding vehicle 1014 is transferred from the memory 1007 to the memory 1008. The previous image data stored in the memory 1008 is updated.

  On the other hand, the CPU 1012 reads out the previous image data from the memory 1008, displays the image of the visual field on the display 1013, generates a window image signal according to the window position set by itself, and further displays the image of the preceding vehicle 1014 in the image of the visual field of the window. Are displayed in a superimposed manner.

By performing the above operation, the image of the preceding vehicle 1014 can be captured and tracked in the window within the field-of-view image.
Next, the distance calculation operation by the apparatus shown in FIG. 4 disclosed in Patent Document 1 will also be described.

  The distance calculator 1011 shifts one pixel of the image of the preceding vehicle 1014 in the image of the field of view represented by the current image data stored in the memories 1007 and 1009 one pixel at a time. Find the sum of absolute values of differences. Here, the time when the sum becomes the smallest is regarded as the time when the image signals output from the image sensors 1003 and 1004 are the best match, and the pixel shift number n at that time and the base line between the lenses 1001 and 1002 When the length L, the focal length f of the lenses 1001 and 1002, and the pitch width p between the pixels forming the image sensors 1003 and 1004 are substituted into the following equation (1), the preceding vehicle is moved from the lenses 1001 and 1002 according to the principle of triangulation. The distance up to 1014, that is, the inter-vehicle distance R is obtained.

R = (f × L) / (n × p) (1)
Note that such an imaging apparatus that can obtain the distance of an object represented in a captured image by using a pair of imaging elements is called a stereo imaging apparatus or the like, and is widely known. Yes.
JP-A-6-265351

  It is necessary to pay close attention to an object present in the field of view of the traveling vehicle, particularly if there is no danger to the traveling vehicle. In particular, it is desirable to check frequently whether or not the object has a possibility of colliding with the host vehicle, and recognize the possibility as soon as possible. It will be so easy to take. However, in the apparatus according to the related art described above, the image of the preceding vehicle 1014 is regarded as a part of the image of the field of view, and therefore the field of view is shorter than the above-described Δt, which is the interval of the imaging timing of the image of the field of view. The behavior of the object inside cannot be monitored.

  The present invention has been made in view of the above-described problems, and a problem to be solved is to increase the frequency of monitoring the behavior of an object whose image is displayed on the screen.

  An obstacle recognition assisting apparatus that is one aspect of the present invention includes an object-of-interest specifying means for specifying an object of interest in which a unique behavior is recognized that is included in a screen to which an image acquired by an imaging unit belongs, and the screen The window is set so that all or part of the screen of the region of interest is located in a window having an area narrower than the entire region of the image, and once the window is set, all or part of the image of the object of interest Is faster than the window setting control means for controlling the position of the window in the entire area of the screen so as to be always located in the window, and the readout rate for reading out the image data corresponding to the entire area of the screen. Window image data reading means for reading image data corresponding to the window from the window at a reading rate; And an imaging device for full-screen data mainly used for reading out image data corresponding to the entire area of the screen, and window data used for reading out image data corresponding to the area of the window. And an image pickup device for use.

  Note that, in the obstacle recognition assisting apparatus according to the present invention described above, the object-of-interest specifying means described above is a predetermined distance in which the relative position with the object provided with the obstacle recognition assisting apparatus suddenly approaches or is relatively close. An object located within or having a probability of reaching these positional relationships may be specified as the above-mentioned object of interest.

  Further, in the obstacle recognition assisting apparatus according to the present invention described above, the window screen data reading means described above has a reading rate of image data corresponding to the entire area of the screen as the reading rate of the image data related to the window. You may comprise so that it may correspond to what multiplied the reciprocal of the area ratio of the window with respect to the said whole area | region.

  In the obstacle recognition assisting apparatus according to the present invention described above, the imaging unit described above is a stereo imaging apparatus, and the above-described attention object specifying unit depends on data of a distance image acquired by the stereo imaging apparatus. Then, the object of interest may be identified by recognizing the unique behavior of the object included in the screen.

  Further, in the obstacle recognition assisting apparatus according to the present invention described above, the above-described imaging means includes the above-described full-screen data imaging element and the above-described window data imaging by means of a light splitting means inserted in a common imaging optical system. You may comprise so that it may have an optical system comprised so that the image which concerns on an imaging field substantially equivalent with an element may be tied.

In the obstacle recognition assisting apparatus according to the present invention described above, the window data imaging device described above may be a MOS type imaging device.
The obstacle recognition assisting apparatus according to the present invention described above may be configured to further include behavior detection means for detecting the behavior of the object of interest based on the image data read from the window.

  Further, in the obstacle recognition assisting apparatus according to the present invention described above, the window setting control means described above is configured to be able to set a plurality of the windows described above in the entire area of the screen related to the window data imaging device. It is good also as what was done.

  Since this invention is comprised as mentioned above, there exists an effect that the frequency which monitors the behavior of the object by which the image is represented on the screen increases.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
First, FIG. 1 will be described. This figure shows the configuration of an apparatus for carrying out the present invention. The device shown in the figure is used by being mounted on a vehicle, and an object whose relative distance to the vehicle (host vehicle) on which the device is mounted becomes an obstacle that may collide with the host vehicle. It is assumed that the image of the object is specified in the image representing the scenery around the host vehicle and the image data representing the obstacle is output. Here, this device is referred to as an obstacle recognition assisting device.

The lenses 1 and 2 are a pair of lenses arranged symmetrically with a distance of a base line length L, respectively, and an image of an obstacle positioned at a distance R from the lenses 1 and 2 is formed behind the focal length f. Let
The half mirrors 3 and 4 transmit part of the light that has passed through the lenses 1 and 2 in the direction of the image sensors 3 and 4, while reflecting the rest of the light and passing the optical path in the directions of the image sensors 7 and 8. This is the light splitting means that is directed to

  The imaging elements 5 and 6 constitute a stereo imaging device. The light receiving surfaces of the image pickup devices 5 and 6 are disposed behind the lenses 1 and 2 by the focal length f, and therefore an image of an obstacle is formed on the light receiving surfaces. The image sensors 5 and 6 are images of landscapes around the host vehicle and including obstacle images under the control of the image sensor drive control unit 33 (hereinafter referred to as “landscape images”). The image signal representing this image is output. In the present embodiment, a MOS (Metal Oxide Semiconductor) type image sensor that is advantageous in terms of reduction of power consumption and part cost is used as the image sensor.

  The imaging elements 7 and 8 also constitute a stereo imaging device, and for the same reason as described above, a MOS type image sensor is used in the present embodiment. The light receiving surfaces of the image pickup devices 7 and 8 are bent by the half mirrors 3 and 4 from the lenses 1 and 2 to reach the light receiving surfaces of the image pickup devices 7 and 8, and the focal length f of the lenses 1 and 2 is exactly the same. Therefore, an obstacle image is formed on the light receiving surface. That is, the optical system of the apparatus shown in FIG. 1 has an image sensor 5 by half mirrors 3 and 4 inserted in the optical path from each of the lenses 1 and 2 to each of the image sensors 5, 6, 7, and 8. And 6 and the image sensors 7 and 8 are configured such that substantially the same image field of view is formed.

  The image sensors used as the image pickup devices 7 and 8 are those having a so-called WOI (Window of Interest) function, and the image pickup range is specified in the landscape image under the control of the image pickup device drive control unit 34. The window image limited to the range (window) is photoelectrically converted, and an image signal representing this image is output.

This WOI function will be further described with reference to FIG.
As shown in FIG. 2A, an image sensor having a WOI function captures an image having the maximum number of pixels that can be imaged (an image having a pixel number of 640 × 480 pixels in the example shown in FIG. 2A). While imaging at a predetermined imaging rate (for example, 30 frames / second) and outputting an image signal for the entire image, at the time of partial imaging by the WOI function, as shown in FIG. Window image in which the imaging range is limited to a specified range (window) among the images to be captured (in the example of the figure, 320 × 240 in which the number of pixels is 1/4 with respect to the entire area of the screen imaged at the time of the entire imaging. Only the image displayed on the pixel screen) has a higher imaging rate than the entire imaging (for example, when the imaging rate at the entire imaging is 30 frames / second, the reduction rate of the number of pixels Images can be captured at 120 frames / second (= 30 × 4) which is a reciprocal multiple, and an image signal for the window image can be output.

  In the present embodiment, the imaging ranges of the imaging devices 5 and 6 that are full-screen data imaging devices are the same as the imaging range during the entire imaging of the imaging devices 7 and 8 that are window data imaging devices, Assume that the imaging rates of the imaging devices 5 and 6 are equal to the imaging rates during the entire imaging of the imaging devices 7 and 8.

Returning to the description of FIG.
When A / D converters 11 and 12 receive image signals, which are analog signals output from the image pickup devices 5 and 6, respectively, the image signals are converted from analog to digital and image data representing a landscape image. Are output respectively.

When A / D converters 13 and 14 receive image signals, which are analog signals output from the image sensors 7 and 8, respectively, the image signals are converted from analog to digital, and image data representing a window image. Are output respectively.
Note that the time required for sampling and conversion of analog signals in the A / D converters 11, 12, 13, and 14 is sufficiently high so that other processing is not hindered.

The memories 15 and 16 temporarily store image data of landscape images output from the A / D converters 11 and 12, respectively, and the memories 17 and 18 are output from the A / D converters 13 and 14, respectively. Temporarily save the image data of each window image,
The distance calculation unit 19 reads the image data each time image data representing a landscape image captured by the image sensors 5 and 6 is newly stored in the memories 15 and 16, and is expressed by the image data. The relative distance between the object represented in each of the pixels constituting the scenery image and the host vehicle is obtained. This relative distance can be obtained by the distance calculation method by the distance calculator 1011 in the apparatus shown in FIG. 4 described above, that is, the calculation method according to the formula (1) based on the principle of triangulation described above. This is made possible by using the imaging elements 5 and 6 as a stereo imaging device.

  The CPU 20 is a central processing unit, and by executing a control program stored in advance in a ROM (Read Only Memory) (not shown), the distance image generating unit 21, the target object specifying unit 22, the window setting control unit 23, the window Each function of the image data reading unit 24 and the image processing unit 25 is realized. Note that each of these functions may be realized by separate hardware instead of being realized by a combination of the CPU 20 and the control program, and the relative distance calculation performed by the distance calculation unit 19 may be performed. You may make it implement | achieve with CPU20.

The distance image generation unit 21 is an image in which the information on the relative distance output from the distance calculation unit 19 is associated with each pixel constituting the landscape image represented by the image data stored in the memories 15 and 16. A certain distance image data is generated.
The object-of-interest specifying unit 22 specifies the position of the object represented in the landscape image in the landscape image based on the distance image data. Specifically, among the pixels in the landscape image, those in which the difference in relative distance associated with adjacent pixels is within a predetermined value are regarded as images of the same object, and the positions of these pixels The range in the landscape image is identified. In addition, here, the shape, size, color, position, etc., between each object represented in the current landscape image and each object represented in the landscape image acquired in the past Correlation is obtained based on this, and objects having the highest correlation above a predetermined level are associated with each other as the same object.

Further, the object-of-interest specifying unit 22 specifies an object (for example, an obstacle) in which a unique behavior is recognized among these objects shown in the landscape image as the object of attention.
In the present embodiment, the target object specifying unit 22 firstly compares the relative distance at the current time between each object represented in the landscape image and the host vehicle from the information on the relative distance for each pixel obtained from the distance calculation unit 19. The distance is calculated by, for example, the arithmetic average of the relative distance for each pixel. Here, an object whose relative distance is within a predetermined distance is an object close to the own vehicle, and can be regarded as having a probability of colliding with the own vehicle, and thus is an object of interest.

  The object-of-interest specifying unit 22 in the present embodiment further includes an output signal V from a vehicle speed detection unit (not shown) that is provided in association with the speedometer of the vehicle and outputs a signal representing the traveling speed of the vehicle. , A steering wheel angle detection means θ (not shown) that is provided in relation to the steering wheel of the host vehicle and outputs a signal indicating the steering angle of the steering wheel is supplied. The object-of-interest specifying unit 22 estimates the behavior of the host vehicle based on these signals V, θ, etc., and can determine the probability of the collision and the pattern of approaching another vehicle or object with practically sufficient accuracy. Has been made.

  In this embodiment, the target object specifying unit 22 stores the relative distance in the past (for example, the previous time) between each object and the host vehicle, and the target object specifying unit 22 stores the relative distance in the past. The approach speed between each object and the host vehicle is calculated from the relative distance at the current time. Here, an object whose approach speed is equal to or higher than a predetermined speed is an object that approaches the host vehicle suddenly, and such an object can be regarded as having a probability of colliding with the host vehicle. .

Furthermore, in the present embodiment, the object-of-interest specifying unit 22 determines that the relative distance between the subject vehicle and the relative distance between the subject vehicle and the subject vehicle is within the predetermined distance described above. It is determined whether or not there is a probability of approaching, and an object having this probability is also set as a target object.
The window setting control unit 23 sets a window so that the image of the target object specified by the target object specifying unit 22 is positioned with respect to the background image at the current time. If the window has already been set, the entire area of the landscape image is based on the association information between the past background image given to each object by the target object specifying unit 22 and the background image at the current time. Control is performed so as to follow the position of the window in the window so that the image of the object of interest is always positioned in the window. Note that the shape and size of the window are set in advance. Further, even when the entire image of the target object cannot be located in the window due to the difference in shape and size between the target object image and the window, at least a part of the target object image is positioned in the window. To control the window position.

  The window setting control unit 23 in the present embodiment further takes into account the behavior of the above-mentioned object of interest, the relative movement of the object with the host vehicle, the size of the object, ease of visual recognition when displayed, and the like. And a necessary frame rate calculation function unit for calculating an appropriate frame rate related to the imaging of the object of interest, and control information for imaging at an appropriate frame rate as required for the above-described WOI. The unit 34 can be supplied.

  The window image data reading unit 24 stores the image data each time image data representing the window image captured by the partial imaging by the above-described WOI function by the imaging elements 7 and 8 is stored in the memories 17 and 18. 17 and 18, and the read image data is sent to the image processing unit 25 and output to the outside.

  When a window is set for the landscape image, the image processing unit 25 reads the image data of the landscape image read from the memories 17 and 18, the image data about the window image sent from the window image data read unit 24, and Based on the information indicating the position of the window in the landscape image obtained from the window setting control unit 23, processing is performed to generate image data for a combined image obtained by fitting the window image into the position of the window in the landscape image. If necessary, the behavior of the target object shown in the window image (for example, the relative moving direction and speed of the target object with reference to the host vehicle) is detected from the composite image history, and the behavior is detected. A process of generating image data for an image in which information indicating the state of the above is superimposed and displayed on the composite image is performed.

The D / A conversion unit 31 performs digital-analog conversion on the image data generated by the image processing unit 25 and outputs an image signal representing the above-described composite image.
The display device 32 is an LCD (Liquid Crystal Display), for example, and displays an image represented by the image signal output from the D / A conversion unit 31.

The image sensor drive control unit 33 controls the operation of the image sensors 5 and 6 and causes the image sensors 5 and 6 to capture a landscape image at a predetermined image rate.
The image sensor drive control unit 34 controls the operation of the image sensors 7 and 8, and in particular, a signal sent from the object-of-interest specifying unit 22 and indicating whether or not the object of interest has been identified from the landscape image. Based on the information indicating the position of the window in the landscape image sent from the window setting control unit 23, partial imaging by the WOI function at an imaging rate faster than the imaging rate of the landscape image by the imaging elements 5 and 6 is performed. The imaging elements 7 and 8 are made to change the setting of the imaging range of the window image at the time of partial imaging.

The drive control of the image sensors 5, 6, 7, and 8 performed by the image sensor drive control units 33 and 34 will be further described with reference to FIG.
In the following description, as shown in FIG. 2A, the image sensors 5 and 6 capture a landscape image with 640 × 480 pixels, and the image sensors 7 and 8 have the WOI function. As shown in FIG. 2B, in the partial imaging by, a window image having a window size of ¼ of the number of imaging pixels at the time of overall imaging, that is, 320 × 240 pixels is captured.

  FIG. 3A shows an example of a change in a signal that is generated inside the image sensor drive control unit 33 and that indicates an instruction to start capturing a landscape image by the image sensors 5 and 6, and FIG. 4 shows a change example of a signal that is generated inside the image sensor drive control unit 34 and that indicates an instruction to start capturing a window image by the image sensors 7 and 8. Each of these signals is a pulse signal, and the rising edge in each pulse represents an instruction to start imaging. Therefore, the period (second) of the rising edge is the image capturing interval, and the reciprocal of this period is the image capturing rate (frame / second).

  FIG. 3B shows a target object specifying signal sent from the target object specifying unit 22 to the image sensor drive control unit 34. This signal takes a binary value of H (high) level and L (low) level, and indicates whether or not the object of interest has been identified from the landscape image. When it is at the H level, it indicates that the object of interest has been identified. The L level indicates that the object of interest is not specified.

  In FIG. 3, the landscape image capturing instruction signal shown in (a) is generated with a pulse regardless of the target object specifying signal shown in (b). The period is always maintained at T. On the other hand, the window image capturing instruction signal shown in (c) has no pulse generated when the target object specifying signal in (b) is at L level, that is, when the target object is not specified. On the other hand, when the target object specifying signal in (b) is at the H level, that is, when the target object is specified, the period of the rising edge is T / 4 and a pulse is generated.

The periods [A] and [B] shown in FIG. 3 are periods in which the target object specifying signal in FIG. 3B is at the L level, that is, a period in which the target object is not specified.
In this period, the image sensor drive control unit 33 sequentially performs the landscape image capturing operation by the image sensors 5 and 6 at the period T according to the timing of the rising edge of the pulse signal in FIG. On the other hand, during this period, the element drive control unit 34 stops the window image capturing operation by the image sensors 7 and 8. Accordingly, during this period, the window image data is not written into the memories 17 and 18, so the window image data is not read out by the reading unit 24. At this time, the image processing unit 25 does not perform the synthesis process. Accordingly, the landscape image is displayed on the display device 32 as it is.

The period [C] shown in FIG. 3 specifies the target object from the landscape image captured in the period when the target object specification signal in FIG. 3B is at the H level, that is, the period [B] immediately before. This is a period after the attention object specifying unit 22 has performed.
Even during this period, the image sensor drive control unit 33 sequentially performs the landscape image capturing operation by the image sensors 5 and 6 at the period T according to the timing of the rising edge of the pulse signal in FIG. Further, during this period, the image sensor drive control unit 34 sequentially performs the window image imaging operation by the image sensors 7 and 8 at a cycle T / 4 according to the timing of the rising edge of the pulse signal in FIG. Make it. At this time, the image sensor drive control unit 34 receives information indicating the setting position of the window in the landscape image from the window setting control unit 23, and captures the range of the window set in the landscape image, that is, the object of interest. Are picked up by the image pickup devices 7 and 8 to obtain window images.

  Here, when attention is paid to the imaging time of one frame of the landscape image and the window image, the imaging elements 7 and 8 capture the image because the imaging interval is ¼ times that when capturing the landscape image when the window image is captured. It is necessary to capture the window image at an imaging rate that is at least four times that when capturing landscape images by the elements 5 and 6. However, as described above, the number of pixels for capturing a window image is 320 × 240 pixels, which is ¼ times the number of pixels for capturing landscape images, which is 640 × 480 pixels. On the other hand, it is possible to cause the image pickup devices 7 and 8 to pick up a window image at a four times higher image pickup rate.

In this way, the window image is picked up by setting the window image imaging rate by multiplying the landscape image imaging rate by the reciprocal of the ratio of the number of pixels of the window image to the landscape image (that is, the area ratio). Can be imaged.
In this way, the landscape image captured by the image sensors 5 and 6 and the window image captured by the image sensors 7 and 8 are converted into images by passing through the A / D converters 11, 12, 13, and 14, respectively. It is converted into data and stored in the memories 15, 16, 17 and 18.

  The image data of the landscape image stored in the memories 15 and 16 is read by the distance calculation unit 19 and the distance image generation unit 21 and the image processing unit 25 in the CPU 20 every time the image data is updated. On the other hand, the image data of the window image stored in the memories 17 and 18 is read by the window image data reading unit 24 in the CPU 20 every time the image data is updated.

  Here, as is clear from FIG. 3, since the window image is captured at the imaging rate four times that of the landscape image, the update interval of the image data of the window image in the memories 17 and 18 is also as follows. This is four times the update interval of the landscape image data in the memories 15 and 16. Accordingly, the window image data reading unit 24 reads the image data of the window image at a reading rate (four times) faster than the reading rate of the landscape image image data read by the distance calculation unit 19 or the like.

  Note that when the image data of the landscape image stored in the memories 15 and 16 is updated, the image processing unit 25 outputs the image data of the landscape image without performing image synthesis while reading the image data of the landscape image. When the image data of the window image stored in FIG. 18 is updated, the image data of the window image sent from the window image data reading unit 24 and the latest data are read from the memories 15 and 16 each time it is updated. A composition process with image data of a landscape image is performed.

As a result, since the image of the object of interest for four frames is displayed on the display device 32 within the display period of one frame of the landscape image, the frequency of monitoring the behavior of the object of interest increases.
In addition, the present invention is not limited to the above-described embodiments, and various improvements and changes can be made without departing from the scope of the present invention.

  For example, if a plurality of imaging ranges can be set as imaging ranges in partial imaging by the WOI function of the imaging elements 17 and 18, the target object specifying unit 22 specifies a plurality of target objects from the landscape image, and window setting control The unit 23 sets a plurality of windows corresponding to individual objects of interest, notifies the image sensor drive control unit 34 of the positions of the windows in the landscape image, and the image sensor drive control unit 34 indicates these positions in the imaging range. As described above, by causing the imaging devices 17 and 18 to capture a plurality of window images, the behavior of a plurality of objects of interest can be monitored. Even when only one imaging range can be set for partial imaging using the WOI function of the imaging elements 17 and 18, the imaging element drive control unit 34 causes the imaging elements 17 and 18 to capture a window image. By switching the position setting of the imaging range based on the notification of the positions of the plurality of windows corresponding to the individual objects of interest, the behavior of the plurality of objects of interest can be monitored.

  Further, in specifying the target object by the target object specifying unit 22, it is determined whether or not the object represented in the landscape image is close to the host vehicle, or whether there is a probability of approaching within a predetermined distance. In this case, for example, information on the behavior of the host vehicle obtained from the host vehicle such as the current speed of the host vehicle and the moving direction (handle angle) may be taken into account. Can be improved.

It is a figure which shows the structure of the apparatus which implements this invention. It is a figure explaining the WOI function of an image sensor. It is a figure explaining drive control of an image sensor. It is a figure which shows the structure of the apparatus which concerns on a prior art.

Explanation of symbols

1, 2, 1001, 1002 Lens 3, 4 Half mirror 5, 6, 7, 8 Image sensor 11, 12, 13, 14, 1005, 1006 A / D converter 15, 16, 17, 18, 1007, 1008, 1009 Memory 19, 1011 Distance calculation unit 20, 1012 CPU
DESCRIPTION OF SYMBOLS 21 Distance image generation part 22 Attention object specific | specification part 23 Window setting control part 24 Window image data reading part 25 Image processing part 31 D / A conversion part 32 Display apparatus 33, 34 Image sensor drive control part 1003, 1004 Image sensor 1010 Tracking calculation Vessel 1013 Display 1014 Advancing vehicle 1120 Window setting means

Claims (8)

An object-of-interest specifying means for specifying an object of interest in which a unique behavior is included in the screen to which the image acquired by the imaging means belongs;
The window is set so that all or part of the screen of the region of interest is located in a window having an area narrower than the entire region of the screen, and once the window is set, all of the image of the object of interest or Window setting control means for controlling the position of the window in the whole area of the screen so that a part is always located in the window;
Window image data reading means for reading image data corresponding to the window from the window at a reading rate faster than a reading rate for reading image data corresponding to the entire area of the screen;
With
And as the imaging means,
An image sensor for full-screen data used mainly for reading image data corresponding to the entire area of the screen;
An image sensor for window data used mainly for reading out image data corresponding to the area of the window;
An obstacle recognition assisting device characterized in that a device comprising the above is applied.   The object-of-interest specifying means is an object whose relative position with respect to the object provided with the obstacle recognition assisting device is abruptly approaching or is located within a predetermined distance that is relatively close or has a probability of reaching these positional relationships. The obstacle recognition assisting apparatus according to claim 1, wherein the obstacle recognition assisting apparatus is configured to identify a target object.   The window screen data reading means corresponds to a read rate of image data relating to the window, which is obtained by multiplying a read rate of image data corresponding to the entire area of the screen by an inverse of the area ratio of the window with respect to the entire area. The obstacle recognition assisting apparatus according to claim 1, wherein: The imaging means is a stereo imaging device,
The target object specifying unit is configured to identify a target object by recognizing a specific behavior of an object included in the screen based on distance image data acquired by the stereo imaging device. The obstacle recognition assisting apparatus according to claim 1, wherein:   The image pickup means is configured such that an image having substantially the same imaging field of view is formed with respect to the image pickup element for full screen data and the image pickup element for window data by a light dividing means inserted in a common image pickup optical system. The obstacle recognition assisting apparatus according to claim 1, wherein the obstacle recognition assisting apparatus has an optical system configured as described above.   The obstacle recognition assisting apparatus according to claim 1, wherein the window data imaging device is a MOS type imaging device.   The obstacle recognition assisting apparatus according to claim 1, further comprising behavior detection means for detecting the behavior of the object of interest based on image data read from the window.   2. The obstacle according to claim 1, wherein the window setting control unit is configured to be capable of setting a plurality of the windows in an entire area of a screen related to the window data imaging device. Recognition assist device.

Images