JP2004251886A - Device for detecting surrounding object - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device for detecting surrounding objects which can detect a surrounding object, even if information on images of a surrounding object in photographed images changes. <P>SOLUTION: Virtual regions 31-38 fixed in a real space around a vehicle 100 are set, and the relative positional relation between the vehicle 100 and each set virtual region 31-38 is computed, and regions which correspond to the virtual regions 31-38 are set on a photographed image of the neighborhood of the vehicle 100, on the basis of the computed positional relation. Characteristic quantities of the virtual regions 31-38 are computed by the quantities of changes in a predetermined time interval of the average luminance values of these set regions, and the object around the vehicle 100 is detected on the basis of these feature quantities. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

  The present invention relates to a surrounding object detection device that detects an object around a vehicle.

  2. Description of the Related Art A device that detects an obstacle on the side of a vehicle and notifies the driver of the obstacle is known. In this device, an image of the side of the own vehicle is photographed by a video camera whose visual axis is directed to the side of the vehicle, and feature points are extracted from each frame of the photographed image. The optical flow in the image on the side of the own vehicle is calculated from the amount of movement of the vehicle. Further, a movement flow length corresponding to the moving amount of the image due to the movement of the own vehicle is obtained from the speed of the own vehicle. The optical flow thus obtained is compared with the moving flow length, and if the two are different, it is determined that a vehicle is present on the side of the own vehicle, and an alarm is issued (Patent Document 1).

JP-A-7-296297

  In this apparatus, it is necessary to determine the same feature point from feature points extracted for each frame of an image based on image information such as luminance distribution. However, in the captured image, image information such as the luminance distribution of a vehicle present on the side of the vehicle is based on a difference in a projection plane due to a change in the relative position between the vehicle and the vehicle, and shadows due to surrounding structures. Is not constant due to changes in Therefore, it is not possible to determine the same feature point between the frames of the image, and the optical flow cannot be calculated. As a result, the vehicle on the side may not be detected.

A surrounding object detection device according to the present invention sets a photographing means for photographing the periphery of a vehicle, and a plurality of virtual areas fixed to the real space on a real space around the vehicle, and photographs the virtual area by the photographing means. Detecting means for detecting an object around the vehicle based on image information.
Further, the surrounding object detection device according to the present invention is applied to a surrounding object detection device mounted on a vehicle and detecting an object around the vehicle, and a photographing means for photographing the periphery of the vehicle and a vehicle movement detecting the movement of the vehicle. Detecting means, moving amount calculating means for calculating the moving amount of the vehicle based on the motion of the vehicle detected by the vehicle motion detecting means, and a plurality of virtual areas fixed to the real space on the real space around the vehicle. A virtual area setting means for setting and setting a new virtual area in accordance with the movement amount; and calculating a relative positional relationship between the vehicle and the virtual area based on the movement amount of the vehicle calculated by the movement amount calculation means. A relative position calculating means, and a projection processing means for calculating a position and a shape in the image when the virtual area is projected on the image photographed by the photographing means, based on the positional relationship calculated by the relative position calculating means. Based on the position and shape calculated by the projection processing means, an image area setting means for setting an area corresponding to the virtual area on the image, and based on image information in the area set by the image area setting means. A feature amount calculating unit that calculates a feature amount representing a feature of the virtual region; and a surrounding object detecting unit that detects a surrounding object of the vehicle existing in the virtual region based on the feature amount calculated by the feature amount calculating unit. It is provided.

According to the present invention, the surroundings of a vehicle are photographed, a plurality of virtual areas fixed to the real space are set on the real space around the vehicle, and the surroundings of the vehicle are set based on information of an image obtained by photographing the virtual area. Objects were detected. As a result, the surrounding objects of the vehicle are detected based on the image information based on the virtual area fixed in the real space. Is changed, the object can be detected.
Further, according to the present invention, a virtual area fixed in a real space around the host vehicle is set, a relative positional relationship between the host vehicle and the virtual area is calculated, and the host vehicle is calculated based on the calculated positional relationship. A region corresponding to the virtual region is set on the image obtained by photographing the periphery of the vehicle, a feature amount of the virtual region is calculated based on image information of the region on the image, and an object around the own vehicle is calculated based on the feature amount. Detected. As a result, the surrounding objects of the own vehicle are detected based on the image information based on the virtual area fixed in the real space. Even if the information changes, the object can be detected.

--First Embodiment--
FIG. 1 shows a first embodiment of a surrounding object detection device according to the present invention. The surrounding object detection device 1 is mounted on a vehicle to photograph an image around the vehicle, and detects an object such as a surrounding vehicle based on a temporal change in the photographed image information. The surrounding object detection device 1 includes a camera 7, a vehicle sensor 8, a memory 9, a CPU 10, and an alarm device 11.

  The camera 7 is a camera that continuously captures images around the vehicle. As shown in FIG. 2, the visual axis of the camera 7 is directed to the rear side of the vehicle 100 on which the surrounding object detection device 1 is mounted. This allows the camera 7 to capture an image of a moving object at a position that is difficult for the driver to check, such as another vehicle traveling behind the adjacent lane. As the camera 7, for example, a CCD camera having a photographing sensitivity to visible light or infrared light is used, but another type may be used. Images continuously captured by the camera 7 are stored in the memory 9 after being converted into image frames divided at predetermined time intervals.

  The vehicle sensor 8 detects the rotation speed of each of the right and left wheels of the vehicle, and outputs a wheel speed pulse corresponding to the rotation speed to the CPU 10. This wheel speed pulse is output when the wheel rotates by a predetermined fraction of one rotation. For example, one wheel speed pulse is output every 1/88 rotation. The number of output wheel speed pulses is counted by the CPU 10 and stored in the memory 9. Based on the left and right wheel speed pulse numbers, the CPU 10 calculates the moving amount of the vehicle every predetermined time by dead reckoning. At this time, the amount of inclination of the axle is calculated based on the difference between the left and right wheel speed pulse numbers, and thereby the amount of change in the traveling direction of the vehicle is calculated.

  As described above, the memory 9 stores the image information from the camera 7 and the information on the count value of the wheel speed pulse output from the vehicle sensor 8, and as a processing result executed by the CPU 10, the movement amount of the vehicle, A virtual area map described later is stored. These various types of information are managed for each address, and any type of information can be read from or written to the memory 9 by specifying the corresponding address.

  The CPU 10 executes various calculations and processes based on the information stored in the memory 9. For example, a moving object existing around the vehicle is determined based on the amount of vehicle movement based on the dead reckoning information calculated based on the number of wheel speed pulses output from the vehicle sensor 8 and image information captured by the camera 7. To detect. At this time, if there is a danger of the moving object colliding, the alarm device 11 is operated, and the driver is notified by, for example, voice or alarm sound.

  A method for detecting a moving object around the vehicle will be described with reference to FIGS. The surrounding object detection device 1 is mounted on the vehicle 100 shown in FIG. 3A that is traveling on the road at the time t1. It is assumed that an image indicated by reference numeral 41 in FIG. At this time, regions indicated by reference numerals 21 to 26 are set on the lane next to the lane in which the vehicle 100 is traveling in the image 41.

  These regions 21 to 26 are set as regions displayed in the real space displayed on the image 41. That is, it is assumed that the regions 31 to 36 shown in FIG. 3A exist in the real space, and when the camera 7 photographs the rear side of the vehicle 100, the images corresponding to the virtual regions 31 to 36 respectively. The area on 41 corresponds to the areas 21 to 26.

  Next, it is assumed that the vehicle 100 moves to the position shown in FIG. 3B at time t2, and at this time, an image indicated by reference numeral 42 in FIG. Since the virtual areas 31 to 36 in the real space shown in FIG. 3B are farther from the vehicle 100 than in the case of FIG. 3A, the relative positional relationship between the vehicle 100 and the virtual areas 31 to 36 changes. I do. Therefore, the positions and shapes of the regions 21 to 26 set on the image 42 corresponding to the virtual regions 31 to 36 in the real space are different from those of the image 41.

  The positions and shapes of the regions 21 to 26 on the image 42 are determined by the relative positional relationship between the vehicle 100 and the virtual regions 31 to 36 in the real space. This relative positional relationship is calculated based on the left and right wheel speed pulses detected by the vehicle sensor 8 by calculating the amount of movement of the vehicle 100 by dead reckoning (referred to as relative position calculation). Based on the calculated relative positional relationship, the positions and shapes when the virtual regions 31 to 36 are projected onto the image 42 are calculated (referred to as projection processing), and the positions and shapes of the regions 21 to 26 are determined. In FIG. 3B, the virtual area 31 exists at a position farther than the visible distance of the camera 7 indicated by the broken line 30. Therefore, the area 21 corresponding to the virtual area 31 is not displayed in the image 42 as shown in FIG.

  In addition, at this time, it is assumed that the vehicle 100 has moved from the position in FIG. 3A to the position in FIG. A new virtual area 37 is set in the real space near the vehicle 100 as a virtual area that fills this distance. As a result, the region 27 corresponding to the virtual region 37 is newly set in the image 42 based on the relative positional relationship between the vehicle 100 and the virtual region 37.

  Next, it is assumed that the vehicle 100 further moves to the position shown in FIG. 3C at time t3, and at this time, an image indicated by reference numeral 43 in FIG. At this time, the above-described relative position calculation and projection are performed on the virtual regions 33 to 37 and the newly set virtual region 38 in the same manner as when the position is moved from the position of FIG. 3A to the position of FIG. Processing is performed, and the positions and shapes of the regions 23 to 28 in the image 43 are determined. Since the virtual area 32 exists at a position farther than the viewing distance 30 of the camera 7, the area 22 is not displayed in the image 43.

  As described above, the areas 23 to 28 are set on the captured image. Focusing this on one region, as the vehicle 100 sequentially moves to the positions shown in FIGS. 3A to 3C, the region of interest 24 (network) shown in FIGS. The position and the shape on the image change as in the case of the hanging portion). Next, a method for detecting an object around the vehicle 100 based on the regions 23 to 28 set on the image as described above will be described.

  The virtual areas 31 to 38 in the real space shown in FIG. 5 are the same as the virtual areas 31 to 38 in FIG. 5A to 5C show the positional relationship between the vehicle 100 at time t1 to t3 and the other vehicle 101 to be detected, which is located on the rear side of the vehicle 100. At time t2, it is assumed that the detection target vehicle 101 has entered the virtual area 34 as shown in FIG.

  FIG. 6 shows a temporal change in the feature amount of the region on the image corresponding to the virtual region 34, that is, the region 24 shown in FIG. 4 at times t1 to t3. This feature amount is the amount by which the average luminance value of the image in the area 24 has changed within a predetermined time. When only the road surface is captured in the region 24, the average luminance value does not change much and the feature amount is small. However, when a three-dimensional object such as a vehicle is also captured in the area 24, when the viewpoint of the camera 7 changes with the movement of the vehicle 100, the surface (direction) on which the three-dimensional object is captured changes, and Changes in the luminance information. Therefore, the change in the average luminance value, that is, the feature amount increases. In FIG. 6, at time t2 when the vehicle 101 enters the virtual area 34, the feature amount exceeds the threshold value. When the feature amount exceeds the threshold value, the vehicle 101 is detected.

  As described above, the surrounding object detection device 1 captures an image around the vehicle 100, sets a virtual region in the real space, and a region on the image corresponding to the virtual region. An object such as a surrounding vehicle is detected based on the feature amount of the upper region. Further, the surrounding object detection device 1 determines the risk of collision by predicting the course of the vehicle 100 and the detected object as described later.

  FIGS. 7 and 8 are flowcharts showing a flow of processing when the surrounding object detection device 1 detects a surrounding object. This flowchart is based on a program executed by the CPU 10 and is always executed when the surrounding object detection device 1 operates. The processing contents in each flow of FIGS. 7 and 8 will be described below using the examples of FIGS. In the following description, it is assumed that time t1 in FIGS. 3 and 4 is replaced with time 0, time t2 is replaced with time t, and time t3 is replaced with time t + Δt.

  In step S1 in FIG. 7, first, an image signal around the vehicle 100 captured by the camera 7 and stored in the memory 9, that is, an image 41 captured at the position of the vehicle 100 illustrated in FIG. This image is used as an initial image.

  In step S2, initial virtual areas 31 to 36 are set on the real space corresponding to the initial image 41 read in step S1, and areas 21 to 26 on the initial image corresponding to the virtual areas 31 to 36 are set. Set. The virtual areas 31 to 36 are, for example, 3 m square in the real space on a motorway, and are arranged on the traveling lane on the rear side of the vehicle 100 at intervals of 4 m along the center of the traveling lane. In addition, on general roads, the sizes and arrangement intervals of the virtual areas 31 to 36 in the real space are set to be smaller than those in the case of a dedicated automobile road so that an object smaller than a vehicle such as a motorcycle or a pedestrian can be detected. You. Note that the size and arrangement of the virtual area described above are merely examples, and other sizes and arrangements may be used.

  Further, in step S2, an ID number for distinguishing each of the set virtual areas 31 to 36 is assigned to each virtual area, and the relative position between the vehicle 100 and each virtual area is calculated. Further, an initial average luminance value is calculated for each of the regions 21 to 26 on the image. The average luminance value is represented by G (n, t), where n is the ID number of the area and t is the time, and the initial average luminance value calculated in step S2 is represented by G (n, 0).

In step S3, the initial virtual area 31 to 36 set in step S2, to create an initial virtual area map _{M 0.} Data structure of the virtual area map including the initial virtual area map M ₀ is as shown in FIG. In one virtual area map indicated by reference numeral 90, for example, information on each virtual area at time 0 is recorded for each ID number. For example, in the portion of ID number n shown by reference numeral 93, the relative position between the virtual region and the vehicle 100, the average luminance value and the feature amount of the region on the image corresponding to the virtual region, the state value and the vehicle Label information is recorded, and the same applies to other ID number portions. For example, similar information is recorded in virtual area maps indicated by reference numerals 91 and 92 corresponding to times t and t + Δt. Virtual area map initial virtual space map M ₀ is created at the beginning in step S3, when subsequently repeats the process from step S4 to be described later to step S23 as one processing cycle, the processing cycle in step S6 to be described later It is created for each.

In the initial virtual area map M ₀ created in step S3, the average luminance value and the relative position calculated in step S2 (initial relative position) only (initial average luminance value) is recorded. Other feature quantities, each state information values and the vehicle labels, because this is the time not yet been determined, not recorded in the initial virtual area map M _0. Initial virtual area map M ₀ created is stored in the memory 9.

  In step S4, the process waits for a time Δt. This time Δt is preset based on the processing time in each step other than step S4 and the time of one processing cycle when one processing cycle from step S4 to step S27 is repeated. In the following description, for simplicity, the processing time in each step other than step S4 is set to 0, and the time of one processing cycle is equal to the time Δt.

  In step S5, an image signal of the rear side of the vehicle 100, which is captured by the camera 7 and stored in the memory 9, is read. This image was taken after a lapse of time Δt from the image read in step S5 or step S1 in the previous processing cycle. Here, it is assumed that the image 42 is read, and the time at which the image 42 was captured is described as t.

In step S6, a virtual area map Mt at time _t is created for the initial virtual areas 31 to 36 set in step S2. When this step S6 is performed for the second time or later, a virtual area map is created for the virtual area set in the previous processing cycle. At this point, the virtual area map M _t created only set ID number corresponding to the virtual area 31 to 36 are each information for each ID number described in Step S3 is not yet recorded. The created virtual area map _Mt is stored in the memory 9.

  In step S7, the movement amount Vt of the vehicle 100 at the time t is obtained. The movement amount Vt is an amount of movement of the vehicle 100 in the real space during one processing cycle, that is, the time Δt, and is based on the wheel speed pulse of the vehicle 100 output from the vehicle sensor 8 as described above. Required by dead reckoning. Further, it is determined whether or not the obtained moving amount Vt is larger than the threshold value Vs. If it is larger, the process proceeds to step S8, and if not, the process proceeds to step S10. Here, the threshold value Vs is an amount of movement that can set one or more virtual areas on the real space anew, and is determined by the size of the virtual areas set in step S2. Note that the comparison between the movement amount Vt and the threshold value Vs is performed based on the movement amount when moving in the same direction.

  In step S8, a new virtual area 37 in the real space is set according to the movement amount Vt obtained in step S7. Further, for the virtual area 37, similarly to the virtual areas 31 to 36 set in step S2, an ID number is assigned and a relative position with respect to the vehicle 100 is calculated. Further, an area 27 corresponding to the virtual area 37 is set on the image 42, and the average luminance value is calculated. Note that the number of new virtual areas set in step S8 is determined by how many virtual areas can be arranged in the movement amount Vt. That is, for example, when Vs <Vt ≦ 2Vs, one new virtual area is set, and when 2Vs <Vt ≦ 3Vs, two new virtual areas are set.

In step S9, the virtual area map _{M t} generated in step S6, the ID number of the virtual regions 37 set in step S8, similarly the relative position between the vehicle 100 and the virtual region 37 calculated in step S8, and the region 27 of the The information of the average luminance value is added, and the virtual area map _Mt is updated.

In step S10, the virtual area 31 to 37 ID number in the virtual space map _{M t} is recorded, based on the amount of movement Vt calculated in step S7, calculates respectively the relative position of the vehicle 100. The virtual area map M _t is created at step S6, those updated in step S9. The calculated relative position is recorded in the virtual space map M _t, the virtual space map M _t is updated.

In step S11, by selecting the ID number recorded in the virtual space map M _t, which selects one of the virtual area 31 to 37. Here, while the processing from step S11 to step S17 described below is repeated in the same processing cycle, the virtual region selected once in step S11 is not selected in subsequent step S11.

In step S12, it is determined whether the virtual area selected in step S11 is within the observation range of the camera 7. This determination is performed by the relative position of the vehicle 100 which is recorded in the virtual area map M _t, it is determined as the relative position when it becomes a certain value or more, not in the observation area. If it is in the observation range, that is, if any of the virtual regions 32 to 37 is selected in step S11, the process proceeds to step S13. If it is not in the observation range, that is, if the virtual region 31 is selected in step S11, the process proceeds to step S17. At this time, information of the virtual area 31 is not in the observation range are removed from the virtual area map M _t, the virtual area map created in subsequent processing cycle the next time, the information is not recorded.

In step S13, one of the regions 22 to 27 corresponding to the virtual region determined to be in the observation range of the camera 7 in step S12 is set on the image 42 captured in step S5. Position and shape of the region 22 to 27 in this case, with respect to any of the corresponding virtual area 32-37, by performing projection processing on the basis of the relative position recorded in the virtual space map M _t in step S10 ,Desired.

In step S14, an average luminance value is calculated for any of the regions 22 to 27 on the image 42 set in step S13. Calculated average luminance value is recorded in the virtual space map M _t, the virtual space map M _t is updated.

In step S15, a feature amount is calculated for any of the regions 22 to 27 on the image 42 set in step S13. This feature amount is defined as an absolute value of a difference between the average luminance value calculated in step S14 and the average luminance value calculated in step S14 of the immediately preceding processing cycle, and is expressed by the following equation (1). Is done. In the equation (1), G (n, t) represents the average luminance value at the time t and the ID number n of the area, and G (n, t−Δt) represents the average luminance value in the processing cycle one time earlier than that. Represents. In the first processing cycle, the initial average luminance value G (n, 0) calculated in step S2 is used instead of G (n, t-Δt). At this time, the feature amount is set to 0 for the area newly set in step S8 of the current processing cycle, that is, the area 27. Calculated feature quantity is recorded in the virtual space map M _t, the virtual space map M _t is updated.
D (n, t) = | G (n, t) −G (n, t−Δt) | (1)

In step S16, based on the feature amount calculated in step S15, a state value is calculated for one of the virtual regions 32 to 37 determined to be in the observation range of the camera 7 in step S12. The state value is determined depending on whether or not the feature value is larger than the threshold value, and is 1 when the feature value is larger than the threshold value and is 0 when the feature value is equal to or smaller than the threshold value. As described above, when the state value of the virtual area becomes 1, an object around the vehicle 100 is detected. Calculated state value is recorded in the virtual space map M _t, the virtual space map M _t is updated.

In step S17, determines all the virtual area 31 to 37 ID number in the virtual space map _{M t} is recorded, whether it has selected at step S11. When all are selected, the process proceeds to step S18 in FIG. 8, and when there is a virtual region that has not been selected, the process returns to step S11.

In step S18 in FIG. 8, the state value of the virtual area 32 to 37 recorded in the virtual area map M _t determined in step S16, it is determined whether the status value is virtual area is 1. When there is at least one or more virtual areas whose state value is 1, the process proceeds to step S19. If the state values of the virtual areas 32 to 37 are all 0, the process returns to step S4 in FIG. At this time, the processing after step S19 is not performed.

In step S19, for the virtual region having the state value of 1, a clustering process is performed in which adjacent virtual regions in the real space are set as one virtual region group. For example, when the status values of the virtual areas 32 and 33 and the virtual areas 36 and 37 among the virtual areas 32 to 37 are 1, the virtual areas 32 and 33 are combined into one virtual area group _AG and the virtual area 36. 37 and a different one of the virtual region group B _G and the virtual region group a _G. Note that this clustering process is not performed for a virtual area having no adjacent virtual area having a state value of 1.

In step S20, vehicle labels are added to the virtual areas 32 and 33 and the virtual areas 36 and 37 whose state values are 1. The vehicle labels, the processing cycle of one time before, that is, when the state value of the virtual area is 1 in the virtual space map M _{t-Delta] t} at time t-Delta] t, are recorded in the virtual space map M _{t-Delta] t} The vehicle label is the same as that of the previous processing cycle. Further, at this time, the same vehicle label is added to a virtual region adjacent to the front (closer to the vehicle 100) in the same virtual region group. On the other hand, when the state value of the virtual area is 0 in the virtual area map _Mt-Δt , a new vehicle label is added. For example, the state value of the virtual area 32 in one previous processing cycle is 1, the vehicle label was A _L, the virtual area adjacent forward in the virtual area 32, and the same virtual area group A _G 33, a vehicle label _AL is added. On the other hand, a new vehicle label _BL is added to the virtual areas 36 and 37.

  In the processing in step S20 described above, the real space in which the virtual area is set assumes a traveling lane in the same direction as vehicle 100, and the traveling direction of the vehicle detected by the virtual area is in the same direction as vehicle 100. It is. In addition, the time Δt of one processing cycle is assumed to be a sufficiently short time compared to the moving speed of the vehicle traveling in the side lane, and the distance that the vehicle moves during the time Δt is one virtual area. Minutes or less. For example, assuming that the moving speed of the vehicle is 120 km / h and the time Δt is a frame rate time of a general camera of 33 ms, the moving distance of the vehicle during that time is 1.1 m. Is greater than this distance 1.1 m. Therefore, as described above, the same vehicle label is added to the same vehicle in the same virtual region group as to the virtual region adjacent to the front in the previous processing cycle by adding the same vehicle label to the same vehicle. be able to. When the real space in which the virtual area is set is the opposite lane of the vehicle 100, the same vehicle label is added to the virtual area adjacent to the rear. As a result, the same vehicle label as that added in the previous processing cycle can be added to the same vehicle.

In step S20 of the first processing cycle, a new vehicle label is added to all virtual areas. In addition, when the same vehicle label is added to the virtual areas of the different virtual area groups, a new vehicle label is added to the virtual area of the rear (farther to the vehicle 100) virtual area group. I do. As described above, by adding the vehicle labels to the virtual areas having the state value of 1, the detected surrounding objects of the vehicle 100 can be determined for each vehicle. The added vehicle label is recorded in the virtual area map M _t, the virtual space map M _t is updated.

  In step S21, the relative position of the detected surrounding object with respect to the vehicle 100 at time t + Δt at which the next processing cycle is executed is predicted. At this time, the relative position of the forefront virtual area in the virtual area group is set as the relative position of the surrounding object detected at that time. By calculating the difference between the relative positions of the surrounding objects with respect to the virtual area group having the same vehicle label at time t and time t-Δt, the movement in which the surrounding objects move with respect to the vehicle 100 during the time Δt Find the quantity. That is, the amount of movement of the surrounding object is obtained from the difference between the relative position of the virtual region 33 at time t and the relative position of the virtual region 32 at time t-Δt. Further, the relative position with respect to the vehicle 100 at the time t + Δt is predicted by adding the obtained movement amount of the surrounding object to the relative position with respect to the vehicle 100 at the time t.

  In step S22, it is determined whether or not the relative positions of the surrounding objects predicted in step S21 are within the expected collision range. If the relative position is less than or equal to the predetermined distance from the vehicle 100, it is determined that the vehicle is within the expected collision range, and the process proceeds to step S23. Otherwise, it is determined that it is not within the expected collision range, and the process returns to step S4 in FIG.

  In step S23, the alarm device 11 is operated to notify the driver that there is a risk of collision. After executing step S23, the process returns to step S4 in FIG.

According to the surrounding object detection device according to the above-described first embodiment, the periphery of the vehicle 100 is photographed by the camera 7 and a plurality of virtual areas 31 to 38 fixed to the real space are set on the surrounding real space. Then, since objects around the vehicle 100 are detected based on information of images obtained by photographing the virtual regions 31 to 38, the objects around the vehicle 100 may be detected due to a difference in a projection plane or a surrounding shadow. Even if the image information changes, the object can be detected.
Further, according to the surrounding object detection device, an object around the host vehicle is detected as follows. The virtual areas 31 to 38 fixed in the real space around the vehicle 100 are set. The relative positional relationship between the vehicle 100 and the virtual regions 31 to 38 is calculated, and the regions 21 to 28 corresponding to the virtual regions 31 to 38 are displayed on an image obtained by photographing the periphery of the vehicle 100 based on the calculated positional relationship. Set. The feature amounts of the virtual regions 31 to 38 are calculated based on the image information of the regions 21 to 28 on the image, and objects around the vehicle 100 are detected based on the calculated feature amounts. As a result, the surrounding objects of the own vehicle are detected based on the image information based on the virtual area fixed in the real space. Even if it changes, the object can be detected.

――Second embodiment――
A second embodiment of the surrounding object detection device according to the present invention will be described. In the second embodiment, the method of calculating the feature amount of the virtual area is performed by a method different from that of the first embodiment. In the first embodiment, a method has been described in which a region on an image corresponding to a virtual region in a real space is obtained, and a feature amount is calculated based on a change in an average luminance value on the image. In the second embodiment, a change in a histogram of luminance values is used for calculating a feature amount, instead of the average luminance value used in the first embodiment. The configuration of the second embodiment is the same as that of the first embodiment shown in FIG.

  FIG. 10 shows a part of a flowchart showing the flow of processing in the second embodiment. This flowchart is based on a program executed by the CPU 10 as in the first embodiment, and is always executed when the surrounding object detection device 1 operates. In FIG. 10, processing steps indicated by the same step numbers as those in FIG. 7 perform the same processing as in the first embodiment, and a description thereof will not be repeated. After step S17 in FIG. 10 is executed, the same processes as those in steps S18 to S23 in FIG. 8 are executed, so that the flowcharts after step S18 are not shown.

ただし、
Ｎ ：着目する領域に含まれる総画素数
ｉ ：着目する領域に含まれる各画素の輝度値
Ｉ_min ：着目する領域におけるｉの最小値
Ｉ_max ：着目する領域におけるｉの最大値
μ ：着目する領域に含まれる全画素の平均輝度値
ｈ（ｉ） ：着目する領域に含まれる輝度値がｉの画素数 In step S2A, as in step S2 of FIG. 7, an initial virtual area is set on the real space corresponding to the initial image read in step S1, and an area on the initial image corresponding to this virtual area is set. . Further, an ID number is assigned to each virtual area, and a relative position between the vehicle 100 and each virtual area is calculated. At this time, for the area on the image, unlike the step S2 in FIG. 7, the variance (initial variance) of the histogram of the luminance value for each area is calculated by the following equation (2). In the equation (2), S (n, t) represents a variance value at time t at the ID number n of the area. In this step S2A, an initial variance value S (n, 0) at time 0 is calculated.

However,
N: Total number of pixels included in the region of interest i: Brightness value of each pixel included in the region of interest I _min : Minimum value of i in the region of interest I _max : Maximum value of i in the region of interest μ: Attention Average luminance value h (i) of all pixels included in the region: Number of pixels whose luminance value included in the region of interest is i

  In step S8A, as in step S8 of FIG. 7, a new virtual area in the real space is set according to the movement amount obtained in step S7. Further, also for the virtual area set here, similarly to the virtual area set in step S2A, an ID number is assigned to each virtual area, and the relative position of the vehicle 100 and each of these virtual areas is calculated. At this time, for the regions on the image corresponding to these virtual regions, the variance of the luminance value is calculated for each region by the above equation (2), unlike step S8 in FIG.

In step S14A, for the region on the image set in step S13, the variance of the luminance value of the region is calculated by the above equation (2). The calculated variance is recorded in the virtual space map M _t, the virtual space map M _t is updated.

In step S15A, the absolute value of the difference between the variance value calculated in step S14A and the variance value calculated in step S14A of the immediately preceding processing cycle is calculated by the following equation (3), and D ′ ( (n, t) is the feature amount of the virtual region. In equation (2), S (n, t-Δt) represents a variance value in the immediately preceding processing cycle. In the first processing cycle, the initial dispersion value S (n, 0) calculated in step S2A is used instead of S (n, t-Δt). At this time, the feature amount is set to 0 for the area newly set in step S8A of the current processing cycle. Calculated feature quantity is recorded in the virtual space map M _t, the virtual space map M _t is updated. Using this feature amount, vehicles around the vehicle 100 are detected by the same method as in the first embodiment.
D ′ (n, t) = | S (n, t) −S (n, t−Δt) | (3)

  11 and 12 show examples of detecting surrounding vehicles by the above-described method. FIG. 11 is a diagram illustrating an example when there is no surrounding vehicle. At time t shown in FIG. 11A, it is assumed that a histogram of luminance values when the virtual area 120 in the real space is projected on the image is, for example, the one shown in FIG. 11C. At time t + α after time t shown in FIG. 11B, a histogram of luminance values when the virtual area 120 in the real space is projected on the image is, for example, a histogram shown in FIG. 11D. Assume that In this case, the histogram of the luminance values shown in FIGS. 11C and 11D shows almost no change because the vehicle does not enter the virtual area 120. At this time, the variance values of the luminance values shown in the histograms of FIGS. 11C and 11D have substantially the same value because the histogram does not change. Therefore, the feature amount in the virtual area 120 calculated based on the variance value is small, and no vehicle is detected.

  FIG. 12 is a diagram illustrating an example when a surrounding vehicle enters. At the time point t shown in FIG. 12A, it is assumed that the histogram of the brightness values when the virtual area 120 in the real space is projected on the image is, for example, the one shown in FIG. 12C. Also, at time t + α after time t shown in FIG. 12B, a histogram of luminance values when the virtual area 120 in the real space is projected on the image is, for example, a histogram shown in FIG. 12D. Assume that In this case, since the vehicle 101 has entered the virtual area 120 in FIG. 12B, the histogram of the luminance values greatly changes as shown in FIGS. 12C to 12D. At this time, the variance values of the luminance values shown in the histograms of FIGS. 12C and 12D are significantly different because the histogram has changed greatly. Therefore, the feature amount in the virtual area 120 calculated by the variance value increases, and when the feature amount exceeds the threshold value, the vehicle 101 is detected.

  In the above example, the change in the histogram of the brightness value is determined based on the difference between the variance values. However, the change may be determined by another method. For example, the determination may be made based on the difference between the frequency and the luminance value at the peak of the histogram (the point where the frequency is highest).

  According to the surrounding object detection device according to the above-described second embodiment, the same operation and effect as those of the first embodiment can be obtained.

--Third embodiment--
A third embodiment of the surrounding object detection device according to the present invention will be described. In the present embodiment, in order to prevent erroneous detection of surrounding objects caused by a discretization error included in the movement amount of the vehicle when calculating the relative position of the virtual region, a method of using a motion component on the virtual region of interest as a feature amount is described. Used. The configuration in the present embodiment is the same as the first and second embodiments shown in FIG.

  In the first and second embodiments described above, the moving amount Vt of the vehicle during the time Δt is obtained by dead reckoning from the number of wheel speed pulses from each of the right and left wheels, and the dead reckoning information of the moving amount Vt is used. The example of calculating the relative position between the vehicle and the virtual area has been described. However, since the wheel speed pulse represents the discretized number of rotations of the wheel that is output every time the wheel rotates by a predetermined number, the moving amount Vt calculated from the wheel speed pulse number includes: The left and right wheels each include a discretization (quantization) error of one pulse or less. This discretization error causes an error in the relative position between the vehicle and the virtual area.

The error in the relative position of the virtual area caused by the discretization error included in the movement amount Vt of the vehicle can be divided into an error component in the traveling direction of the vehicle and an error component in the inclination direction of the axle. The maximum error amount E _f of the former error components are those corresponding to the amount of movement of the traveling direction of the vehicle per pulse, when the outer periphery of the wheel w, per pulse the rotational speed of the wheel and r, the following Equation (4).
E _f = w · r (4)

In the above equation (4), for example, assuming that the outer circumference of the wheel is 187 cm and one wheel speed pulse is output every 1/88 revolution as described in FIG. _Ef is 187 × 1/88 ≒ 2 cm. As described above, since the magnitude of the error component in the traveling direction of the vehicle is smaller than that of the vehicle or the virtual area, the influence on the above-described surrounding object detection processing is small.

On the other hand, in the latter error component with respect to the inclination direction of the axle, the error in the amount of movement by the left and right wheels is the maximum, and the error amount becomes the maximum when the sign of the error with respect to the true value is reversed in the left and right. The maximum error amount E _a, that is equivalent to the amount of movement of the tilt direction per 2 pulses axles, when the tread of the vehicle is d, is represented by the following formula (5).
E _a = 2 · w · r / d (5)

In the above equation (5), for example, the wheel outer circumference and the wheel speed pulse output are the same as those in the above-described equation (4), and if the tread of the vehicle at this time is 142 cm, the maximum error amount with respect to the inclination direction of the axle. E _a is a 2 × 187 × 1/88 ÷ 142 ≒ 0.030rad = 1.72 °. At this time, in the relative position of the virtual area with respect to the vehicle, for example, at a position 20 m behind the vehicle, a lateral position shift of about 60 cm occurs. Here, the lateral position is a direction perpendicular to the traveling direction of the host vehicle, and corresponds to the x-axis direction in a coordinate system based on the host vehicle 100 shown in FIG. FIG. 14 shows an example of this lateral displacement. FIG. 14A shows the original positions of the virtual regions 31 to 36 calculated at the time t. On the other hand, at the time t + Δt, due to the relative position error described above, the lateral positions are shifted as shown in FIG. 14B, and the virtual regions 31 to 36 and the new virtual region 37 are set. At this time, for example, in the virtual area 34, a white line 50 that should not be originally included is included in the virtual area.

  As described above, when an error occurs in the horizontal position of the virtual area due to the discretization error of the dead reckoning information, a white line that should not exist originally may be included in the virtual area. Then, the luminance value of the road surface changes in this virtual area. As in the first and second embodiments described above, when other surrounding vehicles are detected by a change in the brightness value of the road surface in the virtual area, the change in the brightness value leads to erroneous detection. .

  In the present embodiment, in order to prevent erroneous detection from occurring even if an error occurs in the lateral position of the virtual region due to such discretization error of the dead reckoning information, a direction toward the host vehicle as a feature amount, that is, a direction of FIG. When a motion component in the y-axis direction is detected using a motion component on the virtual area in the y-axis direction, it is determined that there is a moving object heading for the own vehicle in the virtual area. With this configuration, when an error occurs in the lateral position of the virtual area, a motion component appears in the x-axis direction but does not appear in the y-axis direction. It is determined that there is no moving object toward. This can prevent erroneous detection due to a discretization error in dead reckoning information. Here, the motion component is a ground speed vector, which is different from the relative speed with respect to the own vehicle. Therefore, it is possible to detect an object whose ground speed vector is heading in the direction of the host vehicle, regardless of the direction of the relative body speed.

  FIG. 15 is a flowchart showing the flow of processing according to the third embodiment. This flowchart is based on a program executed by the CPU 10 as in the first and second embodiments, and is always executed when the surrounding object detection device 1 operates. Note that in FIG. 15, the processing steps indicated by the same step numbers as those in FIGS. 7 and 8 perform the same processing as in the first embodiment, and thus description thereof will be omitted.

At step S12A, it is determined whether it is in the observation area of the camera 7 for all virtual area recorded in the virtual area map M _t at that time, the virtual area which is located outside the observation area than the virtual area map M _t delete. Determination of whether the observation range in step S12A is carried out by the relative positions of the virtual area and the vehicle 100 recorded in the same way virtual area map M _t a step S12, it relative position and a predetermined value or more , It is determined that the virtual area is not in the observation range. The information on the deleted virtual area is not recorded in the virtual area map created in the next and subsequent processing cycles. Although the virtual area map used in the present embodiment has the same structure as that shown in FIG. 9, the vehicle labels described in the first and second embodiments are the same as the virtual area map of the present embodiment. Is not remembered.

In step S13A, virtual area which is recorded in the virtual space map M _t at that time (of the virtual area calculated the relative position in the step S10, excluding the virtual area that is deleted in step S12A) corresponding to all An area on the image is set on the image captured in step S5 by the same method as described in step S13. The position and shape of each region on the image, as in the case of step S13, for the corresponding virtual area, based on the relative position recorded in the virtual area map M _t is calculated in step S10 projection By performing the processing, it can be obtained.

In step S14B, an average luminance value is calculated for each area set on the image in step S13A. On behalf of the average luminance value calculated in step S14B, the average luminance value of the region whose ID number is m calculated at time t is represented by I (m, t). In the following description, description will be made using this average luminance value I (m, t). The average luminance value calculated is recorded in the virtual space map M _t, the virtual space map M _t is updated.

In step S15B, based on the average luminance value of each area calculated in step S14B, the presence or absence of a movement toward the own vehicle in each virtual area is calculated using a calculation method described below. First, for the area of ID number m for which the average luminance value I (m, t) has been calculated in step S14B, the difference from the average luminance value in the previous processing cycle (time t-Δt) is obtained by the following equation (6). . Hereinafter, this is referred to as a time differential value of the average luminance value.
Dt (m, t) = I (m, t) -I (m, t-Δt) (6)

Next, a virtual area corresponding to the area with the ID number m is set as a target virtual area. For example, the virtual area 61 shown in FIG. Further, among the virtual areas 62 to 65 located in the vicinity of the target virtual area 61 in the real space, the virtual area 62 located farthest from the host vehicle is set as the adjacent virtual area, and The ID number of the corresponding area on the screen is m ′. Then, the difference between the average luminance value of the area of the ID number m and the area of the set ID number m ′ in the current processing cycle (time t) is obtained by the following equation (7). Hereinafter, this is referred to as a spatial differential value of the average luminance value.
Dm (m, m ', t) = I (m, t) -I (m', t) (7)

Further, similarly to the above equations (6) and (7), the time differential value of the average luminance value in the area of ID number m ′ and the spatial differential value of the average luminance value in the previous processing cycle (time t−Δt) Is determined as in the following equations (8) and (9). It should be noted that the spatial differential value of the average luminance value in the previous processing cycle of Expression (9) may not be obtained in the current processing cycle, but may be the one obtained by Expression (7) in the previous processing cycle. Good.
Dt (m ′, t) = I (m ′, t) −I (m ′, t−Δt) (8)
Dm (m, m ', t-Δt)
= I (m, t−Δt) −I (m ′, t−Δt) (9)

From the time differential value of the average luminance value obtained by the above expressions (6) and (8), the average of the virtual region of interest and the adjacent virtual region in the time differential value of the average luminance value is calculated by the following expression (10). That is, a spatial average of the time differential value of the average luminance value is obtained. Further, from the spatial differential value of the average luminance value obtained by Expressions (7) and (9), the average of the current processing cycle and the previous processing cycle in the spatial differential value of the average luminance value is calculated by the following Expression (11). That is, the time average of the spatial differential value of the average luminance value is obtained.
St (m, t) = {Dt (m, t) + Dt (m ′, t)} / 2 (10)
Sm (m, m ', t)
= {Dm (m, m ′, t) + Dm (m, m ′, t−Δt)} / 2 (11)

Further, a motion component vector (ground speed vector) in the direction toward the host vehicle is calculated from the above equations (10) and (11) by the following equation (12), and this vector V (m, t) is calculated from 0. It is determined whether it is larger.
V (m, t) = − St (m, t) / Sm (m, m ′, t)> 0 (12)

  When a vehicle traveling toward the host vehicle is present in the virtual area of interest, the ground speed vector V (m, t) calculated in this manner becomes larger than 0, as described below. Therefore, when Expression (12) is satisfied, it is determined that the vehicle traveling toward the own vehicle exists in the virtual area. In making the determination of Expression (12), it is first determined whether or not the time average of the spatial derivative of the average luminance value shown in Expression (11) is larger than a predetermined threshold θ. It is preferable to make the determination of Expression (12) when the value becomes larger than the threshold value θ. In this way, when the vehicle enters the target virtual area 61 from the adjacent virtual area 62 located behind the target virtual area 61, a change in the average luminance value of the corresponding area can be detected.

  The reason why the ground speed vector V (m, t) becomes larger than 0 when a vehicle traveling toward the host vehicle is in the virtual region of interest will be described with reference to an example shown in FIG. First, a case will be described in which the average luminance value of the virtual region changes due to the inclusion of a white line in the virtual region of interest due to the above-described error in dead reckoning. At this time, as shown in FIG. 17A, at time t-Δt, the target virtual area m indicated by the reference numeral 71 and the adjacent virtual area m ′ indicated by the reference numeral 72 contain nothing other than the road, At time t, it is assumed that those virtual areas include the white line 73.

  In the case shown in FIG. 17A, at time t, the target virtual area m and the adjacent virtual area m 'include white lines at almost the same ratio. Therefore, the value of the spatial differential value Dm (m, m ', t) of the average luminance value shown in Expression (7) becomes close to zero. Further, since the white line is not included at the time t−Δt, the value of the spatial differential value Dm (m, m ′, t−Δt) of the average luminance value shown in Expression (9) is almost zero. Therefore, Sm (m, m ', t) ≒ 0 according to equation (11). Therefore, as described above, a predetermined threshold value θ is set for performing the determination of Expression (12), and by setting the θ to an appropriate value, Sm (m, m ′, t ) <Θ, and erroneous detection can be prevented.

  Next, a case will be described in which a vehicle traveling toward the host vehicle exists in the virtual area of interest. At this time, as shown in FIG. 17 (b), the vehicle 74 is included in the adjacent virtual area m ′ at the time t-Δt, and the vehicle 74 approaches the own vehicle thereafter. It is assumed that the vehicle 74 is included in both the virtual area m and the adjacent virtual area m ′. Hereinafter, the case where the luminance value of the vehicle is higher than the road surface and the case where the luminance value is lower will be described separately.

When the brightness value of the vehicle is higher than the road surface, the ratio of the vehicle occupying the virtual region in the virtual region of interest m and the adjacent virtual region m ′ is larger at the time t than at the time t−Δt, The average luminance value increases. From this, for example, at time t-Δt, the average luminance values of the target virtual region m and the adjacent virtual region m ′ are I (m, t−Δt) = 100 and I (m ′, t−Δt) = 150, respectively. At time t, it is assumed that I (m, t) = 150 and I (m ′, t) = 200. At this time, equations (6) to (9) are calculated as follows.
Dt (m, t) = 50
Dt (m ', t) = 50
Dm (m, m ', t) =-50
Dm (m, m ′, t−Δt) = − 50

When the ground speed vector is calculated from the above calculation result, the following is obtained, and the expression (12) is satisfied.
V (m, t) =-{(50 + 50) / 2} / {(-50-50) / 2} = 1> 0

On the other hand, when the luminance value of the vehicle is lower than the road surface, the average luminance value of the target virtual area m and the adjacent virtual area m ′ is lower at the time t than at the time t−Δt. From this, for example, the average luminance values of the target virtual area m and the adjacent virtual area m ′ at time t−Δt are I (m, t−Δt) = 150 and I (m ′, t−Δt) = 100, respectively. It is assumed that at time t, I (m, t) = 100 and I (m ′, t) = 50. At this time, equations (6) to (9) are calculated as follows.
Dt (m, t) =-50
Dt (m ', t) =-50
Dm (m, m ', t) = 50
Dm (m, m ′, t−Δt) = 50

When the ground speed vector is calculated from the above calculation result, the following is obtained, and also in this case, Expression (12) is satisfied.
V (m, t) =-{(-50-50) / 2} / {(50 + 50) / 2} = 1> 0

  Next, a case will be described in which a vehicle traveling in the opposite direction to the own vehicle exists in the virtual region of interest. At this time, as shown in FIG. 17C, the vehicle 75 is included in the virtual region of interest m at the time t-Δt, and the vehicle 75 then moves away from the own vehicle. It is assumed that the vehicle 75 is included in both m and the adjacent virtual area m ′. In this case, when the ground speed vector is calculated in the same manner as described above, V (m, t) <0 in both cases where the vehicle brightness value is high and low compared to the road surface, and the expression (12) is satisfied. Not done.

  As described above, the ground speed vector V (m, t) becomes larger than 0 only when the vehicle traveling toward the own vehicle exists in the virtual region of interest, and satisfies the expression (12). Accordingly, by calculating the ground speed vector V (m, t) for the area with the area ID number m, it is possible to calculate the presence or absence of a movement toward the own vehicle. By doing so, the processing of steps S18 to S20 in FIG. 8 in the first and second embodiments, that is, the processing for determining whether or not the moving object is heading for the own vehicle is performed in the present embodiment. It becomes unnecessary. As a result, the processing amount can be reduced.

In the same manner as described above, the ground speed vectors are calculated for all the areas set on the image in step S13A, and the presence or absence of a movement toward the own vehicle on each virtual area is calculated. The value of the ground speed vector of each such region calculated in step S15B in the are recorded in the virtual space map M _t as the feature quantity, the virtual space map M _t is updated.

In step S16A, obtains the state value of each virtual region by the presence or absence of movement towards the vehicle on each virtual area calculated in step S15B, and records it in the virtual space map M _t. At this time, the state value is set to 1 for the virtual area in which the movement toward the own vehicle is calculated, that is, the virtual area in which the ground speed vector calculated as described above is larger than 0. For the other virtual areas, the state value is set to 0. After execution of step S16A, step S22 is executed, and if step S22 is affirmatively determined, step S23 is executed, and the process returns to step S4.

  According to the third embodiment described above, the time differential value of the average luminance value is calculated by Expression (6) for the region corresponding to the target virtual region. Further, a spatial differential value is calculated by equation (7) for the virtual area of interest and an adjacent virtual area that is adjacent to the virtual area of interest in the real space of the vehicle in the real space. Then, based on the calculated time differential value and spatial differential value of the average luminance value, the ground speed vector of the virtual region of interest is calculated by Expression (12). Further, the feature amount of the virtual area is calculated by calculating the ground speed vector for all the virtual areas (step S15B). Then, a state value of each virtual area is obtained based on the calculated feature amount (step S16A). As a result, the same operation and effect as those of the first and second embodiments can be obtained. Furthermore, when an error occurs in the horizontal position of the virtual area due to the discretization error of the dead reckoning information and a white line or the like which should not exist in the virtual area is included, erroneous detection can be prevented.

  In the above embodiment, for example, the photographing means is realized by the camera 7, the vehicle motion detecting means is realized by the vehicle sensor 8, the moving amount calculating means, the virtual area setting means, the relative position calculating means, the projection processing means, the image area The setting unit, the feature amount calculating unit, and the surrounding object detecting unit are realized by the CPU 10. However, these are merely examples, and each component is not limited to the above-described embodiment as long as the features of the present invention are not impaired.

It is a figure showing composition of the present invention. FIG. 3 is a diagram illustrating a visual axis direction of a camera 7. It is a figure which shows the relationship between the vehicle 100 which is driving | running | working on a road, and the virtual area | region set in real space, (a) shows the state of time t1, (b) at time t2, (c) at time t3. Shown respectively. It is a figure which shows the image image | photographed by the camera 7 mounted in the vehicle 100 which is driving | running | working on a road, and the area | region set on an image, (a) is time t1, (b) is time t2, (c) is The state at time t3 is shown. It is a figure which shows the relationship between the vehicle 100 which is driving | running | working on the road, and the virtual area | region set in real space, and the relationship with the vehicle of the detection object which exists around the vehicle 100, (a) is time t1, time t1, FIG. (B) shows the state at time t2, and (c) shows the state at time t3. FIG. 6 is a diagram illustrating a temporal change of an average luminance value in an area on an image corresponding to a virtual area in the situation illustrated in FIG. 5. It is a figure which shows steps S1-S17 among the flowcharts which show the flow of the process of surrounding vehicle detection in 1st Embodiment. It is a figure which shows steps S18-S23 among the flowcharts which show the process of a surrounding vehicle detection in 1st Embodiment. FIG. 4 is a diagram illustrating a data structure of a virtual area map. It is a figure which shows steps S1-S17 among the flowcharts which show the flow of the process of surrounding vehicle detection in 2nd Embodiment. It is a figure which shows the relationship between the example of the image image | photographed by the camera 7 in 2nd Embodiment, and the histogram of a brightness value, (a) is an example of the image at the time t in the situation where there is no surrounding vehicle, (b) ) Shows an image example at time t + α, (c) shows an example of a histogram of luminance values in the image example shown in (a), and (d) shows an example of a histogram of luminance values in the image example shown in (b). . It is a figure which shows the relationship between the example of the image image | photographed by the camera 7 in 2nd Embodiment, and the histogram of a brightness value, (a) is an example of the image at the time t in the situation where a surrounding vehicle exists, (b) ) Shows an image example at time t + α, (c) shows an example of a histogram of luminance values in the image example shown in (a), and (d) shows an example of a histogram of luminance values in the image example shown in (b). . FIG. 3 is a diagram illustrating a coordinate system based on a host vehicle. It is a figure which shows the example of the shift of the horizontal position of the virtual area | region caused by the discretization error of dead reckoning information, (a) shows the original position at the time t, (b) shows the position where the horizontal position shifted at the time t + (DELTA) t. FIG. It is a figure showing a flow chart which shows a flow of processing of surrounding vehicles detection in a 3rd embodiment. FIG. 4 is a diagram illustrating a positional relationship between a virtual area of interest and an adjacent virtual area. It is a figure for explaining the reason that the ground speed vector becomes larger than 0 when the vehicle which runs toward the own vehicle exists in the attention virtual area, and (a) is when the white line is included in the attention virtual area. (B) shows an example in which a vehicle traveling toward the own vehicle exists in the virtual area of interest, and (c) shows an example in which a vehicle traveling in the direction opposite to the own vehicle exists in the virtual area of interest. Shown respectively.

Explanation of reference numerals

1 Surrounding Object Detecting Device 100 Own Vehicle 101 Other Vehicles 21 to 28 to be Detected Regions 31 to 38 Set on Captured Image Virtual Region Set on Real Space

Claims (6)

Photographing means for photographing around the vehicle;
A plurality of virtual areas fixed to the real space are set on the real space around the vehicle, and an object around the vehicle is detected based on information on an image of the virtual area captured by the image capturing means. A surrounding object detecting device, comprising: detecting means. In a surrounding object detection device mounted on a vehicle and detecting an object around the vehicle,
Photographing means for photographing the periphery of the vehicle;
Vehicle motion detection means for detecting the motion of the vehicle,
A movement amount calculation unit that calculates a movement amount of the vehicle based on the movement of the vehicle detected by the vehicle movement detection unit;
Virtual area setting means for setting a plurality of virtual areas fixed to the real space on the real space around the vehicle, and setting the new virtual area according to the movement amount;
A relative position calculating unit that calculates a relative positional relationship between the vehicle and the virtual area based on the moving amount of the vehicle calculated by the moving amount calculating unit;
Projection processing means for calculating the position and shape in the image when the virtual area is projected on the image shot by the shooting means, based on the positional relationship calculated by the relative position calculation means,
Image area setting means for setting an area corresponding to the virtual area on the image based on the position and shape calculated by the projection processing means,
A feature value calculation unit that calculates a feature value representing a feature of the virtual region based on image information in the region set by the image region setting unit;
A surrounding object detection unit that detects surrounding objects of the vehicle existing in the virtual region based on the feature amount calculated by the feature amount calculation unit. The surrounding object detection device according to claim 2,
Surrounding object detection characterized by comprising a notifying means for predicting a future position of the surrounding object detected by the surrounding object detecting means, and notifying the position when the position is within the expected collision range of the vehicle. apparatus. The surrounding object detection device according to claim 2 or 3,
The surrounding object detection device according to claim 1, wherein the feature amount calculating unit calculates a feature amount representing a feature of the virtual region based on a change amount of an average luminance value of the region set by the image region setting unit. The surrounding object detection device according to claim 2 or 3,
The surrounding object detection device according to claim 1, wherein the feature amount calculating unit calculates a feature amount representing a feature of the virtual region based on a change in a histogram of luminance values of the region set by the image region setting unit. The surrounding object detection device according to claim 2 or 3,
The feature amount calculation unit sets a virtual region corresponding to any one of the regions set by the image region setting unit in the virtual region as a target virtual region,
For the area corresponding to the virtual area of interest among the set areas, a time differential value of an average luminance value that is a difference between the average luminance value at the current time and the average luminance value a predetermined time before the current time is calculated.
Of the virtual areas adjacent to the noted virtual area in the real space, the area farthest from the vehicle is defined as an adjacent virtual area, and the area corresponding to the noted virtual area in the set area is defined as the area. Calculating the spatial differential value of the average luminance value that is the difference between the average luminance value of the area and the average luminance value of the area corresponding to the adjacent virtual area,
Calculating a ground speed component in a direction toward the vehicle on the virtual region of interest based on the time differential value and the spatial differential value of the calculated average luminance value,
A surrounding object detection apparatus, wherein a feature amount representing a feature of the virtual area is calculated by calculating the ground speed component for each virtual area.

Images