JP6114572B2 - Object area estimation method, object area estimation apparatus, object detection apparatus including the object area, and vehicle. - Google Patents

Description

  The present invention relates to an object area estimation method, an object area estimation apparatus, an object detection apparatus including the object area, and a vehicle that estimate an existing area of an object using an image sensor.

  2. Description of the Related Art Conventionally, some golf carts that run on a course in a golf course have an automatic running function. Automatic traveling is performed by detecting a magnetic field generated from an electromagnetic induction wire. In addition, there is an apparatus provided with an obstacle detection device that detects an obstacle in front of the golf cart by providing an ultrasonic sensor in front of the golf cart. When an obstacle is detected by the ultrasonic sensor, the golf cart is controlled to decelerate or stop (see Patent Document 1).

  In addition, when a person is detected as an obstacle, there is a method for detecting a person by image processing. When a person is detected, the person to be detected exists in various sizes at various positions in the image. Therefore, it is necessary to determine the presence of the person at various positions in the image at various sizes. Therefore, an area where a person is likely to exist is detected by scanning the entire image using a rectangular area abstracted as an image model. This rectangular area has various sizes. Thereby, it is possible to detect an area where a person is likely to exist.

  17 and 18 are explanatory diagrams illustrating a method for detecting an area of an object such as an obstacle or a person. By scanning the detection windows W1 and W2 with respect to the image Im, a region where an object is likely to exist is detected. A method of detecting an object by directly obtaining a HOG (Histograms of Oriented Gradients) feature in the detection window, and estimating the existence area of the object using the detection window as a template matching, and then, in the corresponding area, (Oriented Gradients) There is a method of detecting an object by obtaining features (see Patent Document 2).

(1) Technology of Patent Document 1 The golf cart described in Patent Document 1 is equipped with an automatic traveling device. Thereby, a runway can be automatically run along the guide line provided on the runway. It also describes that an ultrasonic sensor is used to measure the distance to an obstacle and the presence or absence of the obstacle is determined according to the distance.

(2) Technology of Patent Document 2 Patent Document 2 describes an object detection device and a detection method. By sequentially scanning a rectangular area as an image model from the left to the right of the image frame and from top to bottom, an area where a person is likely to exist is detected. It is described that it is determined whether or not the person is a person by calculating the HOG feature amount of the detection region.

JP 2003-5832 A JP2011-248525A

  However, scanning the entire image with rectangular areas of various sizes is a large calculation load and requires a lot of calculation time. In particular, when the determination process is performed with an arithmetic device having a low processing capability, more processing time is required. As a result, it takes a long time to detect an object such as an obstacle or a person, which hinders the processing after the object is detected and impairs the function of the entire system.

  The present invention has been made in view of such circumstances, and an object area estimation method, an object area estimation apparatus, an object detection apparatus including the object area, and a vehicle that can easily estimate an existence area of an object The purpose is to provide.

In order to achieve the above object, the present invention has the following configuration.
That is, the first invention according to the present invention is a shooting step for shooting a front image, a parallax image creating step for creating a parallax image based on the image, and a distance for extracting distance information based on the parallax image. An information extraction step, a cluster division step of dividing the pixels on the horizontal line into a plurality of clusters based on the distance information on a predetermined horizontal line in the image or the parallax image, and the image or the parallax in the image, the lateral object area determination step of determining presence region in the lateral direction of the captured object based on the size of the cluster from among the clusters in the image or the parallax images photographed and a longitudinal object area determination step of determining vertical existence region of the object, wherein the lateral line has at least two or more, the object vertical The direction region determining step includes the step of determining the image or the parallax image when there is no other horizontal line above the uppermost horizontal line having a cluster having a size within a predetermined range in the image or the parallax image. When there is another horizontal line above the uppermost horizontal line having a cluster having a size within a predetermined range in the image or the parallax image. The uppermost horizontal line is set as the upper limit of the region where the object exists, and the image or the parallax image has the lowest cluster having a size within a predetermined range. When there is no other horizontal line below the horizontal line, the lower limit of the image or the parallax image is set as the lower limit of the region where the object exists, and the image or the parallax image When there is another horizontal line below the lowermost horizontal line having a cluster having a size within a predetermined range, the horizontal line immediately below the lowermost horizontal line is set as the lower limit of the region where the object exists. This is an object region estimation method.

According to the first aspect of the present invention, the front image is captured by the capturing step. In the parallax image creation step, a parallax image is created based on the captured image. In the distance information extraction step, distance information is extracted based on the created parallax image. In the cluster division step, the pixels on the horizontal line are divided into a plurality of clusters based on distance information on the horizontal line determined in advance in the image or the parallax image. The lateral object area determination step determines the existence region of the lateral object based on the size of the clusters from the cluster.

  Since only the distance information on a predetermined line is used to estimate the object area, the calculation load can be reduced and the calculation speed can be improved. Further, pixel grouping can be easily performed by dividing the cluster, and comparison with the size of an object can be facilitated.

The second invention according to the present invention is an image sensor that captures a forward image, a parallax image creation unit that creates a parallax image based on the image, and a distance that extracts distance information based on the parallax image. An information extraction unit; a cluster division unit that divides pixels on the horizontal line into a plurality of clusters based on the distance information on a predetermined horizontal line in the image or the parallax image; and determining the lateral object region determination unit determines a lateral existence region of the object based on the size of the clusters in the image or the parallax images, the longitudinal existence region of the photographed object comprising a longitudinal object region determination unit, wherein the lateral line has at least two or more, the longitudinal object region determination unit, in the image or the parallax images, within a predetermined range magnitude When there is no other horizontal line above the uppermost horizontal line having a cluster, the upper limit of the image or the parallax image is set as the upper limit of the existence area of the object, and the image or the parallax image is determined in advance. If there is another horizontal line above the uppermost horizontal line having a cluster having a size within a predetermined range, the upper horizontal line is set to the upper limit of the region where the object exists. And in the image or the parallax image, when there is no other horizontal line below the lowest horizontal line having a cluster having a size within a predetermined range, the lower limit of the image or the parallax image is set to There is another horizontal line below the lowest horizontal line having a cluster having a size within a predetermined range in the image or the parallax image as the lower limit of the region where the object exists. If, as an object region estimation apparatus according to the lower limit of the existence region of the object horizontal line of one of a horizontal line of the lowest.

According to the second invention, a front image is taken by the image sensor. The parallax image creation unit creates a parallax image based on the captured image. The distance information extraction unit extracts distance information based on the created parallax image. The cluster dividing unit divides the pixels on the horizontal line into a plurality of clusters based on distance information on the horizontal line determined in advance in the image or the parallax image. The horizontal direction object region determination unit determines the presence region in the horizontal direction of the target object from the clusters based on the size of the cluster.

  Since only the distance information on a predetermined line is used to estimate the object area, the calculation load can be reduced and the calculation speed can be improved. Further, pixel grouping can be easily performed by dividing the cluster, and comparison with the size of an object can be facilitated.

  The size of the cluster is preferably the width of the cluster. Since the width of the object has an upper limit determined to some extent, the region where the object exists can be accurately estimated by using the width of the cluster as a reference.

  The cluster dividing unit divides pixels on the vertical line into a plurality of clusters based on the distance information on a predetermined vertical line in the image or the parallax image, and the object region determination unit Preferably, the presence area of the object is determined from the clusters based on the width of the clusters on the horizontal line and the height of the clusters on the vertical line.

  In addition to the cluster division of pixels on a predetermined horizontal line, the cluster division unit also performs cluster division on pixels on the vertical line based on distance information on the predetermined vertical line. The object area determination unit determines an existing area of the object from the divided clusters based on the cluster width on the horizontal line and the cluster height on the vertical line. In addition to the cluster width in the horizontal line, the object region is estimated based on the height of the cluster in the vertical line, so that the estimation accuracy can be improved. In addition, since the frequency with which the object is on the line is increased as compared with the case of the horizontal line alone, it is easy to estimate the object region even if the distance from the object is large.

  A plurality of horizontal lines, and the object region determination unit includes coordinates of horizontal lines having clusters having a size within a predetermined range on the parallax image, and sizes within a predetermined range. The existence area of the object may be determined based on the coordinates of the horizontal line that does not have the same cluster. By providing a plurality of horizontal lines, it is possible to further limit the estimated range of the region where the object exists. Depending on the positional relationship between a horizontal line having clusters of a size within a predetermined range on a parallax image and a horizontal line having no clusters of a size within a predetermined range, objects in the horizontal direction and the vertical direction An object area can be estimated.

  Moreover, the target object detection apparatus which concerns on this invention is provided with the said target object area estimation apparatus and the target object detection part which detects a target object in the target object area | region estimated by the target object area estimation apparatus. Since the object region estimation device is provided, the object region can be estimated. Since the object only needs to be detected within the estimated object area, the scan area can be limited, and the object detection time can be shortened.

  The object may be a person. A region can be estimated and detected with high accuracy even for an object having a variation in shape or movement like a person.

  Moreover, the vehicle which concerns on this invention is a vehicle provided with the said target object detection apparatus. Since the object detection device is provided, the object can be detected at a high speed, so that contact avoidance with the object can be improved.

  According to the present invention, it is possible to provide an object area estimation method, an object area estimation apparatus, an object detection apparatus provided with the object area, and a vehicle that can easily estimate the existence area of the object.

1 is a front view of a vehicle according to a first embodiment. 1 is a block diagram illustrating a configuration of a vehicle according to a first embodiment. It is a block diagram which shows the structure of the target object detection apparatus which concerns on Example 1. FIG. It is explanatory drawing which shows the relationship between the stereo camera which concerns on Example 1, and the direction of distance information. It is explanatory drawing which shows the relationship between the parallax image which concerns on Example 1, and the preset line. 3 is a visualization graph of forward distance information according to the first embodiment. 6 is a visualization graph of forward distance information divided into clusters according to the first embodiment. It is explanatory drawing which shows the distance information stored concerning Example 1. FIG. 3 is a flowchart illustrating a flow of object detection according to the first embodiment. FIG. 6 is an explanatory diagram illustrating a limited scan area according to the first embodiment. It is a block diagram which shows the structure of the target object detection apparatus which concerns on Example 2. FIG. FIG. 10 is an explanatory diagram illustrating vertical scanning area restriction according to the second embodiment. FIG. 10 is an explanatory diagram illustrating vertical scanning area restriction according to the second embodiment. 10 is a flowchart illustrating a flow of object detection according to the second embodiment. FIG. 10 is an explanatory diagram illustrating a limited scan area according to the second embodiment. It is explanatory drawing which shows the scanning area restriction | limiting of the vertical and horizontal direction which concerns on a modification. It is explanatory drawing which shows the detection method of the target object area | region which concerns on a prior art example. It is explanatory drawing which shows the detection method of the target object area | region which concerns on a prior art example.

  Embodiment 1 of the present invention will be described below with reference to the drawings. As an embodiment of a vehicle in the present invention, a golf cart that automatically runs is given. Vehicles are not limited to golf carts, but also include automated guided vehicles that run in factories and orchards. The vehicle is not limited to a four-wheeled vehicle, and may be a three-wheeled vehicle or a monorail type. In the following description, front and rear and left and right are based on the direction in which the vehicle 1 moves forward.

1. Schematic configuration of vehicle Referring to FIGS. 1 and 2. FIG. 1 is a front view of a vehicle 1 according to the embodiment, and FIG. 2 is a functional block diagram showing a configuration of the vehicle 1. The vehicle 1 is a golf cart that travels automatically or manually in a golf course. The vehicle 1 can be automatically driven by being guided by electromagnetic waves emitted from a guide wire embedded in the runway. A stereo camera 3 is provided in the center of the front surface of the vehicle 1.

  Further, the vehicle 1 includes an object detection device 5 that detects an object in front of the vehicle 1, an automatic travel control unit 7 that controls the vehicle 1 to automatically travel along a guide line, and an object detection device. When 5 detects an object, a warning output unit 8 that generates a warning to the driver and the surroundings, a vehicle speed control unit 9 that controls deceleration or stop by detecting the object, a wheel is driven, and a vehicle speed control unit 9 A drive motor 11 whose rotational speed is controlled is provided. In the present embodiment, the vehicle 1 is driven by the drive motor 11, but is not limited thereto, and may be driven by an engine.

2. Configuration of Object Detection Device Next, the configuration of the object detection device 5 provided in the vehicle 1 will be described with reference to FIG. FIG. 3 is a block diagram illustrating a configuration of the object detection device.

  The object detection device 5 includes an object region estimation device 13 and an object detection unit 15 that detects the object. The object area estimation device 13 detects an area in which the object is estimated to exist in the captured image. Thereby, it is possible to limit the scan area for detecting the object. The object detection unit 15 scans a limited area in the image and detects the object.

  The object region estimation device 13 includes a stereo camera 3 that captures an image in front of the vehicle 1, an image buffer 17 that temporarily stores an image captured by the stereo camera 3, and an image captured by the stereo camera 3. A parallax image creation unit 19 that creates a parallax image based on the distance information extraction unit 21 that extracts distance information of each pixel from the created parallax image, a distance information buffer 23 that stores the extracted distance information, and an extraction A cluster dividing unit 25 that divides the pixels in the parallax image based on the distance information, a target region determination unit 27 that determines a region where the target exists based on each of the divided clusters, and a target And a scan area setting unit 29 for setting an area determined to exist as a scan area. The parallax image creation unit 19, the distance information extraction unit 21, the cluster division unit 25, the object region determination unit 27, the scan region setting unit 29, and the object detection unit 15 are a microprocessor or an FPGA (Field Programmable Gate Array; reconfiguration) Possible gate array). Next, each component will be described in order.

  The stereo camera 3 includes two image sensors 3a and 3b. The stereo camera 3 may be composed of two or more image sensors. The left image sensor 3a and the right image sensor 3b are general visible light sensors such as a CCD and a CMOS. The left image sensor 3a and the right image sensor 3b are installed under a predetermined geometric condition. In the first embodiment, the left image sensor 3a and the right image sensor 3b are installed at a constant distance in the horizontal direction. That is, the left image sensor 3a and the right image sensor 3b are arranged in a parallel stereo positional relationship. In the left image sensor 3a and the right image sensor 3b, camera calibration is performed in advance so that the positions of the respective rows of the captured images are matched, that is, the epipolar lines are matched. An image taken by the left image sensor 3a is taken as a left eye image, and an image taken by the right image sensor 3b is taken as a right eye image. In the first embodiment, the left image sensor 3a is a reference camera, and the left eye image is a reference image. The right eye image may be used as the reference image.

  The image buffer 17 temporarily stores images sent from the stereo camera 3, that is, images sent from the image sensors 3a and 3b. The images stored in the image sensors 3a and 3b are sequentially output to the parallax image creation unit 19. The image buffer 17 uses a memory, a flash memory, a hard disk (HDD), or the like.

  The parallax image creation unit 19 creates a parallax image from each input image. In the first embodiment, a parallax image is created based on the left eye image and the right eye image stored in the image buffer 17. The parallax image can be created by stereo matching, for example. The area correlation method is used as stereo matching. In the area correlation method, the degree of coincidence of each image is calculated in a window unit of a certain size around the pixel of interest, and stereo matching is performed so that the degree of coincidence becomes the highest. Further, the parallax of the target pixel in the state where the degree of coincidence is the highest is calculated. The parallax here indicates the amount of pixel shift between a plurality of images. In the first embodiment, the parallax is a pixel shift amount in the horizontal direction of the right eye image with respect to the left eye image. Other stereo matching methods include SAD (Sum of absolute differences) and Census transform. Note that the X and Y coordinates on the parallax image are the same as the X and Y coordinates on the reference image.

  The distance information extraction unit 21 extracts a distance in real space to a subject displayed by each pixel on a preset horizontal line based on the parallax image. As a method for extracting the distance from the parallax image, a conventionally used parallel stereo method is used. That is, by using the distance (base line length of the stereo camera 3) T between the two image sensors 3a and 3b and the focal length f and the parallax image of the two image sensors 3a and 3b, the parallax image and the reference image are used. Distance information corresponding to each pixel can be calculated.

The distance information for the coordinates (X _C , Y _C ) on the parallax image can be obtained by the following equation.
d = np (1)
Z _D = fT / d (2)
_{X D = Z D (X C} -np) f ··· (3)
Y _D = Z _D · Y _C / f (4)
Here, (X _C , Y _C ) are coordinates in the parallax image with the center of the parallax image as the origin, and (X _D , Y _D , Z _D ) are the coordinates (X _C , Y _C ) of the parallax image. Corresponding distance information. D is the parallax distance, n is the number of parallax pixels, and p is the pixel pitch of the image sensors 3a and 3b. The directions of the stereo camera 3 and the distance information (X _D , Y _D , Z _D ) are shown in FIG. X _D axis is provided in the lateral direction of the stereo camera 3, Y _D axis is provided in the longitudinal direction of the stereo camera 3, Z _D axis is provided toward the front of the stereo camera 3 (the depth direction). Each calculated distance information is stored in the distance information buffer 23 corresponding to each pixel of the parallax image and the reference image.

This will be described with reference to FIG. FIG. 5 is an explanatory diagram showing the relationship between a parallax image and a preset line. Line L1 has a Y coordinate of _{Y CL1} on the parallax image, the line L2 has a Y coordinate of _{Y CL2} on the parallax image, the line L3 has a Y coordinate of _{Y CL3} on the parallax image. Thus, the Y coordinate of each line L1-L3 is preset. The distance information extraction unit 21 extracts distance information corresponding to the pixels on these lines L1 to L3. That is, in the parallax image, distance information corresponding to pixels whose Y coordinates are Y _CL1 , Y _CL2 , and Y _CL3 is extracted. For example, the extracted distance information in the depth direction on the line L1 is shown in FIG. In the first embodiment, three lines are set, but one line may be set or a plurality of other lines may be set. The distance information extraction unit 21 preferentially extracts the distance information of the pixels on the lines L1 to L3, and then extracts the distance information of other pixels in the parallax image.

The cluster dividing unit 25 divides each pixel on the line into a set number of clusters (lumps) based on distance information corresponding to pixels on a predetermined horizontal line in the parallax image. A conventional Ward method is adopted as a method of cluster division. Clusters may be divided using a method other than the Ward method. In Example 1, based on the distance information of the distance information and transverse depth direction (Z _D axis) (X _D axis), it is divided into four clusters. However, the number of clusters to be divided is not limited to four and may be set appropriately. As a result, each pixel on the line is divided into clusters according to the depth distance and the lateral distance, so that pixel groups that are close to each other are grouped as one cluster.

FIG. 7 shows a graph of distance information in the depth direction divided into clusters. Each pixel on the line L1 is divided into four clusters CL1 to CL4. In the first embodiment, the cluster number CL1 is set to the cluster closest to the vehicle 1, and the cluster number CL _NO is sequentially assigned as the distance increases. Cluster numbers CL _NO (CL1 to CL4) corresponding to the divided clusters are stored in the distance information buffer 23 in association with the X coordinate of each pixel.

The distance information buffer 23 stores distance information associated with the parallax image and the reference image pixels. FIG. 8 shows a relationship between predetermined X coordinates of the lines L1 to L3 and distance information. That is, the X coordinate value, the Y coordinate value, the lateral distance X _D , the longitudinal distance Y _D , the depth direction distance Z _D , and the cluster number CL on each of the lines L1 to L3 on the reference image (parallax image). Stored in association with _NO . Examples of the distance information buffer 23 include a memory, a flash memory, and a hard disk (HDD).

The object area determination unit 27 determines whether or not each cluster is an object in order from the cluster having the smallest depth distance. If the cluster width of each cluster is within a predetermined threshold range, it is determined as an object. Here, the cluster width is the width between the minimum value and the maximum value in the actual lateral distance information that each divided cluster has. For example, the minimum value is _{X D1} maximum in cluster CL1 in FIG. 7 because it is _{X D2,} the width of the cluster CL1 is _(X D2 _{-X D1).} As described above, the object region determination unit 27 calculates the cluster width of each cluster, and determines whether each cluster width is within a predetermined setting range. When detecting a person as an object, for example, the setting range may be set to 1 m or less.

  The scan region setting unit 29 sets the maximum value coordinate and the minimum value coordinate on the parallax image (reference image) in the horizontal direction of the cluster determined to be an object as the maximum value coordinate and the minimum value coordinate of the scan region. As a result, the horizontal scanning area for detecting the object is limited.

  The object detection unit 15 acquires a reference image from the image buffer 17 and acquires scan area coordinates from the scan area setting unit. The object detection unit 15 detects the object by raster scanning a plurality of detection windows in the reference image in the set scan region. The object may be an obstacle or a person. When detecting a person, it can be detected using HOG features. When the HOG feature is used, the distance information only needs to be extracted only in the scan region, so that the amount of calculation can be greatly reduced compared to extracting the distance information of the entire image.

  Next, the object detection operation in the first embodiment will be described with reference to FIG. FIG. 9 is a flowchart showing a processing procedure for detecting an object.

A front image is taken by the stereo camera 3 provided on the front surface of the vehicle 1 (step S01). The captured left eye image and right eye image are stored in the image buffer 17. Next, the parallax image creation unit 19 creates a parallax image based on the image stored in the image buffer 17 (step S02). Next, the distance information extraction unit 21 uses the horizontal (X _D ) direction of the pixels on the lines L1, L2, and L3 having predetermined heights (Y _CL1 , Y _CL2 , and Y _CL3 coordinates) in the created parallax image. The distance information in the vertical (Y _D ) direction and the depth (Z _D ) direction is extracted (step S03).

The extracted distance information is stored in the distance information buffer 23. Further, based on the extracted distance information, the cluster dividing unit 25 performs cluster division on the pixels on the lines L1 to L3 (step S04). The cluster number CL _NO assigned after the cluster division is stored in the distance information buffer 23 in association with the X coordinate of the pixel on the corresponding parallax image and other distance information.

  Next, the object region determination unit 27 calculates a cluster width that is an actual distance in the horizontal direction of each divided cluster. (Step S05). Furthermore, the object area determination unit 27 determines whether there is a cluster having a width within a preset range (step S06). If there is no cluster having a width within the set range (No in step S06), the processing for this shooting frame is finished, and the processing for the next shooting frame is started.

  When there is a cluster having a width within the preset range (Yes in step S06), the scan area setting unit 29 sets the minimum value and the maximum value of the X coordinate in the parallax image of the cluster having the width within the set range, respectively. Then, the minimum and maximum values of the horizontal scanning area are set (step S07). That is, it can be estimated that a cluster having a width within the set range is an area where an object exists. When it is determined that the cluster width is within the set range in the plurality of clusters, the minimum value and the maximum value of the cluster closest to the stereo camera 3 among the plurality of clusters are set to the minimum value of the horizontal scanning region and Set as the maximum value.

  Next, the object detection unit 15 detects the object using the HOG feature amount or the like within the scan region Ar1 restricted in the horizontal direction (see FIG. 10) (step S08). When using the HOG feature amount, the distance information extraction unit 21 extracts the distance information only within the scan region Ar1 restricted in the horizontal direction to detect the object. When an object is detected, an alarm is issued to the operator and the surroundings from an alarm output unit 8 composed of a speaker or the like. When the object is detected, the vehicle speed control unit 9 performs deceleration control and brake braking, the drive motor 11 and the rotation of the wheels are braked, and the vehicle 1 decelerates or stops.

  As described above, according to the first embodiment, it is possible to easily estimate the existence area of the object. Since the estimation of the region where the object exists is estimated based on the actual distance information extracted by the distance information extraction unit 21, the estimation accuracy can be improved. Further, if the distance information is extracted only on a preset line, the presence / absence of the target region can be determined, so that the target region can be estimated at high speed.

  Further, since the scan area for detecting the object is limited, the time for detecting the object can be shortened. Thereby, the frame rate for detecting the object can be increased, and the object can be detected at shorter time intervals.

  Next, the target object detection apparatus according to the second embodiment will be described with reference to FIG. FIG. 11 is a block diagram illustrating a configuration of the object detection device according to the second embodiment. In the second embodiment, the parts indicated by the same reference numerals as those in the first embodiment have the same configuration as in the first embodiment, and thus the description thereof is omitted here. The configurations of the vehicle and the object detection device other than those described below are the same as those in the first embodiment.

  The feature of the second embodiment is that the estimation of the existence area of the object is also performed in the vertical direction. In the first embodiment, the estimation of the target area is limited only in the horizontal direction, but in the second embodiment, the estimation is performed in the vertical direction in addition to the horizontal direction. Thereby, it is possible to limit the scan area in the vertical direction.

  The object area estimation device 13 ′ according to the second embodiment has a configuration in which a vertical direction object area determination unit 28 is added to the object area estimation device 13 ′ according to the first embodiment. In addition, the horizontal direction object area | region determination part 27 'in Example 2 is the same structure as the object area determination part 27 in Example 1, and has the same function. In the second embodiment, two or more predetermined lines from which distance information is extracted on the parallax image are set.

The vertical object region determination unit 28 determines the Y coordinate of a line that does not have a cluster having a width within the set range as a candidate for the maximum Y coordinate value or the minimum Y coordinate value of the scan region. In the initial setting, the maximum Y coordinate value of the scan area is set to Y _Cmax which is the maximum Y coordinate of the parallax image (reference image). Further, the minimum Y coordinate value of the scan area is set to Y _Cmin which is the minimum Y coordinate of the parallax image (reference image).

  If the Y coordinate value of a line that does not have a cluster with a width within the setting range is greater than the Y coordinate value of a line with a cluster with a width within the setting range, the line that does not have a cluster with a width within the setting range The Y coordinate value is updated as the maximum Y coordinate value of the scan area. In addition, when the Y coordinate value of a line that does not have a cluster having a width within the setting range is smaller than the Y coordinate value of a line having a cluster having a width within the setting range, the cluster having a width within the setting range is not included. The Y coordinate value of the line is updated as the minimum Y coordinate value of the scan area.

With reference to FIGS. 12 and 13, the region determination of the object in the vertical direction will be further described. FIG. 12 shows a case where a cluster having a width of a set range is detected on one line, and FIG. 13 shows a case where a cluster having a width of the set range is detected on a plurality of lines. Three lines L1 to L3 are set on the parallax image. In FIG. 12, the horizontal direction object area determination unit 27 determines that the width of the cluster CL1 on the line L1 is within the setting range, and determines that the other clusters are not within the setting range. In this case, it is presumed that the object Ob1 exists between the line L2 located above the line L1 where the cluster having the width of the set range is detected and the line L3 located below the line L1. Can do. Therefore, the maximum value of the vertical scan area is updated as Y _CL2 which is the Y coordinate of the line L2, and the minimum value of the vertical scan area is updated as Y _CL3 which is the Y coordinate of the line L3. The scan area can be limited.

In FIG. 13, the lateral object region determination unit 27 determines that the widths of the cluster CL5 on the line L1 and the cluster CL6 on the line L2 are within the set range, and determines that the other clusters are not within the set range. Is done. In this case, since no other line is located above the line L2, the object Ob2 exists between the line L3 located one line below the line L1 from Y _Cmax, which is the coordinate of the upper edge of the parallax image. Then it can be estimated. Therefore, the maximum value of the vertical scan area is set to Y _Cmax , which is an initial setting value, and the minimum value of the vertical scan area is updated to Y _CL3 , which is the Y coordinate of the line L3. Can be limited.

When a cluster having a width within the set range is detected on the line L1 and the line L3, the maximum value of the vertical scan area is set to Y _CL2 which is the Y coordinate of the line L2, and the vertical scan area By setting the minimum value to Y _Cmin that is the initial setting value, the vertical scanning area can be limited.

The scan area setting unit 29 ′ limits and sets the scan area in the vertical direction in addition to restricting the scan area in the horizontal direction as in the first embodiment. That is, the horizontal restriction is performed based on the cluster width, and the vertical restriction is performed based on the Y coordinate of each line, the maximum Y coordinate Y _Cmax and the minimum Y coordinate Y _Cmin of the reference image (parallax image). . If the vertical object area is not determined by the vertical object area determination unit 28, the maximum Y coordinate Y _Cmax and the minimum Y coordinate Y _Cmin of the reference image are set to the maximum of the scan area as the initial setting value. The Y coordinate value and the minimum Y coordinate value are used.

  Next, the object detection operation in the second embodiment will be described with reference to FIG. FIG. 14 is a flowchart showing a processing procedure for object detection.

Since Step S01 to Step S07 are the same as those in the first embodiment, the description thereof is omitted. Following step S07, the vertical object region determination unit 28 determines whether or not all the lines have clusters having a width within the set range (step S09). When all the lines have clusters having a width within the setting range (Yes in step S09), the vertical scan area is set by setting the maximum Y coordinate Y _Cmax and the minimum Y coordinate Y _Cmin set in advance in the reference image. Set (step S10). In other words, the vertical scanning area is not substantially limited.

  In step S09, if all the lines do not have a cluster having a width within the set range (No in step S09), the Y coordinate of the line having no cluster having a width within the set range is within the set range. It is determined whether it is larger than the Y coordinate of a line having a cluster of width (step S11).

  When the Y coordinate of a line not having a cluster having a width within the setting range is larger than the Y coordinate of a line having a cluster having a width within the setting range (Yes in step S11), the cluster having a width within the setting range is further determined. It is determined whether or not the Y coordinate of the line having no dot is smaller than the maximum Y value of the scan area (step S12). If the Y coordinate of the line having no cluster within the set range is smaller than the Y maximum value of the scan area (Yes in step S12), the Y coordinate value of this line is updated as the Y maximum value coordinate of the scan area (step S12). S13). When the Y coordinate of the line having no cluster within the set range is equal to or larger than the Y maximum value of the scan area (No in step S12), the process proceeds to the next process without performing any process.

  When the Y coordinate of a line that does not have a cluster having a width within the setting range is smaller than the Y coordinate of a line having a cluster within the setting range (No in step S11), the cluster having a width within the setting range is further present. It is determined whether the Y coordinate of the line not to be processed is larger than the Y minimum value of the scan area (step S14). When the Y coordinate of a line that does not have a cluster having a width within the set range is larger than the Y minimum value of the scan area (Yes in step S14), the Y coordinate value of this line is updated as the Y minimum value coordinate of the scan area. (Step S15).

  If the Y coordinate of the line not having a cluster having a width within the set range is equal to or smaller than the Y minimum value of the scan area (No in step S14), the process proceeds to the next process without performing any process. The scan area setting unit 29 sets the vertical scan area using the Y maximum value coordinate and Y minimum value coordinate of the scan area set in this way (step S10). Within these scan regions limited in the horizontal and vertical directions, the object detection unit 15 detects the object as in the first embodiment (step S08). It should be noted that when moving to the processing of the next shooting frame, the Y maximum value coordinate and Y minimum value coordinate of the scan area are returned to the initial values.

  Thus, according to the second embodiment, the object region is estimated in the vertical direction in addition to the horizontal direction on the parallax image. As a result, as shown in FIG. 15, the scan area for object detection can also be restricted in the vertical direction, and the time for object detection can be further shortened.

  The present invention is not limited to the above embodiment, and can be modified as follows.

  (1) In the above embodiment, the lines L1 to L3 to be divided into clusters are horizontal lines. The present invention is not limited to this, and in addition to the horizontal line, as shown in FIG. 16, the vertical line may be provided, and the target area may be estimated by dividing the vertical line into clusters. In this case, the cluster dividing unit 25 divides the pixels on the vertical line preset on the parallax image into a plurality of clusters based on the distance information of each pixel. The object area determination unit 27 determines an existing area of the object based on the cluster width on the horizontal line and the cluster height on the vertical line among the divided clusters. For example, in FIG. 16, if the height of the cluster CL7 in the real space is within a preset vertical setting range, it is determined as an object region, and if not within the setting range, it is determined not to be an object region. To do. If it is determined as the object region, the maximum value coordinate and the minimum value coordinate of the cluster CL7 are set as the Y maximum value coordinate and the Y minimum value coordinate of the scan region, respectively. As a result, the scan area can also be restricted in the vertical direction.

  (2) In the above embodiment, the cluster width is used as a reference for estimating the region of the object. However, the present invention is not limited to this, and the area of the object may be estimated based on the length in the depth direction of the cluster in addition to the cluster width.

  (3) In the above embodiment, the stereo camera 3 is employed to create a parallax image, and distance information is acquired by performing stereo matching. For example, the distance information may be acquired by a radar scanner. In this case, the image acquired by the monocular image sensor needs to be associated with the distance information acquired by the radar scanner in advance.

  (4) In the said Example, although the target object detection apparatus was equipped with the vehicle, it is not restricted to this. In addition, for example, it may be adopted in a vision system for a robot that travels autonomously or a support system for a visually impaired person.

DESCRIPTION OF SYMBOLS 1 ... Vehicle 3 ... Stereo camera 3a ... Left image sensor 3b ... Right image sensor 5, 5 '... Object detection apparatus 13, 13' ... Object area estimation apparatus 15 ... Object detection part 19 ... Parallax image creation part 21 ... Distance information extraction unit 25 ... cluster division unit 27 ... object region determination unit 29 ... scan region setting unit

Claims (7)

A shooting step to take a front image;
A parallax image creating step of creating a parallax image based on the image;
A distance information extracting step of extracting distance information based on the parallax image;
A cluster dividing step of dividing the pixels on the horizontal line into a plurality of clusters based on the distance information on a predetermined horizontal line in the image or the parallax image;
In the image or the parallax images, the lateral object area determination step of determining presence region in the lateral direction of the captured object based on the size of the cluster from among the clusters,
In the image or the parallax image, a vertical object region determination step for determining a vertical existence region of the captured object;
Equipped with a,
The horizontal line has at least two or more,
The object longitudinal direction region determining step includes:
In the image or the parallax image, when there is no other horizontal line above the uppermost horizontal line having a cluster having a size within a predetermined range, the upper limit of the image or the parallax image is set as the target. The upper limit of the existence area of
In the image or the parallax image, when there is another horizontal line above the uppermost horizontal line having a cluster having a size within a predetermined range, it is one higher than the uppermost horizontal line. The horizontal line is the upper limit of the area where the object exists,
In the image or the parallax image, when there is no other horizontal line below the lowest horizontal line having a cluster having a size within a predetermined range, the lower limit of the image or the parallax image is set as the object. The lower limit of the existence region of
In the image or the parallax image, when there is another horizontal line below the lowest horizontal line having a cluster having a size within a predetermined range, the horizontal line one line below the lowest horizontal line An object area estimation method in which a line is a lower limit of an object existing area . An image sensor for taking a forward image;
A parallax image creating unit that creates a parallax image based on the image;
A distance information extraction unit that extracts distance information based on the parallax image;
A cluster dividing unit that divides pixels on the horizontal line into a plurality of clusters based on the distance information on a predetermined horizontal line in the image or the parallax image;
And lateral object region determination unit determines a lateral existence region of the object based on the size of the cluster from among the clusters,
In the image or the parallax image, a vertical object region determination unit that determines a vertical existence region of the captured object;
With
The horizontal line has at least two or more,
The vertical direction object region determination unit,
In the image or the parallax image, when there is no other horizontal line above the uppermost horizontal line having a cluster having a size within a predetermined range, the upper limit of the image or the parallax image is set as the target. The upper limit of the existence area of
In the image or the parallax image, when there is another horizontal line above the uppermost horizontal line having a cluster having a size within a predetermined range, it is one higher than the uppermost horizontal line. The horizontal line is the upper limit of the area where the object exists,
In the image or the parallax image, when there is no other horizontal line below the lowest horizontal line having a cluster having a size within a predetermined range, the lower limit of the image or the parallax image is set as the object. The lower limit of the existence region of
In the image or the parallax image, when there is another horizontal line below the lowest horizontal line having a cluster having a size within a predetermined range, the horizontal line one line below the lowest horizontal line An object area estimation apparatus that uses a line as a lower limit of an object existing area . The object region estimation apparatus according to claim 2,
The size of the cluster is the width of the cluster. The object region estimation apparatus according to claim 2,
The cluster dividing unit divides the pixels on the vertical line into a plurality of clusters based on the distance information on a predetermined vertical line in the image or the parallax image,
The object area determination unit determines an object existing area from the clusters based on a width of the cluster on the horizontal line and a height of the cluster on the vertical line. The object region estimation device according to any one of claims 2 to 4 ,
An object detection apparatus comprising: an object detection unit that detects an object in an object area estimated by the object area estimation apparatus. The object detection device according to claim 5 ,
An object detection apparatus in which the object is a person. Vehicle having an object detecting apparatus according to claim 5 or 6.

Images