JP2008146106A - Equipment for detecting three-dimensional object - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To further efficiently detect a three-dimensional object. <P>SOLUTION: A device for detecting a three-dimensional object is provided with a foreground acquisition means for removing a background from an image including the three-dimensional object as an object, and for acquiring a foreground image region; an inclination calculating means for calculating an inclination for a predetermined reference axis of a central line of the foreground image region; a region extraction means for extracting a detection region from the foreground image region based on the inclination; an object detection means for detecting an object from the detection region. The detection region where the object should be detected is extracted based on the inclination of the center line so that even if the object is inclined, it is possible to set the detection region without eliminating the object. In one embodiment, when the inclination is a predetermined value or larger, the detection region is extracted in a line in parallel with the center line. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an apparatus for detecting a three-dimensional object.

  Conventionally, various methods for detecting a three-dimensional object have been proposed. For example, in Patent Document 1 below, a human body part and a face position of an occupant are extracted from temperature data of a thermal image obtained from an output of a matrix type infrared sensor using first and second threshold values. The method is described. In Patent Document 2 below, images obtained by an imaging device are grouped according to distance data, and each group is subjected to thinning processing, and an image portion having a high degree of linearity is approximated to a circle or an ellipse. If it is, a method for determining that it is a human head is described.

According to the technique of Non-Patent Document 1 below, the center line of the foreground part of the image is obtained, and the center point Cfg is set to a third point from the bottom, and is vertically cut out by α at equal intervals on the left and right. In the cut-out area, the center line is obtained again. An inclination of the center line is obtained, and a shoulder position is obtained using a T-shaped edge filter along the inclination. The intersection between the shoulder shape approximated by the quadratic curve and the center line is defined as the neck position. The part above the neck is detected as the head.
JP-A-10-315908 JP 2003-281551 A Daniel B. Russakoff, Martin Herman, “Head tracking using stereo”, Machine Vision and Applications, 2002

  In order to detect a desired object with better accuracy and to reduce the calculation load, it is desirable to specify a region in the image where the object is to be detected. If the region can be limited in the image, an object can be detected from the region with good accuracy, and the entire image does not have to be processed. In the method as described in Non-Patent Document 1, an image is cut out in the vertical direction at equal intervals on the left and right with respect to the center line, and the human head is detected using the cut out image as a processing target. According to such a method, since the image is uniformly cut out in the vertical direction, when the inclination of the object to be detected is large, there is a possibility that a part of the object may be excluded from the cut out image. There is a possibility that it cannot be detected.

  Therefore, there is a need for a technique that can detect an object more accurately while reducing the calculation load even when the inclination of the object is large.

  According to one aspect of the present invention, an apparatus for detecting a three-dimensional object includes means for removing a background from an image including a three-dimensional object as an object and acquiring a foreground image region, and a center of the foreground image region. Means for calculating an inclination (k) of a line (CL) with respect to a predetermined reference axis; means for extracting a detection area from the foreground image area based on the inclination; and means for detecting an object from the detection area; .

  According to the present invention, even when the object to be detected is tilted, the detection area corresponding to the tilt is extracted, so that the detection area can be optimally extracted without eliminating the object to be detected. . Here, an image including a three-dimensional object as an object can be obtained by an imaging device. Alternatively, an image obtained by a distance measuring device that measures the distance to the object may be used.

  According to one embodiment of the present invention, the first representing the position in the foreground image region that equally divides the number of pixels existing in the pixel row that becomes the first and second reference lines (La, Lb) into two. And the second center position (Ma, Mb) are calculated, respectively, and a line connecting the first and second center positions is obtained as the center line. In this way, the detection area can be extracted along the inclination of the object by using the inclination of the center line obtained to bisect the foreground image area.

  According to an embodiment of the present invention, when the inclination is equal to or greater than a predetermined value, the detection area is extracted by lines (PL1, PL2) parallel to the center line. Since the detection area is extracted parallel to the center line of the foreground area, the detection area can be extracted without removing the object even when the inclination of the object to be detected is large.

  According to one embodiment of the present invention, the parallel lines are set to have a predetermined width from the center line to the left and right. In one embodiment, when the slope is larger than a predetermined value, the left and right widths of the left-side width (A) and the right-side width (B) are set to the left side with respect to the center line. The distance can be set based on the relative size between the distance of the object included in the image area and the distance of the object in the image area on the right side of the center line. In one embodiment, when the inclination is smaller than a predetermined value, the foreground image region is defined by vertical lines (VL1, VL2) set to have the same width from the one of the first and second center positions to the left and right. The detection area is extracted by cutting out. Further, in one embodiment, the left width and the right width are set such that the sum of the left width and the right width is a predetermined ratio with respect to the maximum width of the foreground image area.

  In this way, the detection area can be extracted so as to be more suitable for the size and posture of the object. For example, an adult head and a child's head are different in size, but the detection region can be extracted so as to have a width suitable for each. In addition, the detection area can be extracted so as to have a size suitable for the orientation of the person with respect to the imaging device.

  According to one embodiment of the present invention, for each object in the image, each pixel row is obtained based on the means for acquiring the distance value to the object for each pixel and the distance value of the pixel on the center line. Means for calculating a second center line (CD) representing the reference distance value (D) of the pixel, and for each pixel row in the detection region, a reference distance value is obtained from the second center line, and the reference distance value And a first removing unit that removes a pixel having a distance value outside the first predetermined range (D ± z1) from the pixel row. The removal is performed for a predetermined range in the horizontal direction.

  According to the present invention, for example, an area that may erroneously detect an object can be removed from the detection area. For example, when the object is the head, the arm region can be removed by such limitation of the distance direction.

  In one embodiment of the present invention, the second center line has a distance value obtained by averaging distance values of pixels included in the first reference row and a distance value obtained by averaging distance values of pixels included in the second reference row. And based on the value. Alternatively, the second center line is calculated based on the distance value of the pixel at the first center position and the distance value of the pixel at the second center position. Thus, the second centerline can be calculated to represent the center of the object in the distance direction.

  According to one embodiment of the present invention, the first predetermined range is set to become smaller as the distance from the center line increases in the horizontal predetermined range ((d) of FIG. 10). In this way, the first predetermined range can be set along the shape of the object.

  According to an embodiment of the present invention, the first predetermined range has a first width (z3) in the direction in which the distance value increases with respect to the second center line, and the second range in the direction in which the distance value decreases. It is set to have a width (z4), and the first width is greater than the second width.

  According to an embodiment of the present invention, the predetermined range in the horizontal direction is set outside a predetermined central region of the detection region. Thus, the central part of the object can be stored without being removed by the first removing means.

  According to an embodiment of the present invention, in the predetermined central region, each pixel row in the detection region is outside the second predetermined range (D ± z2) with respect to the reference distance value of the pixel row. Second removal means for removing a pixel having a distance value from the pixel row is provided. In one embodiment, the second predetermined range is set to be larger than the first predetermined range.

  According to the present invention, in the second predetermined range, the removal can be realized with a threshold value different from that of the first removal means, and the erroneous detection of the object can be avoided. For example, when the object to be detected is the head, the second removal means can remove the area corresponding to the luggage previously held by the occupant, and the head is erroneously detected in the luggage area. Can be avoided.

  According to one embodiment of the present invention, a recess detection means for detecting whether or not a recess is formed in a predetermined lower area of the detection area, and if a recess is detected, the detection area is closed by a line segment composed of pixels. Means, means for detecting whether or not there is a hole in the detection area, and means for filling the hole with a pixel having a predetermined distance value if the hole is detected.

  If there is a recess, an object may be erroneously detected in the peripheral area of the recess. According to the present invention, since the concave portion is closed to form a hole and the hole is filled, such erroneous detection can be avoided.

  In one embodiment, the lowest point (Ta, Tb) in the vertical direction is obtained in each of the regions divided by the center line, and the midpoint (Tm) of the line segment connecting the two lowest points. And determining whether or not a pixel is present in a region between the midpoint and a point located at a predetermined height.

  According to an embodiment of the present invention, further, a means for obtaining a distance value to the object for the object in the image, and a gray scale value representing the distance value for each area obtained by subdividing the foreground image area. And a means for generating a gray scale image having a gray scale value corresponding to each region. The area extraction means extracts the detection area from the gray scale image. Furthermore, storage means for storing a model obtained by modeling a three-dimensional object and means for calculating a correlation value representing the similarity between the model and the image area in the detection area are provided. A three-dimensional object is detected by detecting an image region having the highest correlation value with the model in the detection region.

  According to the present invention, since the model is correlated with the limited detection region, the object can be detected more accurately while reducing the calculation amount. Since the three-dimensional object is detected by two-dimensional image processing, the amount of calculation can be further reduced.

  Next, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 shows a configuration of an object detection apparatus 1 according to an embodiment of the present invention.

  In this embodiment, the object detection device 1 is mounted on a vehicle, and an embodiment for detecting the head of an occupant will be described. However, it should be noted that the object detection device is not limited to this form. This will be described later.

  The light source 10 is disposed in the vehicle so that the head of an occupant sitting on a seat (seat) can be irradiated. In one example, it is arranged in the upper front part of the seat.

  The irradiated light is preferably near infrared light (IR light). Generally, the brightness at the time of shooting varies depending on environmental changes such as daytime and nighttime. Further, when strong visible light strikes the face from one direction, a shaded gradation is generated on the face. Near-infrared light has robustness against such illumination variations and shadow gradations.

  In this embodiment, a pattern mask (filter) 11 is provided on the front surface of the light source 10 so that a grid pattern light is applied to the passenger. With the pattern light, the calculation of the parallax described later can be made more accurate. This will be described later.

  The pair of imaging devices 12 and 13 are arranged near the occupant's head so as to capture a two-dimensional image including the occupant's head. The imaging devices 12 and 13 are arranged so as to be separated from each other by a predetermined distance from left to right, up and down, or diagonally. The imaging devices 12 and 13 have an optical bandpass filter so as to receive only near-infrared light from the light source 10.

  The processing device 15 is realized by, for example, a microcomputer including a CPU that executes various operations, a memory that stores programs and data, an interface that inputs and outputs data, and the like. Considering the above, in FIG. 1, the processing device 15 is represented by a functional block. Part or all of these functional blocks can be realized by any combination of software, firmware, and hardware. In one example, the processing device 15 is realized by a vehicle control device (ECU) mounted on the vehicle. The ECU is a computer including a CPU and includes a memory for storing a computer program and data for realizing various controls of the vehicle.

  The parallax image generation unit 21 generates a parallax image based on the two images captured by the imaging devices 12 and 13.

  Here, an example of a method for calculating the parallax will be briefly described with reference to FIG. The imaging devices 12 and 13 include imaging element arrays 31 and 32 and lenses 33 and 34 that are two-dimensionally arranged, respectively. The imaging element is, for example, a CCD element or a CMOS element.

  A distance (base line length) between the imaging devices 12 and 13 is represented by B. The image sensor arrays 31 and 32 are disposed at the focal lengths f of the lenses 33 and 34, respectively. An image of an object at a distance L from the plane on which the lenses 33 and 34 are located is formed at a position shifted by d1 from the optical axis of the lens 33 in the image sensor array 31, and only d2 from the optical axis of the lens 34 in the image sensor array 32. It is formed at a shifted position. The distance L is determined by L = B · f / d according to the principle of triangulation. Here, d is the parallax, and d = d2−d1.

  In order to obtain the parallax d, for a certain block of the image sensor array 31, a corresponding block in which the same object portion as that block is imaged is searched on the image sensor array 32. The size of the block can be arbitrarily set. For example, it may be one pixel, or a plurality of pixels (for example, 7 × 7 pixels) may be used as one block.

  According to one method, a block of an image obtained by one of the imaging devices 12 and 13 is scanned with respect to an image obtained by the other of the imaging devices 12 and 13, and a corresponding block is searched (block matching). The absolute value of the difference between the luminance value of one block (for example, the average of the luminance values of the pixels in the block) and the luminance value of the other block is obtained and used as the correlation value. The block having the smallest correlation value is found, and the distance between the two blocks at this time indicates the parallax d. Alternatively, the search process may be realized using other methods.

  If there is a block whose luminance value does not change, it becomes difficult to match the blocks, but the above pattern light can cause the luminance value to change in such a block, so block matching Can be realized more accurately.

  In the captured image obtained from one of the imaging devices 12 and 13, the parallax value calculated for each block is associated with the pixel value included in the block, and this is used as the parallax image. The parallax value represents the distance of the object imaged by the pixel with respect to the imaging devices 12 and 13. The larger the parallax value is, the closer the position of the object is to the imaging devices 12 and 13.

  The background removal unit 22 removes the background area from the parallax image thus obtained, and extracts the foreground area. Any appropriate technique can be used for such separation of the background region and the foreground region.

  In this embodiment, as shown in FIG. 3A, a parallax image generated from an image of the background is stored in the memory 23 in advance. In this example, an image obtained by capturing a state where no person exists (that is, a vacant seat) is used as a background image. An image area 42 indicates a sheet, and an image area 43 indicates a background of the sheet. The parallax image at the time of the vacant seat can be generated by the same method as described above. When the parallax image at the time of vacant seat is compared with the parallax image generated when the occupant is sitting on the seat as shown in FIG. 3B, the parallax value of the region 41 in which the occupant is imaged is Different. Therefore, by extracting pixels having parallax values different from each other by a predetermined value or more, the occupant area 41 can be extracted as a foreground area as shown in FIG.

  The parallax image at the vacant seat changes according to the position and inclination of the seat. A plurality of vacant seat parallax images can be stored in the memory 23 according to the position and inclination of the seat. The vehicle is provided with a sensor (not shown) that detects the position and inclination of the seat. The background removing unit 22 can read the corresponding parallax image at the time of vacant seat from the memory 23 according to the position and inclination of the seat detected by the sensor, and can remove the background using this. By doing so, the foreground region can be more accurately extracted by the background removal unit 22 even when the position and / or inclination of the sheet is changed.

The normalizing unit 24 performs normalization on the passenger area 41. By this normalization, a gray scale value is assigned to each pixel in the occupant area according to the distance from the imaging devices 11 and 12. Normalization is performed according to equation (1).

  Here, when the horizontal (lateral) direction of the captured image is set to the x coordinate and the vertical (vertical) direction is set to the y coordinate, d (x, y) indicates the parallax value at the xy position of the occupant region 41. N is the number of grayscale bits. In one example, N = 9 and 512 gradations with grayscale values from 0 to 511 are realized. 0 is black and 511 is white. d ′ (x, y) indicates a gray scale value calculated for the xy position of the passenger area 41. The parallax maximum value dmax corresponds to the minimum value of the distance from the imaging device, and the parallax minimum value dmin corresponds to the maximum value of the distance from the imaging device.

  As shown in Equation (1), normalization assigns the highest gray scale value (white in this example) to the pixel corresponding to the maximum parallax value dmax, and the smaller the parallax value d, the gray scale Decrease the value gradually. In other words, the highest gray scale value is assigned to the pixel corresponding to the minimum distance value, and the gray scale value is gradually lowered as the distance to the imaging device increases. In this way, a grayscale image in which each pixel has a grayscale value representing the distance to the imaging device is generated from the passenger area 41.

  The detection area extraction unit 25 extracts an area where the head should be detected from the gray scale image. Thereby, since the area to be processed for detecting the head is limited, the head can be detected more accurately and the calculation load can be further reduced. Detailed processing of the detection area extraction unit 25 will be described later.

  The memory 26 (which may be the same memory as the memory 23) stores a head model obtained by modeling a human head. The correlation unit 27 scans the head model with respect to the detection region extracted by the detection region extraction unit 25, and calculates a correlation value between the scanning region and the head model. The scanning area with the largest correlation value is determined as the head area. Thus, the physical position and size of the head can be obtained from the detected position and size of the head region.

  FIG. 4 is a flowchart of processing executed by the detection region extraction unit 25 according to an embodiment of the present invention. This process will be described with reference to FIGS.

  (A) of FIG. 5 shows the gray scale image produced | generated based on the image which image | photographed the state which the passenger | crew is sitting normally, (b) is the state which the passenger | crew is inclining and sitting The gray scale image produced | generated based on the image which imaged is shown. For ease of explanation, a frame of the captured image in which the x and y coordinates are set is shown, but the background area indicated by reference numeral 45 in the captured image has already been removed by the background removal unit 22. Please be careful. In addition, although the passenger area indicated by the reference numeral 46 is shown in white in the figure, it should be noted that this area is actually a grayscale image. The y coordinate of the bottom of the grayscale image 46 is y1, and the y coordinate of the top is y2.

  In step S1 of FIG. 4, reference pixel rows (referred to as reference rows) are set in the upper and lower portions of the grayscale image 46, respectively. In this example, the pixel row 1/5 (ie, (y2−y1) × 4/5) from the top is set as the first reference row La, and 4/5 (ie, (y2−y1) from the top. ) × 1/5) is set as the second reference row Lb. Since these reference rows are rows for determining a center line representing the center of the occupant (head to torso) in the xy plane, preferably, as shown in this embodiment, the upper and lower portions of the grayscale image are displayed. Set to pass through each of the.

  Further, a central position Ma representing a position for equally dividing the number of pixels included in the first reference row La into two is calculated. Here, referring to FIG. 6, as an example, an arrangement of pixels existing in the reference row La is shown. The shaded portion is where the pixel exists. Here, the presence of a pixel preferably indicates that there is a pixel (which can be referred to as an effective pixel) from which an appropriate parallax value is obtained. xa1 and xa2 are coordinates at which the first reference row La crosses the edge of the grayscale image 46. In the example of (a), since the range from xa1 to xa2 is filled with pixels, the x coordinate of the central position Ma is represented by (xa2-xa1) / 2. In (b), as indicated by reference numerals 48 and 49, there is a portion where no effective pixel exists. In this case, the x coordinate of the center position Ma is a position different from (a). By obtaining a position that equally divides the total number of effective pixels of a pixel row into two, the center position can be positioned at a location that represents the center of gravity of the pixel row.

  Similarly, a central position Mb representing a position where the total number of effective pixels existing in the second reference row Lb is equally divided into two is calculated. In the arrangement as shown in FIG. 6A, the x coordinate of the center position Mb is expressed as (xb2-xb1) / 2. 5A and 5B show the center positions Ma and Mb thus specified.

  In step S2, a straight line connecting the center position Ma and the center position Mb is set as the center line CL, and the inclination k of the center line CL with respect to the y axis is calculated. The inclination k can be calculated by (x coordinate of central position Ma−x coordinate of central position Mb) / (y coordinate of central position Ma−y coordinate of central position Mb). The inclination k may be expressed by an angle.

  Alternatively, the center position of a plurality of pixel rows (which may or may not include the above-described reference row) is obtained by the above-described method, the center line CL is determined by the primary approximation method, and the inclination thereof May be calculated. FIG. 7 plots the center positions obtained in such a plurality of pixel rows. The center line CL can be obtained by linearly approximating these center positions by any appropriate method.

  In step S3, it is determined whether or not the absolute value of the inclination k of the center line is smaller than a predetermined value T. Here, FIG. 5A shows a case where the absolute value of the slope k is smaller than the predetermined value T. FIG. 5B shows a case where the absolute value of the slope k is greater than or equal to the predetermined value T. As is apparent from the figure, when the person is sitting normally, the calculated inclination k is smaller than the predetermined value T, and when the person is sitting inclined, the calculated inclination k is equal to or greater than the predetermined value T. Become.

  Alternatively, the inclination of the center line CL with respect to the x axis may be used. In this case, when the inclination is larger than the predetermined value, it is determined that the person is sitting normally, and when the inclination is equal to or less than the predetermined value, it is determined that the person is inclined and sitting.

  If the absolute value of the slope k is smaller than the predetermined value T, the process proceeds to step S4. As shown in FIG. 5C, first and second vertical lines VL1 and VL2 perpendicular to the x-axis are set so as to have the same width A on the left and right with respect to the center position Mb. The gray scale image 46 is cut out by the vertical lines VL1 and VL2, and the detection area 47 is extracted.

  In this example, the width A is set so that “2 × A” is a predetermined ratio (for example, 80%) of the maximum width of the grayscale image 46. Here, the maximum width of the gray scale image 46 is indicated by, for example, reference numeral 51 ((a) of FIG. 5). The maximum width is obtained by scanning each pixel row and obtaining the maximum value of the length between the left end edge and the right end edge of the grayscale image 46.

  Thus, when the inclination k is small, it can be determined that the occupant is normally seated with almost no inclination, so that the region having the same width is centered on the central position Mb below (that is, on the body side) The detection area 47 can be extracted so as to include the head.

  On the other hand, when the absolute value of the slope k is greater than or equal to the predetermined value T in step S3, the process proceeds to step S5. As shown in (d) of FIG. 5, the pixel row at a half point from the top of the grayscale image 46 is specified as the third reference row Lc. Further, as in the case of the first and second reference rows, for the third reference row Lc, a central position C is obtained that equally divides the total number of effective pixels present in the third reference row Lc into two. .

  In step S6, the first and second lines PL1 and PL2 parallel to the center line CL are set so as to have widths A and B on the left and right, respectively, with the center position C as a reference. The gray scale image 46 is cut out by the first parallel line PL1 and the second parallel line PL2, and the detection area 47 is extracted.

  The relative sizes of the left cut-out width A and the right cut-out width B are preferably determined according to the distance between the right and left occupants of the occupant. That is, when the occupant tilts his / her body with respect to the imaging device, one of the occupant's left and right bodies may be closer to the imaging device than the other. A shorter distance is imaged larger, and a longer distance is imaged smaller. Therefore, it is preferable to make the cut-out width of the shorter distance between the left and right body larger than the cut-out width of the longer distance.

  In the example of FIG. 5B, the image is taken from the right direction from the occupant, and the right half of the occupant is closer to the imaging device than the left half. Therefore, A> B is set. The ratio of A and B can be determined in advance. In one embodiment, A + B is set to be 80% of the maximum width of the grayscale image 46 and A: B = 3: 2. When imaging from the left direction from the occupant, A <B is set, and for example, A: B = 2: 3 is set. The maximum width can be determined as described above.

  The numerical values given here are examples of values obtained by the inventor as a value that can limit the detection region without removing the head, and thus are not limited to these numerical values. Please be careful. Alternatively, the left and right widths may be set the same.

  Whether the right half body or the left half body is closer to the imaging apparatus can be determined by an arbitrary method. As described above, the gray scale value of each pixel in the gray scale image represents the distance. Therefore, for example, by comparing the average value of the gray scale values of the pixels included in the area on the right side of the center line with the average value of the gray scale values of the pixels included in the area on the left side of the center line, It can be determined which of the left half is closer to the imaging device.

  Thus, even when the inclination k is large, the detection region 47 can be extracted without removing the head by cutting out with a line parallel to the center line CL. Also, according to whether the left half body or the right half body is inclined with respect to the imaging device, it is possible to extract a detection region that is more suitable for the posture of the person by making the cutout width unequal on the left and right, and the calculation load is reduced. Can be reduced.

  Here, referring to FIG. 8, a case where an example of a person who has one arm on the side of the head is acquired as the grayscale image 46 is schematically shown. As described above, how the detection region 47 is cut out by the parallel lines PL1 and PL2 in the xy plane is shown. If A and B are set larger as described above so that the head is surely included in the detection region 47, the arm region may be included in the detection region as shown in the figure. If the region of the arm (or part thereof) mentioned above is included in the detection region, the head may be erroneously detected in the arm region in the correlation processing described later. In order to avoid this, if A and B are narrowed as indicated by A ′ and B ′, a part of the head is excluded from the detection region as indicated by reference numeral 52 by parallel lines PL1 and PL2. There is a fear.

  In an embodiment of the present invention, in order to avoid such over-removal of the object to be detected (in this example, the head), A and B are widened as described above (eg, 80% of the maximum width). A method for removing unnecessary areas (in this example, arm areas) that may be included in the detection area as a result is further proposed. That is, the process shown in FIG. 9 is performed on the detection area 47 extracted as described above in the xy plane, and the detection area 47 is further limited to the distance direction (z direction).

  The z-direction limiting process will be described with reference to FIG. In order to clarify the intention of this method, an image example different from that in FIG. 5 is used in FIG. 10A, and a gray scale image 46 of a person with both arms on both sides of the head is schematically shown. Has been shown. In this example, since the gradient k is greater than or equal to a predetermined value, the grayscale image 46 is cut out by the parallel lines PL1 and PL2. The widths A and B are set to 3: 2, and A + B is 80% of the maximum width of the grayscale image 46.

  For each pixel row, the intersections of the parallel lines PL1 and PL2 and the pixel row are XB and XA, respectively. A line QL1 is set to pass through the center between the center line CL and the parallel line PL1, and a line QL2 is set to pass through the center between the center line CL and the parallel line PL2. Intersections between the respective pixel rows and the lines QL1 and QL2 are Xb and Xa, respectively. In the figure, XB, Xb, Xa and XA are shown for a pixel row passing through a point M on the center line CL.

  Each pixel in the grayscale image 46 has a grayscale value representing a distance (hereinafter also referred to as a distance value). The distance values of the pixels included in the first reference row La where the central position Ma exists are averaged, and the average distance value is set as a distance value corresponding to the central position Ma. Similarly, the distance values of the pixels included in the second reference row Lb where the center position Mb exists are averaged, and the average distance value is set as a distance value corresponding to the center position Mb. As shown in FIG. 10B, the y-coordinates of the central positions Ma and Mb and the distance values (represented by the z-coordinate) corresponding to Ma and Mb are plotted on the yz plane. Ma and Mb are connected to determine the center line CD (step S11 in FIG. 9). The center line CD can be expressed by a linear expression.

  In step S12, the pixel rows are scanned from the top to the bottom of the grayscale image 46. In step S13, for each pixel row, a reference distance value D for the pixel row is calculated from the center line CD. Specifically, the reference distance value D of the pixel row can be calculated by substituting the y coordinate value of the pixel row into a linear expression representing the center line CD. In the figure, a reference distance value D for a pixel row passing through the point M is shown.

  In step S14, pixels having distance values outside the first predetermined range (D ± z1) between (XB to Xb) and (Xa to XA) of each pixel row are removed from the pixel row. (First removal). The first predetermined range is set in consideration of the thickness of the object in the region to be left by the first removal.

  FIG. 10B shows a first predetermined range calculated for the point M. A line DL2 parallel to the center line CD is set so as to pass through the point D-z1, and a line DL1 parallel to the center line CD is set so as to pass through the point D + z1. FIG. 10C shows the lines DL1 and DL2 on the xz plane. In short, the area to be removed is viewed from above. As can be seen from these figures, in the range of XB to Xb in the x direction, the pixels surrounded by the lines DL1, DL2, and PL1 remain, and the other pixels are removed. Similarly, in the range of Xa to XA in the x direction, the pixels surrounded by the lines DL1, DL2, and PL2 are left, and the other pixels are removed.

  The technical significance of this first removal will be described. As described above, the center line CL is considered to be a line representing the center of a person (head to torso) on the xy plane. The area between the lines QL1 and QL2 corresponds to the central part of the person. On the other hand, the area outside the lines QL1 and QL2 is an area including the arm as shown in FIG. One of the purposes of the first removal is to remove such an arm region from the detection region 47 as described with reference to FIG.

  In order to achieve this object, first, the regions outside QL1 and QL2 are removed while preserving the regions QL1 to QL2 representing the central portions of the head and the trunk. Since the first predetermined range (D ± z1) is a range reflecting the thickness of the head to the torso, pixels having distance values that do not fit in the range of the head to the torso are removed by the first removal. The Usually, when an arm is raised, the raised arm often comes to the front of the head, but such an area of the arm can be removed. In this example, the first predetermined range D ± z1 is 50 cm in consideration of the thickness of the trunk. Thus, the region corresponding to the arm as shown in FIG. 10A can be removed from the detection region 47.

  Returning to FIG. 9, in step S15, pixels having distance values outside the second predetermined range (D ± z2) are removed between Xb to Xa of the respective pixel rows (second removal). Referring to FIG. 10C, the second predetermined range D ± z2 is indicated by lines DL3 and DL4. The pixels surrounded by QL1, QL2, DL3 and DL4 are left, and the other pixels are removed.

  The significance of this second removal will be described. If the occupant has an accessory (baggage) such as a bag or a ball, the baggage area remains in the detection area 47 as shown by the hatched area 53 in FIG. It is. If such a luggage area remains in the detection area 47, the head may be erroneously detected in the luggage area in the correlation processing described later. Therefore, it is preferable to remove such a region other than the body from the detection region 47.

  Therefore, removal using the threshold value z2 is performed between QL1 and QL2. In order to avoid the body part from being removed, the threshold value z2 is preferably set to a value larger than the threshold value z1. Moreover, it is preferable to determine the threshold value z2 according to the assumed size of the luggage. For example, assuming that a baggage having a thickness of about 30 cm is present, D ± z2 is set to be approximately 30 cm larger than D ± z1.

  Returning to FIG. 9, the z-direction limiting process using z1 and z2 as described above is executed for each pixel row until the scan reaches the bottom row of the grayscale image 46 (S16).

  Alternatively, the distance value of the pixel at the center position Ma and the distance value of the pixel at Mb may be used as the distance values (z coordinate values) corresponding to the center positions Ma and Mb plotted on the yz plane.

  In an alternative embodiment, the center line CD may be obtained by a primary approximation method by plotting distance values for a plurality of pixel rows in the same manner as the center line CL. In this case, as described above, the distance value may be the average of the distance values of the pixels included in the corresponding pixel row, or may be the distance value of the pixel at the center position of the pixel row.

  In this embodiment, in the first removal, the width of the line DL1 and the width of the DL2 with respect to the center line CD are the same z1. When plotting on the yz plane with an average distance value obtained by averaging the distance values of the pixels included in the reference rows La and Lb, the same width may be used. However, when the distance values of the pixels at the center positions Ma and Mb of the pixel row are plotted on the yz plane, it is preferable to set the former to the latter so that DL1 to CD: CD to DL2 is, for example, 3: 2. It is preferable to make it larger.

  The pixel distance values at the central positions Ma and Mb represent the distance to the head surface and the distance to the body surface, respectively. On the other hand, the average of the distance values of the pixels included in the pixel rows Ma and Mb (reference rows La and Lb) is a value in consideration of the thickness of the head and the trunk. That is, the former distance value is smaller than the latter average distance value. Since the width between DL1 and DL2 is preferably set to reflect the thickness of the head and torso, DL1 and DL2 as shown in FIG. 10B are shifted in the z direction with respect to the center line CD. The width of CD to DL1 is preferably greater than the width of CD to DL2. An example of this is shown in FIG. 12, where z3: z4 = 3: 2.

  FIG. 10 (d) shows an alternative setting for lines DL1 and DL2. In this alternative setting, z1 changes according to the x direction. The human torso tends to become thicker toward the center. Therefore, the width between the center line CD and the lines DL1 and DL2 increases as the x value increases between XB and Xb, and the center line CD increases as the x value increases between Xa and XA. The width between the lines DL1 and DL2 is reduced.

  As is apparent with reference to FIGS. 10C and 10D, the limitation in the z direction is based on the fact that the thickness increases toward the center of the body. Therefore, it is not necessarily limited to z1 setting like (c) and (d), For example, you may set z1 so that a convex curve may be drawn from the intersection of DL1 and QL toward the intersection of DL2 and QL1. . However, forms such as (c) and (d) can realize high-speed calculation processing because DL1 and DL2 are straight lines.

  It should be noted that the detection region 47 extracted when the inclination k is equal to or less than the predetermined value T can also be limited to the z direction by the method shown in FIG. In this case, vertical lines VL1 and VL2 are used instead of parallel lines PL1 and PL2.

  As described above, when a person holds a luggage or the like as shown in FIG. 11, the area is removed from the detection area 47. As a result, a recess is formed in the detection region 47.

  If such a recess is formed in the detection region 47, for example, the head may be erroneously detected in the correlation process described later, for example, around the recess. In order to avoid this, in the embodiment of the present invention, the process shown in FIG. 13 is further executed, and if the obtained detection region 47 has a recess, it is filled. This process will be described with reference to FIG.

  FIG. 14 shows an example in which a recess 54 is present in a detection region 47 cut out by parallel lines PL1 and PL2 and limited to the z direction using the distance value. No pixel is present in the recess 54.

  In step S21 of FIG. 13, the distance values of the pixels included in the detection region 47 are averaged to calculate an average distance value.

  In step S22, the lowest point, that is, the lowest point in the y direction is obtained in the region below the reference line Lc. The lowest point is obtained in each of the region between the center line CL and the parallel line PL1 and the region between the center line CL and the parallel line PL2, and these are Ta and Tb.

  In step S23, the midpoint Tm of the line segment connecting the two lowest points Ta and Tb is obtained. Further, a value obtained by adding α to the y coordinate Tmy of the midpoint Tm is obtained. For example, α is set to 1/3 of the y coordinate value of the reference line Lc. A point Tα at a height of Tmy + α from the midpoint is obtained. Alternatively, a normal line with respect to the line segment may be drawn from the middle point, and the point where the length of the normal line is Tmy + α may be set as the point Tα. It is determined whether or not a pixel exists on a line segment connecting the middle point Tm and the point Tα.

  In the example of FIG. 14, this determination is No, and no pixel exists. This indicates that a recess 54 is formed in the lower portion of the detection region 47. If this determination is Yes, it indicates that the lower portion of the detection area 47 is closed.

  If the determination in step S23 is No, in step S24, the points Ta and Tb are connected by a line segment having a width t (a width corresponding to several pixels) to close the lower portion of the detection region 47 and form a “hole”. Form. Here, the line segment is configured to include pixels having the average distance value calculated in step S21 described above. If judgment of Step S23 is Yes, it will progress to Step S25.

  Alternatively, by forcing several pixel rows from the bottom of the detection region 47 (eg, 1/10 of the number of pixel rows between the bottom and top of the detection region) with pixels having an average distance value, A hole may be formed.

  In step S25, it is checked whether or not a “hole” exists in the region below the reference line Lc of the detection region 47 thus closed. The detection of “hole” can be realized by any appropriate technique. For example, a hole can be detected by obtaining a contour line in a region below the reference line Lc by a method of extracting a contour line by eight connections. Since such a hole detection method is well known, a detailed description is omitted. Briefly, the region is scanned to determine the start point of the contour line (boundary line). At the start point, the neighboring 8 pixel positions are searched counterclockwise to find the next pixel. By proceeding one after another, one contour line can be traced.

  In step S26, it is determined whether a hole has been detected. If a hole is detected, in step 27, the hole is filled with pixels having the above average distance value.

  The hole includes not only a hole formed by closing the concave portion as described above, but also a hole formed by removing the load region when a load or the like is present in the center part of the trunk. In addition, when the pattern light described above is strong and a part of the object is white (for example, a white part of clothes), a case where the parallax value cannot be calculated may occur. In a portion where the parallax value cannot be obtained, there is a case where a pixel having a distance value (that is, the above effective pixel) does not exist and is left as a hole. Such holes are filled with pixels of average distance values.

  Other methods may be used as a method for detecting the recess. For example, a well-known packaging method or Graham method for obtaining a convex hull can be used. Preferably, an edge of a region below the reference line Lc is extracted, and a convex hull is obtained for the edge. By obtaining the convex hull, a polygon circumscribing the region is obtained, and therefore it is not necessary to close the lower part of the detection region with a line segment as described above. Hole detection may be performed inside the obtained polygon.

  In step S28, any appropriate particle analysis process is performed. For example, expansion and contraction processing can be performed. The expansion and contraction processing is a well-known method and will not be described in detail. When the expansion process is performed, for example, the image is expanded by one pixel with a pixel having the maximum distance value among the distance values of eight neighboring pixels. In the contraction process, for example, the boundary pixel of the image is replaced with a pixel having a minimum distance value among the distance values of eight neighboring pixels. By repeating this, noise can be removed. After performing the particle analysis process, a region having the maximum area is extracted as a final detection region 47. For example, even if pixels that are noise remain outside the detection area, the area having the largest area is extracted, so that only the area including the head and the torso is extracted as the detection area 47.

  It should be noted that the detection region 47 extracted when the inclination k is equal to or less than the predetermined value T can be detected and filled with the technique shown in FIG. In this case, vertical lines VL1 and VL2 are used instead of parallel lines PL1 and PL2.

  Next, processing by the correlation unit 27 in FIG. 1 will be described.

  First, a method for generating a head model to be stored in the memory 26 will be described with reference to FIG. As shown in FIGS. 15A and 15B, the human head has a feature similar to the shape of an elliptical sphere. Therefore, the human head can be represented by an elliptic sphere.

  The elliptic sphere is constructed based on the space coordinates in the vehicle. The spatial coordinates in the vehicle have, for example, the positions of the imaging devices 11 and 12 as the origin. The z-axis extends vertically from the imaging device, and the z value indicates the distance from the imaging device. The x-axis is set perpendicular to the z-axis in the vehicle width direction, and the y-axis is set perpendicular to the z-axis in the vehicle height direction.

FIG. 15C is an example of an elliptic sphere representing a human head, and the elliptic sphere has center coordinates O (X _0, Y _0, Z ₀ ). As the center coordinates, a coordinate point that can exist when the occupant is sitting on the seat is selected. a, b, and c are parameters that define the size of the elliptic sphere, and are set to appropriate values to represent the human head.

  The elliptic sphere as a three-dimensional model of the head is converted into a two-dimensional image. Since the elliptical sphere has an elliptical shape when viewed from the z-axis direction as indicated by an arrow 55, the two-dimensional image is generated to have an elliptical shape. Further, the two-dimensional image is generated such that each xy position has a gray scale value representing a z value corresponding to the xy position.

Here, a method for generating a two-dimensional image model will be described in more detail. Since the elliptic sphere is represented by Expression (2), the z value at each xy position of the elliptic sphere is represented by Expression (3).

Next, as indicated by the arrow 55, the elliptic sphere is projected onto the two-dimensional image in the z-axis direction. This projection is realized by equation (4). Here, R11 to R33 are rotation determinants. When the ellipsoidal sphere is not rotated, a value 1 is set for R11 and R22, and zero is set for the other parameters. When the elliptic sphere is rotated, values representing the rotation angle are set to these values. fx, fy, u0, and v0 indicate internal parameters of the imaging apparatus, and include parameters for correcting lens distortion and the like, for example.

By the projection according to the equation (4), each point (x, y, z) on the surface of the ellipsoid sphere is projected onto (x, y) of the two-dimensional image. The gray scale value I representing the z value of the projection source (x, y, z) is associated with the coordinates (x, y) of the two-dimensional image. The gray scale value I is calculated according to equation (5).

  Here, Zmin indicates a value closest to the imaging device among the z values of the coordinates of the surface of the ellipsoid (or the ellipsoid after rotation when rotated). N is the same as N used in normalization by the normalization unit 24. According to Equation (5), when the z value is Zmin, the highest gray scale value is assigned to the pixel corresponding to the z value. As the distance of the z value from Zmin increases, that is, as the distance to the imaging device increases, the gray scale value of the pixel corresponding to the z value gradually decreases.

  FIG. 15D schematically shows a two-dimensional image of the head model created in this way. The gray scale value gradually decreases from the center of the ellipse toward the periphery. For example, a point 56 shown in (c) of FIG. 15 is projected onto a point 57 of (d) in FIG. Since the point 56 has a z value of Zmin, the gray scale value of the point 57 has the highest value (in this example, white). Further, although the point 58 is projected onto the point 59, since the z value of the point 58 is larger than Zmin, the gray scale value of the point 59 is lower than the gray scale value of the point 57.

  Thus, the elliptical shape when the head is viewed from the z direction as shown in FIG. 15A is reflected in the shape of the two-dimensional image model shown in FIG. 15D, as shown in FIG. The distance from the imaging device in each part of the head is reflected in the gray scale value of the corresponding pixel in the two-dimensional image model shown in FIG. As described above, the human head is represented by three-dimensional data, but the human head can be modeled into a two-dimensional image by expressing the distance in the z direction as a gray scale value. The generated two-dimensional image model is stored in the memory 26.

  Several types of head models are preferably provided. For example, since an adult's head and a child's head are different in size, separate head models are used. By adjusting the above a, b, and c, an elliptical sphere model having a desired size can be generated. The size of the ellipsoidal sphere model to be used may be determined based on a result of statistical examination of a plurality of human heads. The elliptical sphere model generated in this way is converted into a two-dimensional image model by the above-described method and stored in the memory 26.

  It is preferable to prepare a plurality of head models according to the posture of the occupant. For example, in order to detect the head of an occupant with a tilted head, head models having different inclinations can be prepared. For example, an ellipsoidal model rotated by a desired angle can be generated by adjusting the values of the parameters R11 to R33 in the rotation matrix shown in the above equation (4). Similarly, the elliptic sphere model is converted into a two-dimensional image model and stored in the memory 26.

  FIG. 16 shows a head model of a two-dimensional image stored in the memory 26 in one embodiment. There are three sizes, and the larger the size is on the right side of the figure. Furthermore, a model without inclination, a model rotated by π / 4, and a model rotated by 3π / 4 are prepared for each size.

  The correlation unit 27 reads the head model of the two-dimensional image from the memory 26 and scans the detection region 47 extracted by the detection region extraction unit 25 with the head model. The correlation unit 27 performs matching between the head model and the scanned image area in the detection area, and calculates a correlation value. Any matching technique can be used.

In this embodiment, the correlation value r is performed according to the normalized correlation equation (6) that takes into account the deviation of the normalization factor, position and orientation errors, and the like. Here, S indicates the size of the matching block. The block size may be, for example, one pixel, or a group of a plurality of pixels (for example, 7 × 7 pixels) may be a single block. The correlation value r having a higher value is calculated as the similarity between the head model and the scanning target region is higher.

  As described above, in the head portion of the detection region 47, the grayscale value is set so that the distance from the imaging device toward the periphery gradually increases from the point near the imaging device of the head, and the value gradually decreases accordingly. Assigned. On the other hand, the head model is also assigned a grayscale value so that the distance from the imaging device gradually increases from the center toward the periphery, and the value gradually decreases accordingly. Therefore, if the head model is correlated with the head portion of the occupant region, a correlation value r having a higher value is calculated than when correlated with other portions. Thus, the head portion can be detected from the detection region 47 based on the correlation value r.

  It should be noted that both the normalization of the foreground region represented by equation (1) and the normalization of the head model represented by equation (5) are assigned gray scale values so that the values become lower as the distance to the imaging device increases. The point is the same, but strictly speaking, there is a gap between the normalization of the two. For example, in equation (2), the gray scale value is minimized at the minimum parallax value dmin, but in equation (5), the gray scale value is minimized when the z value becomes infinite. However, such a shift in scaling can be compensated by the correlation as described above. Also, positional errors such as a shift in the center coordinates when constructing the ellipsoidal sphere model of the head can be compensated by such correlation. That is, correlation determines whether the trend of changing grayscale is similar, so that the correlated image area has a trend of grayscale change similar to the head model (ie, from the center to the periphery). If the gray scale value gradually decreases), a high correlation value is output.

  The image area where the highest correlation value r is calculated is detected from the detection area 47, and the position of the human head can be specified based on the position of the detected image area in the captured image and the gray scale value. Further, the size and inclination (posture) of the head can be determined based on the correlated head model.

  When there are a plurality of head models, the correlation processing as described above is performed for each head model of the two-dimensional image stored in the memory 26. The head model for which the highest correlation value r is calculated is specified, and the image area correlated with the model is specified in the detection area. From the position of the detected image area in the captured image and the gray scale value, the position of the human head can be specified. Further, the size and inclination (posture) of the head can be determined based on the head model used for calculating the highest correlation value r.

  Such gray-scale matching of a two-dimensional image can be performed at a higher speed than the conventional identification processing in a three-dimensional space.

  (A) and (b) of FIG. 17 show the head region 60 specified by the above correlation processing in the detection region 47 extracted based on the image example shown in FIG.

  In another embodiment, the object detection device 1 uses a distance measurement device 70 instead of using a light source and a pair of imaging devices. FIG. 18 shows an object detection apparatus 1 according to this embodiment. In one example, the distance measuring device 70 is a TOF (time of flight) type distance image sensor. The distance image sensor has image sensors arranged in two dimensions. The distance image sensor uses an LED (light emitting diode) to emit infrared light, for example, to irradiate the object, and receives reflected light from the object through the lens. In each image sensor, the phase difference (time delay) between the irradiation light and the reception light is obtained, and the distance from the distance measuring device 70 to the object is calculated for each pixel based on the phase difference. Thus, a distance image is generated.

  The background removal unit 72 of the processing device 15 removes the background area from the distance image and extracts the foreground area. As described above, this is realized by any appropriate method. For example, the distance image measured and generated by the distance measuring device 70 when the seat is vacant is stored in the memory 73 in advance, and the foreground region is obtained by comparing the vacant seat distance image with the distance image including the occupant captured this time. The background can be removed to extract.

  The normalizing unit 74 normalizes the foreground area. Normalization can be calculated by replacing dmin in equation (1) with the maximum distance value Lmax in the foreground area and dmax with the minimum distance value Lmin in the foreground area.

  The detection area extraction unit 75 extracts a detection area where the head should be detected from the foreground area. The detection area can be set using a method similar to that of the detection area extraction unit 25 in FIG.

  The memory 76 (which may be the same as the memory 73) stores a two-dimensional image of the head model, similar to the memory 26. The head model is generated by the method as described above. Here, the imaging device in FIG. 15C can be considered as a distance measuring device. The correlation unit 76 can perform correlation by the same method as the correlation unit 26.

  FIG. 19 shows a flowchart of processing for detecting an object. This process is performed at predetermined time intervals. In one embodiment, the processing is realized by the processing device 15 shown in FIG.

  In step S31, an image including a human head is acquired by a pair of imaging devices. In step S32, a parallax image is generated from the acquired image. In step S33, the background is removed from the parallax image, and the foreground area is extracted. After removing the background, particle analysis processing such as expansion and contraction and smoothing processing (for example, using a median filter) may be performed to remove noise.

  In step S34, the foreground area is normalized according to the equation (1). In step S35, a detection area is extracted from the normalized foreground area. In one embodiment, the processes of FIGS. 4, 9 and 13 are performed in step S35. In step S36, a pre-generated head model is read from the memory. In step S37, the head model is scanned with respect to the detection region, and a correlation value indicating the similarity between the head model and the image region where the head model overlaps is calculated. In step S38, the image area where the highest correlation value is calculated is stored in the memory in association with the correlation value and the head model.

  In step S39, it is determined whether there is a head model to be correlated. If this determination is Yes, the processing from step S36 is repeated.

  In step S40, the highest correlation value is selected from the correlation values stored in the memory, and the position and size of the human head are determined based on the image region and the head model corresponding to the correlation value. Output.

  When using a distance measuring device instead of the imaging device, a distance image is generated using the distance measuring device in steps S31 and S32.

  In the embodiment described above, the correlation between the two-dimensional image model and the image region is performed with respect to the tendency that the gray scale value decreases as the distance to the imaging device increases. A tendency different from the tendency of the gray scale value may be used. For example, similarity may be determined for the tendency to increase the gray scale value as the distance to the imaging device increases.

  Although the form for detecting the human head has been described, the object detection apparatus of the present invention can also be applied to a form for detecting other objects. For example, an object different from a human can be detected by generating a two-dimensional image model that has a characteristic shape viewed from a predetermined direction and represents a distance in the predetermined direction as a gray scale value. it can.

  Further, although the embodiment in the case where the object detection device is mounted on a vehicle has been described, the object detection device of the present invention can be applied to various forms. For example, the present invention can also be applied to a form in which a certain object (for example, a human head) is detected and some action is performed (for example, a message is issued) in response to the detection.

The block diagram of the apparatus which detects an object according to one Example of this invention. The figure explaining the method of calculating the parallax according to one Example of this invention. The figure explaining the removal of a background according to one Example of this invention. The flowchart of the process which extracts a detection area | region according to one Example of this invention. The figure for demonstrating the extraction method of a detection area according to one Example of this invention. The figure for demonstrating the method of calculating | requiring a center position according to one Example of this invention. The figure for demonstrating the other method of calculating | requiring a centerline according to one Example of this invention. The figure for demonstrating the technical significance which further limits a detection area to a distance direction according to one Example of this invention. The flowchart of the process which further limits a detection area to a distance direction according to one Example of this invention. The figure for demonstrating limitation of the distance direction of a detection area according to one Example of this invention. The figure for demonstrating the other technical significance which further limits a detection area to a distance direction according to one Example of this invention. The figure which shows the other setting of the threshold value which limits a detection area to a distance direction according to one Example of this invention. The flowchart of the process which detects the recessed part and hole in a detection area | region according to one Example of this invention. The figure for demonstrating the process which detects the recessed part in a detection area | region according to one Example of this invention. The figure explaining the head model according to one Example of this invention. The figure which shows an example of several types of head models according to one Example of this invention. The figure which shows an example of the detected head area | region according to one Example of this invention. The block diagram of the apparatus which detects an object according to the other Example of this invention. The flowchart of the process which detects an object according to one Example of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Object detection apparatus 23,26 Memory 12,13 Imaging device 15 Processing apparatus 70 Distance measurement apparatus

Claims (22)

An apparatus for detecting an object,
Foreground acquisition means for acquiring a foreground image area by removing a background from an image including a three-dimensional object as an object;
An inclination calculating means for calculating an inclination of a center line of the foreground image area with respect to a predetermined reference axis;
Area extracting means for extracting a detection area from the foreground image area based on the inclination;
Object detection means for detecting the object from the detection region;
An object detection apparatus comprising: Further, in the foreground image area, center positions for respectively obtaining first and second center positions representing positions at which the number of pixels existing in the pixel rows serving as the first and second reference lines are equally divided into two. A calculation means;
Center line calculating means for obtaining a line connecting the first and second center positions as the center line;
The object detection apparatus according to claim 1. When the inclination is equal to or greater than a predetermined value, the area extraction unit extracts the detection area by cutting out the foreground image area by a line parallel to the center line;
The object detection apparatus according to claim 1 or 2. The parallel lines are set to have a predetermined width on the left and right from the center line,
The object detection apparatus according to claim 3. In addition, for the object in the image, provided with a distance acquisition means for acquiring the distance to the object,
When the inclination is equal to or greater than the predetermined value, the region extracting unit calculates a relative size of the left-side width and the right-side width among the left and right widths to the left image with respect to the center line. Set based on the relative size of the distance about the object included in the area and the distance about the object in the image area on the right side of the center line;
The object detection apparatus according to claim 4. If the distance to the object of the left image area is shorter than the distance to the object of the right image area, the area extraction means makes the left width larger than the right width, If the distance for the object in the right image area is shorter than the distance for the object in the left image area, the width in the right direction is made larger than the width in the left direction.
The object detection apparatus according to claim 5. When the inclination is smaller than a predetermined value, the area extracting means cuts out the foreground image area with a vertical line set so as to have the same width to the left and right from one of the first and second center positions. To extract the detection region,
The object detection apparatus according to claim 2. The left width and the right width are set such that the sum of the left width and the right width is a predetermined ratio with respect to the maximum width of the foreground image area.
The object detection apparatus according to claim 4. further,
Distance acquisition means for acquiring, for each pixel, a distance value to the object for the object in the image;
Second center line calculating means for calculating a second center line representing a reference distance value of each pixel row based on the distance value of the pixels on the center line;
For each pixel row in the detection area, the reference distance value is obtained from the second center line, and pixels having a distance value outside the first predetermined range with respect to the reference distance value are extracted from the pixel row. First removing means for removing,
The removal by the first removing means is performed for a predetermined range in the horizontal direction.
The object detection apparatus according to claim 1. The second center line calculating means is configured to average the distance value of the pixels included in the first reference row and the distance value averaged of the distance values of the pixels included in the second reference row. And calculating the second center line based on
The object detection apparatus according to claim 9. The second center line calculation means calculates the second center line based on the distance value of the pixel at the first center position and the distance value of the pixel at the second center position. calculate,
The object detection apparatus according to claim 9. The first predetermined range is set to become smaller as the distance from the center line in the horizontal predetermined range is increased.
The object detection device according to claim 9. The first predetermined range is set to have a first width in the direction in which the distance value increases and a second width in the direction in which the distance value decreases with respect to the second center line. The width is greater than the second width;
The object detection apparatus according to claim 9. The predetermined range in the horizontal direction is set outside a predetermined central region of the detection region;
The object detection device according to claim 9. further,
In the predetermined central region, for each pixel row in the detection region, a pixel having a distance value outside a second predetermined range with respect to the reference distance value of the pixel row is removed from the pixel row. 2 removal means,
The object detection apparatus according to claim 14. The second predetermined range is set to be larger than the first predetermined range;
The object detection apparatus according to claim 15. further,
A recess detection means for detecting whether or not a recess is formed in a predetermined lower region of the detection region;
If the recess is detected, means for closing the recess with a line consisting of pixels;
Means for detecting whether there is a hole in the detection region;
Means for filling the hole with pixels having a predetermined distance value if the hole is detected;
The object detection device according to claim 1, comprising: The recess detecting means further includes:
Means for determining the lowest point in the vertical direction in each of the regions obtained by dividing the detection region by the center line;
Means for determining whether or not a pixel exists in a region between a midpoint of a line connecting the two obtained lowest points and a point located at a predetermined height from the midpoint;
The object detection device according to claim 17, comprising: The predetermined distance value is an average distance value obtained by averaging distance values of pixels included in the detection area.
The object detection device according to claim 17 or 18. Further, for the object in the image, distance acquisition means for acquiring a distance value to the object;
For each area obtained by subdividing the foreground image area, a grayscale value representing the distance value is calculated, and a grayscale image having the grayscale value corresponding to each area is generated.
The region extraction means extracts the detection region from the grayscale image,
Model storage means for storing a model obtained by modeling the three-dimensional object;
Means for calculating a correlation value representing the degree of similarity between the model and the image region in the detection region;
The object detection means detects the three-dimensional object by detecting an image area having the highest correlation value with the model in the detection area.
The object detection apparatus according to claim 1. The three-dimensional object is a human head;
The object detection apparatus according to claim 1. The detected human head is a human head riding in a vehicle.
The object detection apparatus according to claim 21.

Images