JP6273156B2 - Pedestrian recognition device - Google Patents

Description

  The present invention relates to a pedestrian recognition device that specifies which specific object corresponds to an object existing in a detection region.

  Conventionally, a specific object such as a vehicle positioned in front of the host vehicle is detected, and a collision with a preceding vehicle is avoided (collision avoidance control), or the distance between the preceding vehicle and the preceding vehicle is controlled to be a safe distance ( (Cruise Control) technology is known. Such a forward monitoring technology for the host vehicle is expected to be effective not only for the preceding vehicle but also for avoiding and reducing contact accidents with pedestrians.

  As a pedestrian detection technique, for example, a road surface in an image is specified using a distance image generated by stereo image processing, and a pedestrian is determined based on the shape of a block group of an image located at a predetermined height from the road. A technique for identifying the above has been disclosed (for example, Patent Document 1). In this technology, the block group is divided into an upper part and a lower part in the height direction, and the detection accuracy of the pedestrian is improved by determining whether the edge of the lower image is similar to the shape of the lower body of a person. .

  In addition, when detecting an object using a distance image generated by stereo image processing, a technique has been proposed in which a plurality of conditions for specifying an object are provided in advance according to the type of object assumed as a detection target. (For example, patent document 2). Furthermore, the process which detects an illuminant of the external field of the own vehicle, each edge from a visible image and an infrared image, evaluates the difference between edges, and detects a pedestrian is disclosed (for example, patent document 3).

JP 2013-109720 A JP 2008-45974 A JP 2007-259076 A

  When the vehicle is traveling at high speed, it takes a short time to approach the pedestrian, even if the pedestrian is located far in front of the vehicle. Therefore, it is necessary to start braking control and the like. However, if the pedestrian is located far away, the image of the pedestrian acquired from the imaging device or the like is too small, and thus the shape of the pedestrian is often not similar even if the edge of the image is evaluated. Therefore, in the conventional technology, it is difficult to detect the pedestrian at an early stage because the detection accuracy cannot be increased for a pedestrian far away.

  In view of such a problem, an object of the present invention is to provide a pedestrian recognition device capable of detecting an obstacle located far in front of a vehicle with high accuracy at an early stage.

In order to solve the above-described problem, a pedestrian recognition apparatus according to the present invention includes a three-dimensional position deriving unit that derives a three-dimensional position in a real space of a plurality of target parts in a captured image obtained by capturing a detection region, Grouping target parts whose differences are within a predetermined range to specify a partial image specifying unit for specifying a partial image including the target object in the captured image, and a partial image based on the luminance and color difference of the constituent parts constituting the partial image Is derived as a target average, and a plurality of components adjacent to the partial image are used as reference images, and the reference image is determined based on the luminance or color difference of the components constituting the reference image. At least one of the luminance average value or the color difference average value is derived as the background average, and the back of the components constituting the partial image using the target average and the background average. The number of groups and the target background specifying unit configured to specify at least one of the constituent parts showing the component parts and the object, at least one of the component part showing the component parts and the object indicating the background in the partial image is horizontally continuous shows the Or a pedestrian determination unit that determines whether or not the object is a pedestrian based on the distribution of one of the numbers .
The target background specifying unit may divide and normalize the partial image into a plurality of blocks and use the blocks as constituent parts.
In order to solve the above-described problem, another pedestrian recognition apparatus according to the present invention includes a three-dimensional position deriving unit that derives a three-dimensional position in a real space of a plurality of target parts in a captured image obtained by imaging a detection region, and a three-dimensional Grouping target parts whose positional differences are within a predetermined range, and specifying a partial image specifying unit for specifying a partial image including the target object in the captured image, and a plurality of blocks obtained by dividing and normalizing the partial image, As a component constituting the partial image, at least one of the luminance average value or the color difference average value of the partial image is derived as a target average based on the luminance and color difference of the block, and a plurality of blocks adjacent to the partial image are used as reference images. Based on the luminance or color difference of the blocks constituting the reference image, at least one of the average luminance value or the average color difference value of the reference image is derived as the background average. A target background specifying unit for specifying at least one of a block indicating a background and a block indicating an object among blocks constituting the partial image using the target average and the background average, and a block and an object indicating the background in the partial image are indicated. Based on the distribution of at least one of the blocks, the distribution of blocks with a low vertical position is weighted more than the distribution of blocks with a high vertical position to determine whether the object is a pedestrian or not. And a pedestrian determination unit for determining.

  The target background specifying unit selects a pixel constituting the partial image and a pixel constituting the reference image from a plurality of pixels having the same vertical position in the captured image and arranged in the horizontal direction. A component that shows the background of the component part that constitutes the partial image by deriving a threshold value using the target average and the background average derived based on this, and comparing the luminance or color difference of each component part that constitutes the partial image with the threshold value Or a constituent part indicating the object may be specified.

  The pedestrian determination unit determines whether or not the target object is a pedestrian based on the contour shape of the target object when the target object included in the partial image is at a position less than a predetermined distance from the acquisition position of the captured image. When the target object included in the partial image is at least a predetermined distance from the acquisition position of the captured image, the target object walks based on the distribution of at least one of the constituent part indicating the background and the constituent part indicating the target object in the partial image. It may be determined whether the person is a person.

  ADVANTAGE OF THE INVENTION According to this invention, it becomes possible to detect the pedestrian located far ahead of a vehicle with high precision early.

It is the block diagram which showed the connection relation of the environment recognition system. It is the functional block diagram which showed the schematic function of the external environment recognition apparatus. It is explanatory drawing for demonstrating a color image and a distance image. It is a 1st figure for demonstrating a pedestrian specific process. It is a 2nd figure for demonstrating a pedestrian specific process. It is a flowchart for demonstrating the flow of a pedestrian specific process.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The dimensions, materials, and other specific numerical values shown in the embodiment are merely examples for facilitating understanding of the invention, and do not limit the present invention unless otherwise specified. In the present specification and drawings, elements having substantially the same function and configuration are denoted by the same reference numerals, and redundant description is omitted, and elements not directly related to the present invention are not illustrated. To do.

  In recent years, an in-vehicle camera mounted on a vehicle images a road environment ahead of the host vehicle, identifies an object such as a preceding vehicle based on color information and position information in the image, and collides with the identified object. Vehicles equipped with a so-called anti-collision function, which avoids the above-mentioned problem and keeps the inter-vehicle distance from the preceding vehicle at a safe distance (ACC: Adaptive Cruise Control), are becoming popular. With such a collision prevention function, various obstacles ahead of the host vehicle are identified, and avoidance control is appropriately performed depending on whether the object is a stationary object or a moving object. Yes. Vehicles and pedestrians are mainly assumed as objects with movement, but in particular, pedestrians may suddenly appear in various directions in the detection area, and their absolute size is also compared with that of vehicles. Therefore, there is a problem that it is difficult to specify or it takes time to specify.

  For example, even if technology that identifies an object as a pedestrian by performing pattern matching with the appearance of the object is used, multiple frames can be used to avoid malfunction of the vehicle due to erroneous recognition that the object is a pedestrian. It must be identified as a pedestrian when the appearance matches. Therefore, it takes a long time to specify the pedestrian. Therefore, in the present embodiment, not only the appearance information related to the appearance of the target object but also the color information related to the color is used, and the pedestrian located far away in front of the vehicle is determined early by determining the pedestrian-likeness at an early stage. And it aims at detecting with high precision. Hereinafter, an environment recognition system for achieving such an object will be described, and a vehicle exterior environment recognition device (pedestrian recognition device) as a specific component will be described in detail.

(Environment recognition system 100)
FIG. 1 is a block diagram showing a connection relationship of the environment recognition system 100. The environment recognition system 100 includes an imaging device 110, an outside vehicle environment recognition device 120 (pedestrian recognition device), and a vehicle control device (ECU: Engine Control Unit) 130 provided in the vehicle 1. .

  The imaging device 110 is configured to include an imaging element such as a CCD (Charge-Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor). The imaging device 110 captures an environment corresponding to the front of the vehicle 1 and captures three hues (R (red ), G (green), and B (blue)). Here, the color image picked up by the image pickup apparatus 110 is distinguished from a distance image described later.

  In addition, the imaging devices 110 are arranged in a substantially horizontal direction so that the optical axes of the two imaging devices 110 are substantially parallel on the traveling direction side of the vehicle 1. The imaging device 110 continuously generates image data obtained by imaging an object existing in a detection area in front of the vehicle 1, for example, every 1/60 second frame (60 fps). Here, the objects to be recognized are not only three-dimensional objects such as vehicles, pedestrians, traffic lights, roads (travel paths), guardrails, and buildings, but also three-dimensional objects such as taillights, blinkers, and lighting parts of traffic lights. The thing which can be specified as a part of is also included. Each functional unit in the following embodiment performs each process for each frame in response to such update of the image data.

  The vehicle exterior environment recognition device 120 acquires image data from each of the two imaging devices 110, derives parallax using so-called pattern matching, and associates the derived parallax information (corresponding to a depth distance described later) with the image data. Generate a distance image. The color image and the distance image will be described in detail later. Further, the outside-vehicle environment recognition device 120 uses the brightness and color difference based on the color image and the depth distance from the vehicle 1 based on the distance image to correspond to any specific object in the detection area in front of the vehicle 1. To identify.

  Further, when the outside environment recognition device 120 identifies a specific object, the specific object (for example, a preceding vehicle) is tracked and the relative speed of the specific object is derived, and the specific object and the vehicle 1 may collide. It is determined whether or not is high. Here, when it is determined that the possibility of the collision is high, the outside environment recognition device 120 displays a warning (notification) to the driver through the display 122 installed in front of the driver, and the vehicle control device. Information indicating that is output to 130.

  The vehicle control device 130 receives a driver's operation input through the steering wheel 132, the accelerator pedal 134, and the brake pedal 136, and controls the vehicle 1 by transmitting it to the steering mechanism 142, the drive mechanism 144, and the brake mechanism 146. In addition, the vehicle control device 130 controls the drive mechanism 144 and the braking mechanism 146 in accordance with instructions from the outside environment recognition device 120.

  Hereinafter, the configuration of the outside environment recognition device 120 will be described in detail. Here, the procedure for identifying an object as a pedestrian, which is characteristic of the present embodiment, will be described in detail, and the description of the configuration unrelated to the features of the present embodiment will be omitted.

(Vehicle environment recognition device 120)
FIG. 2 is a functional block diagram showing a schematic function of the outside environment recognition device 120. As shown in FIG. 2, the vehicle exterior environment recognition device 120 includes an I / F unit 150, a data holding unit 152, and a central control unit 154.

  The I / F unit 150 is an interface for performing bidirectional information exchange with the imaging device 110 and the vehicle control device 130. The data holding unit 152 includes a RAM, a flash memory, an HDD, and the like. The data holding unit 152 holds various pieces of information necessary for the processing of each function unit described below, and temporarily holds image data received from the imaging device 110. To do.

  The central control unit 154 is configured by a semiconductor integrated circuit including a central processing unit (CPU), a ROM storing a program, a RAM as a work area, and the like, and through the system bus 156, an I / F unit 150, a data holding unit 152 and the like are controlled. In the present embodiment, the central control unit 154 also functions as an image processing unit 160, a three-dimensional position deriving unit 162, a partial image specifying unit 164, a target background specifying unit 166, and a pedestrian determining unit 168. In the following, detailed operations of such functional units will be described in the order of image processing, partial image specifying processing, pedestrian specifying processing, based on the general purpose.

(Image processing)
The image processing unit 160 acquires image data from each of the two imaging devices 110, and selects a block corresponding to a block arbitrarily extracted from one image data (for example, an array of horizontal 4 pixels × vertical 4 pixels) as the other image data. The parallax is derived using so-called pattern matching that is searched from the above. Here, “horizontal” indicates the horizontal direction of the screen, and “vertical” indicates the vertical direction of the screen.

  As this pattern matching, it is conceivable to compare the luminance (Y color difference signal) in units of blocks indicating an arbitrary image position between two pieces of image data. For example, SAD (Sum of Absolute Difference) that takes the difference in luminance, SSD (Sum of Squared Intensity Difference) that uses the difference squared, or NCC that takes the similarity of the variance value obtained by subtracting the average value from the luminance of each pixel There are methods such as (Normalized Cross Correlation). The image processing unit 160 performs such a block-unit parallax derivation process for all blocks displayed in the detection area (for example, horizontal 600 pixels × vertical 180 pixels). Here, the block is assumed to be horizontal 4 pixels × vertical 4 pixels, but the number of pixels in the block can be arbitrarily set. Hereinafter, a block that is a unit for deriving such parallax information is referred to as a parallax block.

  However, the image processing unit 160 can derive the parallax for each parallax block that is a unit of detection resolution, but cannot recognize what kind of target object the parallax block is. Therefore, the parallax information is derived independently not in units of objects but in units of detection resolution (for example, units of parallax blocks) in the detection region. Here, an image in which the parallax information derived in this way (corresponding to a depth distance described later) is associated with image data is referred to as a distance image.

  FIG. 3 is an explanatory diagram for explaining the color image 210 and the distance image 212. For example, it is assumed that a color image (image data) 210 as shown in FIG. 3A is generated for the detection region 214 through the two imaging devices 110. However, for ease of understanding, only one of the two color images 210 generated by each of the imaging devices 110 is schematically shown here. In the present embodiment, the image processing unit 160 obtains the parallax for each parallax block from such a color image 210, and forms a distance image 212 as shown in FIG. Each parallax block in the distance image 212 is associated with the parallax of the parallax block. Here, for convenience of explanation, the parallax block from which the parallax is derived is represented by black dots.

  Returning to FIG. 2, the three-dimensional position deriving unit 162 uses the so-called stereo method to calculate disparity information for each disparity block in the detection region 214 based on the distance image 212 generated by the image processing unit 160. The information is converted into three-dimensional position information in real space including the horizontal distance, height, and depth distance. Here, the stereo method is a method of deriving the depth distance of the target part with respect to the imaging device 110 from the parallax in the distance image 212 of the target part by using a triangulation method. At this time, the three-dimensional position deriving unit 162 determines the target part based on the depth distance of the target part and the detected distance on the distance image 212 between the point on the road surface and the target part at the same depth distance as the target part. The height from the road surface is derived. Then, the derived three-dimensional position information is associated with the distance image 212 again. Since various known techniques can be applied to the depth distance deriving process and the three-dimensional position specifying process, description thereof is omitted here.

(Partial image identification processing)
The partial image specifying unit 164 groups the target parts (pixels and blocks) in the detection region 214 using the three-dimensional position information based on the distance image 212, and specifies a partial image that includes the target object in part. To do. Specifically, the partial image specifying unit 164 uses an arbitrary target part as a base point, and the difference between the target part, the horizontal distance, the height difference, and the depth distance is within a predetermined range. Other target parts are grouped, and the target parts are also integrated into a partial image. Here, the predetermined range is represented by a distance in the real space, and can be set to an arbitrary value (for example, 1.0 m). Further, the partial image specifying unit 164 also sets the difference in the horizontal distance, the difference in the height, and the difference in the depth distance within a predetermined range with respect to the target part newly added by the grouping, with the target part as a base point. Group some other target area. As a result, if the distance is within the predetermined range, all of the target parts are grouped as a partial image showing the target object.

(Pedestrian identification processing)
4 and 5 are diagrams for explaining the pedestrian identification process. In the pedestrian identification process, it is determined whether or not the target object is a pedestrian based on the appearance of the target object grouped by the target part. When the pedestrian is close to the vehicle 1, the pedestrian determination unit 168 identifies the pedestrian by enhancing the edge and comparing the contour shape with the model shape.

  On the other hand, as shown in FIG. 4A, when the pedestrian (here, the object 220) is far from the vehicle 1 (captured image acquisition position), the gait in the color image 210 acquired from the imaging device 110. A person's image becomes small and it becomes difficult to specify a pedestrian from a contour shape. For example, as shown in FIG. 4B, an image 216a that can be clearly identified as a pedestrian from the contour shape is originally an image having a rough shape with a fine contour shape collapsed as shown in FIG. 4C. 216b.

  Therefore, in the present embodiment, when the object 220 is separated by a predetermined distance or more, image processing by the target background specifying unit 166 is performed.

  As shown in FIG. 4D, the partial image specifying unit 164 generates a rectangular partial image 222a. When the partial image 222b in which the target parts (here, parallax blocks) are grouped is not rectangular as a whole, adjacent parallax blocks 224 are added and grouped into a rectangular shape so as to be rectangular. As a result, in the partial image 222a, the number of blocks in the vertical direction of each column is equal, and the number of blocks in the horizontal direction of each column is equal.

  As shown in FIG. 4E, the target background specifying unit 166 includes, for each row of the parallax blocks constituting the partial image 222a, four blocks adjacent in the horizontal direction from the partial image 222a and a total of eight blocks on the left and right sides. The image 222a is selected by the vertical height. Hereinafter, the selected parallax block is referred to as a reference image 226. That is, the target background specifying unit 166 selects a pixel constituting the partial image 222a and a pixel of the reference image 226 from a plurality of pixels in which the vertical positions of the parallax blocks are equally aligned in the horizontal direction. Become.

  Then, the target background identification unit 166 derives an average value of luminance and an average value of color differences for each of the partial image 222a and the reference image 226 for each row. Here, for example, the luminance Y and the color difference U (difference between the luminance Y and the blue component) are used in the color information expressed in the YUV color space.

  As described above, when the partial image 222b in which the parallax blocks are grouped is not rectangular as a whole, the partial image specifying unit 164 adds the adjacent parallax blocks 224 so as to be rectangular and groups them into a rectangular shape. A partial image 222a is generated. Therefore, the partial image 222a includes parallax blocks that are highly likely not to include the object 220, such as the adjacent parallax block 224. Therefore, for example, the target background specifying unit 166 excludes the parallax blocks at both ends in the horizontal direction from the partial image 222a, and derives the target average that is the average value of the luminance of the pixels constituting the two blocks on the center side.

  With respect to the reference image 226, an average value of luminance of pixels constituting a total of eight parallax blocks on both the left and right sides of the partial image 222a and a background average that is an average value of color differences are derived.

  After that, the target background specifying unit 166 derives the average of the target average and the background average for each luminance and color difference for each row of the parallax block, and sets it as a threshold value for the binarization process described later.

  Further, as illustrated in FIG. 5A, the target background specifying unit 166 divides the partial image 222a into blocks different from the disparity blocks that are the unit for deriving the disparity information. Specifically, the target background specifying unit 166 divides the pixels constituting the partial image 222a roughly equally into 10 blocks in the vertical and horizontal directions. In this way, the partial image 222a is normalized so as to be composed of 10 blocks in the vertical and horizontal directions regardless of the size (the number of pixels). Hereinafter, a block generated by such normalization is referred to as a normalization block.

  Subsequently, the target background specifying unit 166 uses the target average and the background average based on the average luminance value and the average color difference for each normalized block, among the normalized blocks that form the partial image, A normalization block indicating the object 220 is specified.

  Specifically, the target background specifying unit 166 performs binarization processing for each row of the normalized block. As described above, the threshold value for the binarization process is derived for each row of the parallax block. However, since the normalization target is not a disparity block but a normalization block, one of the threshold values derived for each row of the disparity block must be associated with each row of the normalization block. Therefore, the target background specifying unit 166 associates the row of the parallax block closest to the center in the vertical direction of the row of the normalized block with respect to each row of the normalized block. Then, a threshold derived using the parallax block row associated with the normalization block row is used for normalization processing of the normalization block.

  In this way, the target background specifying unit 166 compares the average value of luminance and the average value of color difference of each normalized block with the specified threshold value, and the average value of luminance and the average value of color difference are compared with the threshold value. Is also a value close to the target average or a value close to the background average.

  For each normalized block, the target background specifying unit 166 sets the luminance to 1 if the average luminance value is closer to the target average than the threshold value, and the average luminance value is closer to the background average value than the threshold value. In this case, the luminance is 0. The target background specifying unit 166 performs the same process for the color difference.

  For example, when the target average is larger than the background average, the target background specifying unit 166 sets the luminance to 1 if the average value of the luminance of the normalized block is larger than the threshold, and increases the luminance if the average is smaller than the threshold. 0.

  The target background specifying unit 166 performs the above binarization process on each row of the normalized block, and binarizes the luminance values and the color differences of all the normalized blocks constituting the partial image 222a. If there are one or more of the two values of the luminance value and the color difference having a value of 1, the representative value of the normalized block is set to 1. In addition, if the values of the luminance value and the color difference are both 0, the representative value of the normalized block is set to 0. In FIG. 5B, a normalized block with a representative value of 1 is shown in black, and a normalized block with a representative value of 0 is shown in white.

  Here, it is assumed that a normalized block with a representative value of 1 indicates a part of the object 220, and a normalized block with a representative value of 0 indicates a part of the background of the object 220.

  As described above, the target background specifying unit 166 specifies the partial image 222a having a predetermined shape (for example, a rectangle) indicating the target object 220 from the color image acquired from the imaging device 110, the luminance of the partial image 222a, and Based on the color difference, a normalization block indicating the object 220 in the normalization block constituting the partial image 222a is specified.

  Here, a case has been described in which whether or not the object 220 is indicated for each normalized block has been described. However, the constituent part that is a unit of the specified process may be a pixel instead of each normalized block. However, other pixel groups may be used.

  The pedestrian determination unit 168 determines whether or not the partial image 222a indicates a pedestrian, that is, whether or not the target object 220 is a pedestrian, based on the normalized block distribution indicating the target object 220 in the partial image 222a. To do. In this embodiment, since the partial image 222a is normalized to a predetermined number of blocks, the pedestrian determination unit 168 determines that the object 220 is a pedestrian based on the distribution of normalized blocks indicating the background in the partial image 222a. It can be determined whether or not.

  In the example illustrated in FIG. 5C, the pedestrian determination unit 168 specifies that the normalized blocks in the first row from the top are all 10 normalized blocks having a representative value of 0, which are continuous. Similarly, the pedestrian determination unit 168 determines that the normalization blocks in the second row from the top are eight normalization blocks with a representative value of 0 and two groups on the left and right with the normalization block with a representative value of 1. Identify that they are separated.

  In addition, the pedestrian determination unit 168 divides the normalized block in the ninth row from the top into eight normalized blocks with a representative value of 0 and a normalized block with a representative value of 1 so as to be divided into three groups. Identify that they are separated.

  In this way, the pedestrian determination unit 168 specifies the total number and the number of groups for the normalized block whose representative value is 0 for all the rows of the normalized block.

  And the pedestrian determination part 168 shall correspond the normalization block from the top to the 1st line to the pedestrian's head, and the normalization block from the top to the 4th to 7th line is the pedestrian. The following determination processing is performed on the assumption that the normalized blocks from the top to the 10th row from the top correspond to the legs of the pedestrian.

As shown in FIG. 5D, the pedestrian determination unit 168 determines the degree of similarity to the pedestrian image based on the distribution of normalized blocks indicating the background with the representative value of 0 in the partial image 222a. If the feature is similar to a pedestrian image, an evaluation point is added. Below, features similar to pedestrian images are listed.
a. Head (1st to 3rd lines)
-The normalization block indicating the object 220 continues more than the trunk and legs (the width is large).
The normalization block indicating the object 220 is grouped into one group for each row (not divided into a plurality of groups).
b. Torso (4th to 7th lines)
The normalization block indicating the object 220 is grouped into one group for each row (not divided into a plurality of groups).
The number of normalized blocks representing the object 220 is 3-5 per row c. Leg (8th to 10th lines)
The normalization block indicating the object 220 is divided into two groups for each row. The number of normalization blocks constituting each of the two groups is 1 to 3.

  In the partial image 222a, the feature of the pedestrian image is more likely to appear in the portion closer to the leg than in the portion closer to the head. Therefore, the pedestrian determination unit 168 determines that the shape (distribution) of a set of normalized blocks having a low vertical position of the image is more than the shape (distribution) of a set of normalized blocks having a high vertical position. Add a large weight and add evaluation points.

  As described above, the pedestrian determination unit 168 determines whether or not the distribution of the normalized blocks indicating the background in the partial image 222a corresponds to a feature similar to the above pedestrian image. Points corresponding to the corresponding feature are added to the image 222a. And after the determination about all the characteristics is complete | finished, the pedestrian determination part 168 will show the pedestrian in the partial image 222a, when the integrated evaluation point is more than predetermined value, ie, the target object 220 will be a pedestrian. It is determined that

  FIG. 6 is a flowchart for explaining the flow of the pedestrian identification process. The pedestrian specifying process shown in FIG. 6 is repeatedly executed for each frame of the color image acquired from the imaging device 110.

  As illustrated in FIG. 6, the image processing unit 160 derives the parallax of the parallax block using pattern matching to generate the distance image 212 (S300). Then, the three-dimensional position deriving unit 162 derives a three-dimensional position in the real space of the parallax block (S302). The partial image specifying unit 164 groups the parallax blocks in the detection area 214 using the three-dimensional position information based on the distance image 212, and specifies the partial image 222a that includes the object 220 in part (S304). ).

  The pedestrian determination unit 168 selects one unselected one of the identified partial images 222a (S306). Then, the pedestrian determination unit 168 determines whether or not the selected object 220 is separated from the vehicle 1 (captured image acquisition position) by a predetermined distance or more (S308). When the distance is not greater than the predetermined distance (NO in S308), the pedestrian determination unit 168 performs edge enhancement processing on the image of the object 220 (S312), and the contour shape indicated by the edge is similar to the shape of the pedestrian. It is determined whether it can be regarded as a pedestrian (S314). When it cannot be regarded as a pedestrian (NO in S314), the process proceeds to an unselected determination process S318.

  When the contour shape indicated by the edge can be regarded as a pedestrian similar to the shape of a pedestrian (YES in S314), the object 220 is identified as a pedestrian (S316), and the partial image that has not yet been selected. It is determined whether or not there is 222a (S318). If there is no partial image 222a that has not yet been selected (NO in S318), the pedestrian identification process ends. If there is a partial image 222a that has not yet been selected (YES in S318), the process proceeds to partial image selection processing S306.

  When the target object 220 is separated from the vehicle 1 by a predetermined distance or more (YES in S308), the target background specifying unit 166 uses the partial image 222a and the reference image 226 to perform each row based on the pixel brightness and the color difference. A threshold is derived for each (S320). Then, the target background specifying unit 166 normalizes the pixels constituting the partial image 222a by dividing the pixels in the vertical and horizontal directions roughly equally into 10 blocks (S322).

  Thereafter, the target background specifying unit 166 derives the average value of the luminance of the pixel and the average value of the color difference for each normalized block, and compares the average value with the corresponding threshold value. If the average luminance value is closer to the target average than the threshold, the luminance is set to 1. If the average luminance value is closer to the background average than the threshold, the binarization processing is set to 0. I do. The binarization process is also performed for the color difference. If there are one or more of the two values of the luminance value and the color difference having a value of 1, the representative value of the normalized block is set to 1. If two values of the luminance value and the color difference are two, the representative value of the normalized block is set to 0 (S324).

  The pedestrian determination unit 168 identifies the distribution of normalized blocks indicating the background in the partial image 222a, determines the degree of similarity with the pedestrian image, and derives an evaluation point (S326). If the evaluation point exceeds the predetermined value (YES in S328), the pedestrian determination unit 168 specifies the object 220 as a pedestrian (S316), and moves the process to the unselected determination process S318. If the evaluation point does not exceed the predetermined value (NO in S328), the process proceeds to the unselected determination process S318.

  As described above, according to the pedestrian specifying process of the present embodiment, the pedestrian is far away from the vehicle 1, the pedestrian image acquired from the imaging device 110 is too small, and the contour of the pedestrian is Even if identification is difficult, it becomes possible to detect a pedestrian early and with high accuracy.

  Also provided are a program that causes a computer to function as the vehicle exterior environment recognition device 120 and a computer-readable storage medium that stores the program, such as a flexible disk, magneto-optical disk, ROM, CD, DVD, and BD. Here, the program refers to data processing means described in an arbitrary language or description method.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to this embodiment. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Is done.

  For example, in the above-described embodiment, the target background specifying unit 166 normalizes the partial image 222a as a constituent part that is a processing unit of a specific process for specifying whether the target object or the background is indicated. However, as described above, the constituent part may be a pixel or another pixel group. However, if the partial image 222a is normalized to be a normalized block, it is easy to determine whether or not the person is a pedestrian from the distribution of the normalized blocks indicating the object regardless of the size of the partial image 222a. Become.

  In the above-described embodiment, the case where the normalized block indicating the target object is specified by comparing the luminance and color difference of the pixels constituting the normalized block with the corresponding threshold value has been described. You may identify the normalization block which shows a target object by processes other than a comparison. However, it is possible to perform the specifying process with a simple process by specifying the normalized block indicating the object by comparison with the threshold.

  In the above-described embodiment, the target background specifying unit 166 determines the target from the luminance and color difference of the pixels constituting the partial image 222a and the pixels of the reference image 226 outside the partial image 222a adjacent to the partial image 222a. A case has been described in which thresholds for specifying normalized blocks indicating objects are derived. However, an arbitrary value may be set as the threshold value. However, since there is a high possibility that the reference image 226 includes a background, the threshold value is derived from the luminance and color difference of the pixels constituting the partial image 222a and the pixels of the reference image 226. The object and the background can be separated from the image 222a with high accuracy. As a result, pedestrian detection accuracy is improved.

  In the above-described embodiment, when the target background specifying unit 166 performs the normalization process of the partial image 222a and then performs the binarization process of the luminance and color difference of the partial image 222a by comparison with the threshold value. Explained. However, the target background specifying unit 166 may perform the normalization process of the partial image 222a after performing the binarization process of the luminance and the color difference of the partial image 222a.

  In the above-described embodiment, the pedestrian determination unit 168 determines that the shape (distribution) of a set of normalized blocks having a lower vertical position in the image has a shape of a set of normalized blocks having a higher vertical position. The case where evaluation points are added by weighting more than (distribution) has been described. However, the pedestrian determination unit 168 may not perform such weighting. However, by performing such weighting, the influence of the point on the lower body of a person, which makes it easy to specify the shape, is increased, and the detection accuracy can be improved.

  In the above-described embodiment, the target background specifying unit 166 specifies a normalization block indicating the target object among the normalization blocks constituting the partial image, and the pedestrian determination unit 168 indicates the target object in the partial image. The case where it is determined whether or not the object is a pedestrian based on the distribution of normalized blocks has been described. However, the target background specifying unit 166 specifies a normalized block indicating the background among the normalized blocks constituting the partial image, and the pedestrian determination unit 168 is based on the distribution of the normalized block indicating the background in the partial image. It may be determined whether the object is a pedestrian. Furthermore, it may be determined whether or not the object is a pedestrian based on the distribution of both the normalized block indicating the object and the normalized block indicating the background.

  Further, in the above-described embodiment, the target background specifying unit 166 indicates the target object among the normalized blocks constituting the partial image using the luminance Y and the color difference U among the color information represented in the YUV color space. The case where the normalization block is specified has been described. However, the target background specifying unit 166 configures a partial image using the luminance Y and the color difference V or both the luminance Y and the color differences U and V among the color information expressed in the YUV color space. You may identify the normalization block which shows a target object among normalization blocks. Furthermore, the target background specifying unit 166 may specify a normalization block indicating the target object from among the normalization blocks constituting the partial image using either one or both of the color differences U and V.

  Further, the target background specifying unit 166 may use not only the color information expressed in the YUV color space but also the color information expressed by the luminance of each hue of RGB, not only the color image but also the gray scale. Image information represented by luminance may be used. However, the captured image is a color image, and the target background specifying unit 166 specifies a normalization block indicating the target object from the normalization blocks constituting the partial image based on the color difference in addition to the luminance. Even if the difference between the luminance Y is difficult for the background and the object, identification is easy if there is a difference in the color differences U and V.

  Further, in the above-described embodiment, the pedestrian determination unit 168 has a case where the object included in the partial image is located at a position less than a predetermined distance from the vehicle 1 (captured image acquisition position and position of the imaging device 110). Based on the contour shape of the object, it is determined whether or not the object is a pedestrian, and when the object included in the partial image is separated from the vehicle 1 by a predetermined distance or more, normalization indicating the object in the partial image The case where it is determined whether the object is a pedestrian based on the block distribution has been described. However, the pedestrian determination unit 168 determines whether the target object is a pedestrian based on the distribution of normalized blocks indicating the target object in the partial image regardless of the position of the target object included in the partial image. Also good. However, when the target object included in the partial image is at a position less than a predetermined distance from the pedestrian recognition device, it is determined whether the target object is a pedestrian based on the contour shape of the target object. By avoiding the determination based on the distribution, the processing load can be reduced.

  In addition, each process of the pedestrian specification process of this specification does not necessarily need to process in time series along the order described as a flowchart, and may include the process by parallel or a subroutine.

  INDUSTRIAL APPLICABILITY The present invention can be used for a pedestrian recognition device that specifies to which specific object an object existing in a detection region corresponds.

1 Vehicle 100 Environment Recognition System 120 Outside Environment Recognition Device (Pedestrian Recognition Device)
160 Image processing unit 162 Three-dimensional position deriving unit 164 Partial image specifying unit 166 Target background specifying unit 168 Pedestrian determination unit 220 Object 222a Partial image 226 Reference image

Claims (5)

A three-dimensional position deriving unit for deriving a three-dimensional position in real space of a plurality of target parts in a captured image obtained by imaging a detection region;
A partial image specifying unit for grouping target parts having a difference in the three-dimensional position within a predetermined range, and specifying a partial image including the target in the captured image;
Based on the luminance and color difference of the constituent parts constituting the partial image, at least one of the luminance average value or the color difference average value of the partial image is derived as a target average, and a plurality of constituent parts adjacent to the partial image are referred to as a reference image And at least one of the luminance average value or the color difference average value of the reference image is derived as a background average based on the luminance or the color difference of the constituent parts constituting the reference image, and the target average and the background average are used to A target background specifying unit for specifying at least one of a constituent part showing a background and a constituent part showing the object among constituent parts constituting an image;
Whether the target object is a pedestrian based on the number of groups in which at least one of the constituent part indicating the background and the constituent part indicating the object in the partial image is continuous in the horizontal direction, or the distribution of the number of the one. A pedestrian determination unit for determining whether or not,
A pedestrian recognition device comprising:   The pedestrian recognition device according to claim 1, wherein the target background specifying unit divides and normalizes the partial image into a plurality of blocks, and uses the blocks as the constituent parts.   A three-dimensional position deriving unit for deriving a three-dimensional position in real space of a plurality of target parts in a captured image obtained by imaging a detection region;
  A partial image specifying unit for grouping target parts having a difference in the three-dimensional position within a predetermined range, and specifying a partial image including the target in the captured image;
  A plurality of blocks obtained by dividing and normalizing the partial image are used as components constituting the partial image, and at least one of the luminance average value or the color difference average value of the partial image is targeted based on the luminance and color difference of the block A plurality of blocks adjacent to the partial image are used as a reference image, and at least one of the luminance average value or the color difference average value of the reference image is used as a background based on the luminance or the color difference of the blocks constituting the reference image. A target background specifying unit for determining at least one of a block indicating the background and a block indicating the target object among the blocks constituting the partial image using the target average and the background average,
  Based on the distribution of at least one of the blocks indicating the background and the blocks indicating the object in the partial image, the distribution of the blocks having a lower vertical position is higher than the distribution of the blocks having a lower vertical position. Than, a pedestrian determination unit that determines whether or not the object is a pedestrian,
A pedestrian recognition device comprising: The target background specifying unit selects a pixel constituting the partial image and a pixel constituting the reference image from a plurality of pixels in which the vertical position of the captured image is equally aligned in the horizontal direction, The partial image is constructed by deriving a threshold using the target average and the background average derived based on the selected pixel, and comparing the luminance or color difference of each component constituting the partial image with the threshold. The pedestrian recognition device according to any one of claims 1 to 3 , wherein the constituent part to be identified is a constituent part showing a background or a constituent part showing the object. . The pedestrian determination unit
When the target object included in the partial image is at a position less than a predetermined distance from the acquisition position of the captured image, based on the contour shape of the target object, determine whether the target object is a pedestrian,
When the object included in the partial image is separated from the acquisition position of the captured image by a predetermined distance or more, based on the distribution of at least one of the component part indicating the background and the component part indicating the object in the partial image The pedestrian recognition device according to any one of claims 1 to 4, wherein it is determined whether or not the object is a pedestrian.