JP5838940B2 - Image processing apparatus and vehicle control system - Google Patents

Description

  The present invention relates to an image processing technique for detecting a target in a captured image.

  2. Description of the Related Art Conventionally, for example, an image processing technique for detecting (identifying) a target (object) such as a pedestrian in an image captured by a camera mounted on a vehicle is known. A typical method of this type of image processing is to identify a target by using a target model obtained in advance by machine learning, scanning the image, and performing matching at each position. Here, the target model is a representative pattern obtained by collecting a large number of images of various states of the target (for example, in the case of a person, clothes, orientation, posture, etc.) and learning them.

  However, the appearance of the target varies greatly depending on its state. For example, in the case of a person, the positions of limbs and the like differ in each state, such as a stationary state, a walking state, and a running state. For this reason, it is required to construct a discriminator that is low in discrimination performance only with a conventional simple learning model and is robust against such changes in target appearance.

  Therefore, together with the conventional model of the entire target, distinctive parts of the target (for example, parts such as head, arms, torso, and legs for humans) are separately learned, and identification taking into account the positional relationship (spring model) of the parts A technique for performing the above has been proposed. Here, the spring model (DPM: Deformable Part Model) defines a reference position of each part with respect to the entire model and a value (spring cost) corresponding to the position of the part with respect to the reference position. Specifically, in the spring model, it is defined that the spring cost (penalty) increases as the deviation of the part from the reference position increases, and the position of the part can be evaluated based on the spring cost. In other words, even if the position is slightly different from the reference position, it can be determined that the part shape is similar to the model, and it can be identified flexibly even if the position of the part changes to some extent. can do.

P. Felzenszwalb, D. McAllester, D. Ramanan, A Discriminatively Trained, Multiscale, Deformable Part Model, Proceedings of the IEEE CVPR 2008

  By the way, in a state in which a plurality of targets constitute a group, the front target is detected, but the back (rear) target is partially hidden by the front target, so that it is difficult to detect. There's a problem. For example, in the group of pedestrians, the pedestrian closest to the camera is detected, but the pedestrian hidden behind the pedestrian is a part of the body of the pedestrian in front. By hiding by the body, the score of the part decreases, so that it becomes difficult to be detected as a pedestrian. Note that such a problem is not limited to pedestrians, and may occur similarly for other targets.

  The present invention has been made in view of these problems, and an object of the present invention is to provide a technique for enabling a target population to be detected with high accuracy.

  The image processing apparatus (10) of the present invention inputs an image picked up by the image pickup apparatus (32) and performs image processing for detecting a target in the image. The storage means (60) and parts Detection means (S12) and group detection means (S13, S14) are provided.

  The storage means stores a part model for identifying a plurality of types of parts of the target in the image, and a spring model representing a reference position of the plurality of types of parts on the target and a spring cost corresponding to the position of the part relative to the reference position. To do. And a part detection means detects parts in an image based on the similarity calculated by the matching process using a parts model. Further, the group detection unit detects a group of targets in the image based on the parts detected by the part detection unit.

  Specifically, the group detection means includes setting means (S21, S29), spring cost calculation means (S23), and determination means (S24, S25). The setting means sequentially sets attention areas for determining whether or not the target exists at a plurality of positions in the image. Then, the spring cost calculation means sets the reference position of the part when it is assumed that the target is present in the attention area based on the spring model, and the spring cost that is a value indicating the degree of deviation from the reference position is detected by the part detection means. Calculate for each of the parts. Further, the determination means determines the attention area as the target area where the targets constituting the group exist on the condition that there are a plurality of parts of the same type whose spring cost calculated by the spring cost calculation means is within a predetermined range.

  In other words, when multiple parts of the same type are gathered, it is considered that there is a high possibility that a target group exists.Therefore, there are parts of the same type that have a small degree of deviation from the reference position determined based on the attention area. On the condition that there are a plurality, it is determined that the target constituting the group exists in the attention area. According to such a determination method, it is possible to make it easier to detect a target in the back that is partially hidden by the target in front as a target constituting the group. Therefore, the target group can be detected with higher accuracy than the case where individual targets constituting the group are detected independently.

  In addition, the code | symbol in the parenthesis described in this column and a claim shows the correspondence with the specific means as described in embodiment mentioned later as one aspect, Comprising: The technical scope of this invention is shown. It is not limited.

It is a block diagram which shows the structure of a vehicle control system. It is a figure which shows a collision risk determination table. It is a flowchart of a target group determination process. (A) is a diagram showing a matching process for detecting a target part existing at a first distance, (B) is a diagram showing a matching process for detecting a target part existing at a second distance; (C) is a figure which shows the process which detects a high score part. It is a flowchart of a group condition determination process. It is a figure which shows the process which detects a target area | region. It is a figure which shows a target group detection process. It is a figure which shows the integration method of the target area | region in a target group detection process. (A) is a figure which shows the process which detects the focused pedestrian independently, (B) is a figure which shows the process which detects the focused pedestrian as one of the groups. It is a figure which shows the process which calculates a spring cost low according to the positional relationship with another part of a different kind. It is a figure which shows the process which calculates a spring cost low according to the positional relationship with other parts of the same kind. It is a figure which shows the process which determines the area | region containing a high score part and an attention area as a target area.

Embodiments to which the present invention is applied will be described below with reference to the drawings.
[1. Constitution]
As shown in FIG. 1, the vehicle control system of the present embodiment is configured with an image processing device 10 and a vehicle control device 20 mounted on the vehicle as the center. A distance measuring device 31 and an imaging device 32 mounted on the vehicle are connected to the image processing device 10. The vehicle control device 20 is connected to a speaker 41, a brake 42, and a steering 43 mounted on the vehicle.

  The image processing device 10 and the vehicle control device 20 are configured as a so-called computer. As a result, the image processing apparatus 10 detects the target (pedestrian in the present embodiment) in the image (captured image) input from the imaging apparatus 32 and determines the collision risk based on the distance information from the distance measuring apparatus 31. . On the other hand, the vehicle control device 20 performs notification by the speaker 41 according to the collision risk output from the image processing device 10, or performs avoidance control by the brake 42 and the steering 43 when the degree of emergency is large. To go.

Next, the configuration of each unit will be specifically described.
The distance measuring device 31 is embodied as a laser radar or the like. Of course, another configuration such as a millimeter wave radar may be used. The distance measuring device 31 can detect the distance to the object in front of the vehicle.

  The imaging device 32 is embodied as a monocular camera or the like. The imaging device 32 images the front of the vehicle. The image processing apparatus 10 detects the target in this image as described above.

The speaker 41 is configured for audio output. When the risk of collision with the target increases, a sound such as “Be careful ahead” is output through the speaker 41.
The brake 42 includes a brake driving device, and decelerates the vehicle by a signal from the vehicle control device 20. Further, the steering 43 includes an EPS (Electric Power Steering) device, and assists the driver's steering by a signal from the vehicle control device 20. Such vehicle control is performed when the risk of collision with the target is further increased, and when the degree of emergency is large.

  The image processing apparatus 10 includes a control unit 50 and a storage unit 60. The control unit 50 is embodied as a so-called microcomputer. The control unit 50 includes a target group detection unit 51 and a collision risk determination unit 52 as functional blocks. The storage unit 60 is embodied as a ROM, for example. Of course, a flash memory or HDD other than the ROM may be used. The storage unit 60 stores a part model, a spring model, and a collision risk determination table.

  Here, the target group detection unit 51 performs image processing for detecting the target in the image output from the imaging device 32. More specifically, as will be described later, by scanning an image, a plurality of types of parts included in the target are detected, and the presence and position of the target group are estimated based on the detected parts.

  Since the distance to the target is known by the distance measuring device 31 after the target is detected, the collision risk determination unit 52 determines the collision risk according to the distance to the target. The collision risk is output to the vehicle control device 20.

  The part model stored in the storage unit 60 is obtained by modeling a plurality of types of parts that the target has, and is used to identify each part of the target in the image. The part model is represented, for example, as a feature vector that represents the characteristics of each part such as contour information. In the present embodiment, a part model corresponding to each of the head, left arm, right arm, torso, left leg, and right leg, which are a plurality of types of parts that a pedestrian has, is used.

  The spring model stored in the storage unit 60 is a model of the positional relationship of multiple types of parts (head, left arm, right arm, torso, left leg and right leg) in the target, and the reference position of each part in the target And a value (spring cost) corresponding to the position of the part with respect to the reference position. By using such a spring model, it is possible to express (evaluate) the deviation from the reference position of the part detected by the part model by the spring cost. The spring cost increases as the deviation from the reference position based on the spring model increases, and it can be said that the reliability based on the position increases as the spring cost decreases.

  The collision risk determination table stored in the storage unit 60 is a table that defines the collision risk according to the detected distance to the target and the collective score (reliability of the detection result) described later. Specifically, as shown in FIG. 2, for example, a two-dimensional table indicating the level of collision risk (L1 to L3) according to the distance to the target and the collective score is used. In the present embodiment, when the target group is detected, it is determined that the collision risk is higher as the distance is shorter and the group score is higher (the reliability of the detection result is higher). The collision risk determination unit 52 determines the collision risk by referring to the collision risk determination table.

  The vehicle control device 20 includes a notification control unit 70 and a vehicle control unit 80, both of which are embodied as so-called microcomputers. The notification control unit 70 is configured to perform notification via the speaker 41. When the collision risk level increases, first, notification via the speaker 41 is performed. The vehicle control unit 80 is configured to perform vehicle control via the brake 42 and the steering 43, and collision avoidance control is performed when the risk of collision further increases. Specifically, when the collision risk is level 1 (L1), alarm control (notification control) for notifying the presence of the target is performed, and when it is level 2 (L2), brake control (deceleration control) is performed. In the case of level 3 (L3), steering control (collision avoidance control) is performed. In addition, when the target is not detected (in the case of level 0), vehicle control is not performed.

[2. processing]
The present embodiment is characterized by target group determination processing as a function of the target group detection unit 51. The details of this target group determination processing will be described with reference to the flowchart of FIG. The target group determination process is repeatedly executed at predetermined time intervals.

As shown in FIG. 3, in the first step S <b> 11, an image input process for inputting an image ahead of the vehicle imaged by the imaging device 32 is executed.
Subsequently, in S12, a part detection process for detecting each part based on the similarity (matching degree) calculated by the matching process using the target part model stored in the storage unit 60 in the input image. Execute. The calculated similarity represents a reliability based on the shape. The similarity between each position on the image and the part model is calculated by a known matching algorithm using, for example, HOG (Histograms of Oriented Gradients) feature amount.

  Specifically, as shown in FIG. 4A, the head part model HM is moved rightward at predetermined intervals, for example, sequentially from the upper left part of the image. When the right end of the image is reached, the image is returned to the left end and moved downward at a predetermined interval. Thereafter, the same movement is repeated, and the image is moved in order to the lower right part of the image. At this time, for each position of the head part model HM, the similarity with the partial area overlapping the head part model HM is stored as a score, thereby generating a score map in which the score is recorded for each position in the image. For parts other than the head part, processing similar to that for the head part is performed, and a score map is generated for each type of part.

  In addition, the closer the target from the imaging device 32 (the distance from the imaging position), the larger the image appears in the image. Therefore, as shown in FIG. 4B, the same image is formed by reducing the image to be scanned while keeping the size of the part model HM as it is. That is, the size of the part model relative to the image is relatively increased. In this way, a scan assuming the target existing at the first distance (FIG. 4A) and a scan assuming the target existing at the second distance (FIG. 4B) are performed.

  Here, the scanning in two stages, that is, the case where the target exists at a first distance that is relatively far away and the case where the target exists at a second distance that is relatively close, has been described. It is conceivable to scan in three or more stages. In the stepwise scan, the image to be scanned may be enlarged / reduced as described above, or the part model may be enlarged / reduced on the contrary.

  In the score map generated for each part type in this way, as shown in FIG. 4C, a position (area) having a score equal to or higher than the set threshold value is detected as the high score parts HP1 and HP2. That is, a high score part is an area with high matching with a part model (high possibility of being a part) in an image. In the example of FIG. 4C, the high score part HP1 is a high score part obtained by scanning assuming a target existing at the first distance, and the high score part HP2 is a target present at the second distance. It is a high score part by the scan which assumed the. Note that parts other than the head part are also processed in the same manner as the head part, and a high score part is detected for each type of part.

  Returning to FIG. 3, in S <b> 13, a group condition determination process is performed to determine whether or not a group condition that is a condition for determining that a target group exists in the image is satisfied. The collective condition defined in the present embodiment is that there are a predetermined number or more of high score parts having a small spring cost with respect to one reference position.

Here, the details of the collective condition determination process executed in S13 will be described using the flowchart of FIG.
First, in S21, a region of interest for determining the presence or absence of a target is set in the image. The attention area is an area of a size corresponding to the entire target body (rectangular area in this embodiment), and by setting the attention area, it is assumed that the target exists in the attention area. The position (spring cost evaluation criteria) is set in the image based on the spring model. As will be described later, the region of interest moves to the right at predetermined intervals, for example, in order from the upper left part of the image, and returns to the left end when reaching the right end of the image, as in the case of the above-described part model scan. , Move downward at a predetermined interval. Thereafter, the same movement is repeated and the image is moved in order to the lower right part of the image. In other words, according to a predetermined rule, the attention area is sequentially set at a plurality of positions in the image, and immediately after the start of the collective condition determination process, it is set at the initial position (for example, the upper left portion of the image). The

  Subsequently, in S22, the head part is set as a part to be processed. That is, since the following processes of S23 to S25 are performed for each type of part, the part to be processed is first set as the head part. Note that the parts to be initially processed and the processing order are not particularly limited.

Subsequently, in S23, for each of the high score parts of the part to be processed, the spring cost with respect to the reference position of the part to be processed determined based on the attention area is calculated. The spring cost is a value representing a deviation from the reference position, and the higher the deviation, the higher the value. The spring cost can be simply a value proportional to the magnitude of the deviation (distance from the reference position to the position of the high score part). Since there is a characteristic such as the tendency to shift in the direction, it is preferable to set the value according to the characteristic. For example, the spring cost y with respect to the distance x from the reference position is represented by a quadratic function y = ax ² , and the coefficient a of the quadratic function is a vertical distance (deviation amount) and a lateral distance (deviation amount). May be different. When the rate of increase of the spring cost with respect to the amount of displacement in the vertical direction is set to be larger than the rate of increase of the spring cost with respect to the amount of displacement in the horizontal direction, for example, as shown in FIG. A long oval shape. That is, the horizontal shift is more easily tolerated than the vertical shift.

  Subsequently, in S24, it is determined whether or not the number of high score parts having a spring cost equal to or less than a threshold value is equal to or greater than a certain number (three in the present embodiment). That is, it is determined whether or not there are a plurality of target parts (in this embodiment, three persons) having high score parts with a small deviation from the reference position determined based on the attention area. If it is determined that the number is equal to or greater than a certain number, the process proceeds to S25, and the current attention area is set as an area (hereinafter referred to as “target area”) in which it is estimated that a target constituting a part of the group exists. After detection, the process proceeds to S26. For example, in the example illustrated in FIG. 6, since the five high score parts HP are included in the region CR in which the spring cost with respect to the reference position BP is equal to or less than the threshold value, the current attention region FR is detected as the target region TR. On the other hand, if it is determined in S24 that the number of high score parts whose spring cost is equal to or less than the threshold value is not a certain number or more, S25 is skipped and the process proceeds to S26.

  Subsequently, in S26, it is determined whether or not all parts have been processed in the current attention area. If it is determined that all parts have not been processed, the process proceeds to S27, and after changing the part to be processed, the process returns to S23. For example, the parts to be processed are changed in the order of head part → left arm part → right arm part → trunk part → left leg part → right leg part.

  On the other hand, if it is determined in S26 that all parts have been processed, the process proceeds to S28, where it is determined whether or not all the attention areas have been processed in the current captured image. If it is determined that all the attention areas have not been processed, the process proceeds to S29, the position of the attention area is changed, and the process returns to S22. As described above, when the attention area is sequentially moved from the upper left part to the lower right part of the image, the position of the attention area is changed according to the movement order.

  On the other hand, if it is determined that the processing for all the attention areas has been completed, the collective condition determination processing in FIG. 5 ends. In other words, in the collective condition determination process, the spring cost for each high score part is calculated for each attention area, and when there are a certain number or more of high score parts within a certain range, the attention area is set as the target area. Detected. The target area is an area that is detected for each type of part, and is likely to have a target that constitutes a group. As described above, since the score map is generated for each different distance (for example, the first distance and the second distance), the collective condition determination process is also executed for each different distance.

  Returning to FIG. 3, in S14, the target areas (rectangular areas) detected for each type of part in S13 are integrated (grouped) according to their positions, and detected as a collective area that is an area where the target population exists. Target group detection processing is executed. Specifically, as shown in FIG. 7, a plurality of target regions TR are grouped together, and a circumscribed rectangle including the grouped target regions TR is defined as a final collective region GR. At this time, as shown in FIG. 8, the distances estimated from the target size (for example, the first distance and the second distance) are integrated with each other (a distant target and a nearby target) , And separately integrating them), the detection accuracy of the collective region can be improved. Here, the distance of the target region TR can be estimated based on the lower end position of the target region (the position under the target's feet). In this way, it is possible to perform detailed classification by distance.

  In subsequent S15, a collective score calculation process for calculating a collective score that is a value representing the reliability of the detection result of the collective region is executed. Specifically, the sum of the scores of the target regions included in the collective region is calculated as the collective score. The score of the target area is calculated as the sum of the scores of a plurality of high score parts used for determining the target area. Thereafter, the target group determination process in FIG. 3 ends.

Here, the collective score is calculated based on the score of each part. However, the present invention is not limited to this, and the collective score may be calculated based on the spring cost of each part. For example, according to the following equation, the collective score is calculated higher as the plurality of parts are gathered. In the following formula, n is the number of parts (type of part), ave _i is the average value of spring cost by part type, AVE _i is a constant related to the average value of spring cost, and ver _i is the variance value of spring cost by type of part. , VER _i is a constant related to the spring cost variance value. A value obtained by adding the group score based on the score of each part and the group score based on the spring cost of each part may be used as the group score.

  When the target group is detected in the image by the target group determination process as described above, the vehicle control device 20 performs control according to the distance to the target and the group score as described above. Specifically, when the risk of collision is low, alarm control (notification control) that notifies the presence of the target is performed, and when the risk of collision becomes high, brake control (deceleration control) is performed, and the risk of collision is further increased. Then, steering control (collision avoidance control) is performed. In each control, a plurality of stages of control may be performed. For example, the alarm level or the like may be changed according to the position (distance) of the target relative to the lane in which the vehicle travels. Further, for example, in steering control, the avoidance path may be changed according to the group score.

[3. effect]
According to the embodiment described above in detail, the following effects can be obtained.
(1) The image processing apparatus 10 detects parts in the image captured by the imaging apparatus 32 (S12), and detects a target group in the image based on the detected parts (high score parts) (S13, S14). ). Specifically, the attention area for determining the presence or absence of the target is sequentially set at a plurality of positions in the image (S21, S29), the spring cost is calculated for each detected part (S23), and the spring cost is within a predetermined range. The region of interest is determined as the target region on the condition that there are a plurality of parts of the same type in S <b> 24 and S <b> 25.

  The number of parts detected in the image is one for each target (one pedestrian). If there are multiple parts of the same type, there is a high possibility that multiple targets are gathered. it is conceivable that. For this reason, on the condition that there are a plurality of parts of the same type with a small degree of deviation from the reference position determined based on the attention area, it is determined that the target constituting the group exists in the attention area. According to such a determination method, it is possible to make it easier to detect a target in the back that is partially hidden by the target in front as a target constituting the group. Therefore, the target group can be detected with higher accuracy than the conventional method of detecting individual targets constituting the group independently.

  For example, as shown in FIG. 9A, when attention is paid to a pedestrian surrounded by a rectangular frame in a group of pedestrians, the left arm is hidden by the pedestrian in the forefront (center in FIG. 9A). (Region HR). For this reason, if it is going to detect the pedestrian to pay attention independently, the score of a left arm will fall, As a result, the whole score will fall and it will become difficult to be detected as a pedestrian.

  On the other hand, as shown in FIG. 9B, the left arms of other pedestrians are exposed near the left arm (region HR) of the pedestrian of interest (region ER), and these are high scores. Detected (as a high score part). Therefore, according to the method of the present embodiment on condition that there are a plurality of parts of the same type with a low spring cost, it can be easily detected as a pedestrian constituting a group.

  (2) The image processing apparatus 10 integrates the target regions determined for each type of part, and detects a collective region where a target population exists (S14). The size of the area occupied by the group in the image differs depending on whether the number of targets constituting the group is small or large, but the target area is integrated to detect the group area. It is possible to detect a collective region having a size corresponding to the number of.

Specifically, since a region including a plurality of target regions overlapping each other is detected as a collective region, the collective region can be easily specified.
In addition, for each group classified according to the distance from the imaging device 32, a region including a plurality of target regions that overlap each other is detected as a collective region, so that a target population can be detected with high accuracy. Can do.

  (3) The image processing apparatus 10 calculates a collective score of the detected collective region (S15). Since the detection of the collective region is based on image processing, erroneous detection may occur, and the reliability of the detection result is not always constant. In this regard, in the present embodiment, since the collective score of the detected collective region is calculated, the reliability of the detection result can be evaluated on an objective basis.

  Specifically, the total score of the parts used to determine each target area included in the collective area is calculated as a collective score, so a highly reliable collective score can be calculated easily. Can do. Further, even if a value based on the spring cost of the parts used for determining each target region included in the collective region (for example, a value indicating the degree of variation of the spring cost) is calculated as the collective score, the same effect can be obtained. it can. Further, a value obtained by adding a value based on the sum of scores of each part and a value based on the spring cost of each part may be calculated as a collective score.

  (4) The vehicle control system according to the present embodiment includes a vehicle control device 20 that performs notification control for vehicle occupants and vehicle collision avoidance control as control based on the collective score of the collective region calculated by the image processing device 10. Prepare. Therefore, the target group can be notified to the vehicle occupant, and a collision with the target group can be avoided.

  In particular, since the vehicle control device 20 is configured to vary the control content according to the collective score of the collective region, the vehicle control device 20 can perform control according to the reliability of the collective region detected by the image processing device 10. . In addition, in the present embodiment, as control based on the group score, control according to the distance to the target and the group score is performed, so that more appropriate control can be realized.

  In the above embodiment, the storage unit 60 corresponds to an example of a storage unit, and S12 corresponds to an example of a process as a part detection unit. Further, S13 and S14 correspond to an example of processing as a group detection unit, in particular, S14 corresponds to an example of processing as an integration unit, S21 and S29 correspond to an example of processing as a setting unit, and S23. This corresponds to an example of processing as a spring cost calculation unit, and S24 and S25 correspond to an example of processing as a determination unit. The group score corresponds to an example of an evaluation value, and S15 corresponds to an example of processing as an evaluation value calculation unit.

[4. Other Embodiments]
As mentioned above, although embodiment of this invention was described, it cannot be overemphasized that this invention can take a various form, without being limited to the said embodiment.

  (1) In the above embodiment, it is determined that there is a target group when there are a predetermined number or more of parts having a high score in the score map and a low spring cost based on the spring model (parts having a high score and a low spring cost). It is a collective condition that is a condition for This is because there is a high possibility that a group of targets is present when there are a large number of target parts with high reliability within a certain range both in shape and position. However, the collective conditions in the above embodiment are merely examples, and the present invention is not limited to these.

  For example, a collective condition may be that a predetermined number or more of high-score and low-spring cost parts exist, and that the scores of these parts are generally high. Specifically, for example, in S24 of the above embodiment, the number of high score parts whose calculated spring cost is less than or equal to a threshold value is a certain number or more, and the sum of the scores of these high score parts is a predetermined threshold value. If it is determined whether or not this is the case and an affirmative determination is made, the process proceeds to S25 to detect the current attention area as the target area. In this way, it is possible to improve the accuracy of determining whether or not a target group exists. Note that whether or not the score of a plurality of parts is generally high is not limited to the sum of scores of a plurality of parts being equal to or greater than a threshold value. For example, the average value of the scores of a plurality of parts It may be evaluated that is equal to or greater than a threshold value.

  (2) Further, for example, it may be set as a collective condition that there are a predetermined number or more of high score and low spring cost parts, and that the variation of the spring cost of the plurality of parts is small. Specifically, for example, in S24 of the above embodiment, the number of high score parts whose calculated spring cost is equal to or less than a threshold value is equal to or greater than a certain number, and the variation degree of the spring cost of these high score parts is within a predetermined range. When it is determined whether or not there is an affirmative determination, the process proceeds to S25 to detect the current attention area as the target area. Here, the value indicating the variation degree of the spring cost can be calculated as an average value and a variance value of the spring cost. For example, when the average value of the spring cost of a plurality of parts having a high score and a low spring cost is within a certain range, and the variance of the spring cost of the plurality of parts is within a certain range, it is determined that a target group exists. In this way, it is possible to improve the accuracy of determining whether or not a target group exists.

  (3) In the above embodiment, the spring cost is simply defined as a value representing a deviation from the reference position based on the spring model, but is not limited to this. For example, when calculating the spring cost of a part, it is determined that another part of a different type from the part to be calculated and the positional relation with the part to be calculated is appropriate as the positional relation in the same target When other parts are present, the spring cost may be calculated lower than when the other parts are not present. Specifically, in the example shown in FIG. 10, the head part HP1 is at an appropriate position based on the head part (a position based on the spring model, specifically, within a predetermined range based on the position). Since the left arm part LP1 and the right arm part RP1 exist, the spring cost is reduced. Similarly, the head part HP2 also has a left arm part LP2 and a right arm part RP2 at appropriate positions with respect to the head part, so that the spring cost is reduced. As a result, the spring costs of the other head parts HP3, HP4, and HP5 are relatively high, erroneous detection is suppressed, and determination accuracy can be improved.

  (4) Also, for example, when calculating the spring cost of a part, other parts of the same type as the part to be calculated and the positional relationship with the part to be calculated have other targets constituting the group If there is another part that is determined to be appropriate as a positional relationship with the part (for example, satisfying the periodicity peculiar to the group), the spring cost is reduced compared to the case where the other part does not exist. You may calculate low. For example, when a plurality of targets are arranged in the horizontal direction, each part of the target is likely to exist with a certain periodicity in the horizontal direction (for example, with a period corresponding to the horizontal width of the target). Lower the spring cost of parts that exist in different locations. In the example shown in FIG. 11, positions SP1 and SP2 shifted to the left and right by the width of the attention area FR (the shoulder width of the target) are set with respect to the head reference position BP, and the positions SP1 and SP2 The spring cost of the head parts HP1 and HP2 existing within a predetermined range based on the position is reduced. As a result, the spring costs of the other head parts HP3, HP4, and HP5 are relatively high, erroneous detection is suppressed, and determination accuracy can be improved.

  (5) In the above embodiment, when there are a certain number or more of high score parts having a spring cost within a certain range for each region of interest, the region of interest is detected as a target region. However, the present invention is not limited to this. Absent. For example, as shown in FIG. 12, a region (for example, a circumscribed rectangle) including a plurality of high score parts HP of the same type within a predetermined range and the attention region FR may be determined as the target region TR. . Thereafter, as described above (FIG. 7), the target regions TR detected for each type of parts are integrated (grouped) according to the position, and the circumscribed rectangles of the target regions TR belonging to the group are converted into target groups. Is detected as a collective region GR, which is a region where is present. In this way, it is possible to detect the target area in which the position of the high score part is taken into account.

  (6) In the above embodiment, the attention area is sequentially moved to each position on the image (entire image area), but the present invention is not limited to this. For example, an image may be scanned with a model of the entire target body, matching may be performed at each position, and a region having a high score may be detected in advance, and then a position where an attention region should be set may be narrowed down. In this way, the processing load for detecting the target area can be reduced.

  (7) In the above embodiment, the configuration including the distance measuring device 31 that detects the distance to the object in front of the vehicle is exemplified. However, since the distance to the object can be estimated based on the image, the distance measurement is not necessarily performed. There is no need to provide the device 31. For example, the distance to the target can be estimated based on the lower end position of the target area in the image (the position of the target's feet). However, according to the configuration provided with the distance measuring device 31, there is an advantage that the distance to the object can be detected with high accuracy, and vehicle control with higher accuracy is possible.

  (8) In the above embodiment, the parts of the head, left arm, right arm, torso, left leg and right leg are exemplified as the target parts. However, the number of parts and the parts to be used are limited to the above embodiment. It is not something. Moreover, a target is not limited to a pedestrian, For example, it is good also considering a vehicle as a target.

  (9) In the above embodiment, the target is detected from an image obtained by imaging the front of the vehicle. However, the present invention is not limited to this. For example, the target is detected from an image obtained by imaging the side and rear of the vehicle. May be. Moreover, you may detect a target from the image imaged other than the vehicle.

  (10) The function of each component of the above embodiment may be realized by software instead of hardware, may be realized by hardware instead of software, or realized by a combination of hardware and software. May be. Each component is conceptual and is not limited to the above embodiment. For example, the functions of one component may be distributed to a plurality of components, or the functions of a plurality of components may be integrated into one component. Further, at least a part of the configuration of the above embodiment may be replaced with a known configuration having the same function.

  (11) The above embodiment is merely an example of an embodiment to which the present invention is applied. The present invention can be realized in various forms such as a system, an apparatus, a method, a program, and a recording medium on which the program is recorded (an optical disc such as a CD-ROM or DVD, a magnetic disc, a semiconductor memory, etc.).

  DESCRIPTION OF SYMBOLS 10 ... Image processing device, 20 ... Vehicle control device, 31 ... Distance measuring device, 32 ... Imaging device, 41 ... Speaker, 42 ... Brake, 43 ... Steering, 50 ... Control part, 51 ... Target group detection part, 52 ... Collision Risk determination unit, 60 ... storage unit, 70 ... notification control unit, 80 ... vehicle control unit

Claims (15)

An image processing apparatus (10) for inputting an image captured by an imaging apparatus (32) and performing image processing for detecting a target in the image,
A part model for identifying a plurality of types of parts possessed by the target in the image, and a spring model representing a reference position of the plurality of types of the part on the target and a spring cost corresponding to the position of the part relative to the reference position are stored. Storage means (60) for
Parts detection means (S12) for detecting the parts in the image based on the similarity calculated by the matching process using the parts model;
Group detection means (S13, S14) for detecting the target group in the image based on the parts detected by the parts detection means;
With
The group detection means includes
Setting means (S21, S29) for sequentially setting a region of interest for determining the presence or absence of the target at a plurality of positions in the image;
The reference position when the target is assumed to be present in the region of interest is set based on the spring model, and a spring cost that is a value representing the degree of deviation from the reference position is detected by the parts detection unit. Spring cost calculating means (S23) for calculating each of the parts;
Determining means for determining the region of interest as a target region where the target constituting the group exists, on condition that there are a plurality of parts of the same type within a predetermined range in which the spring cost calculated by the spring cost calculating unit (S24, S25),
An image processing apparatus comprising: The image processing apparatus according to claim 1,
The determination means calculates a value representing a variation degree of the spring cost for the plurality of parts of the same type whose spring cost is within a predetermined range, and the region of interest is on condition that the calculated value is within the predetermined range. Is determined as the target region. The image processing apparatus according to claim 2,
The determination unit calculates a value using at least one of an average value and a variance value of the spring cost as a value representing a degree of variation of the spring cost. An image processing apparatus according to any one of claims 1 to 3, wherein
The determination means calculates the sum of the similarities for the plurality of parts of the same type whose spring cost is within a predetermined range, and the region of interest is on condition that the calculated sum is equal to or greater than a predetermined threshold value. Is determined as the target region. The image processing apparatus according to claim 1, wherein:
When calculating the spring cost of the part, the spring cost calculation unit is another part of the type different from the part to be calculated, and the positional relationship with the part to be calculated is the same in the target The image processing apparatus, wherein when there is another part determined to be appropriate as a positional relationship, the spring cost is calculated lower than when the other part does not exist. The image processing apparatus according to claim 1, wherein:
When calculating the spring cost of the part, the spring cost calculation means is another part of the same type as the part to be calculated, and the positional relationship with the part to be calculated constitutes a group When there is another part that is determined to be appropriate as the positional relationship with the part of the target, the spring cost is calculated lower than when the other part is not present. An image processing apparatus. An image processing apparatus according to any one of claims 1 to 6, wherein
The part detection means calculates the similarity for a plurality of partial areas in the image, and detects the partial area having the similarity equal to or higher than a predetermined threshold as the part represented by the part model. An image processing apparatus. An image processing apparatus according to any one of claims 1 to 7,
The image processing apparatus according to claim 1, wherein the determination unit determines, as the target area, an area including the plurality of parts of the same type whose spring cost is within a predetermined range and the attention area. An image processing apparatus according to any one of claims 1 to 8, wherein
The group detection means further comprises integration means (S14) for integrating the target areas determined for each type of part by the determination means and detecting a group area where the target group exists. An image processing apparatus. The image processing apparatus according to claim 9,
The image processing apparatus characterized in that the integration unit detects, as the collective region, a region including a plurality of the target regions that overlap each other. The image processing apparatus according to claim 10,
The integration unit detects, as the collective region, a region including a plurality of target regions that overlap each other for each group classified according to a distance from the imaging device. An image processing apparatus according to any one of claims 9 to 11,
An image processing apparatus, further comprising: an evaluation value calculation unit (S15) that calculates an evaluation value of the group region detected by the group detection unit. The image processing apparatus according to claim 12,
The image processing apparatus, wherein the evaluation value calculation means calculates the evaluation value based on a sum of the similarities of the parts used for determining the target area included in the collective area. The image processing apparatus according to claim 12 or 13, which inputs an image captured by an imaging apparatus mounted on a vehicle,
A vehicle control device (20) that performs at least one of notification control to the vehicle occupant and collision avoidance control of the vehicle as control based on the evaluation value of the collective region calculated by the image processing device; ,
A vehicle control system comprising: The vehicle control system according to claim 14,
The vehicle control system varies the contents of the control according to an evaluation value of the collective region.

Images