JP7373972B2 - object identification device - Google Patents

Description

The present invention relates to an object identification device that identifies the type of object from an image.

In order to support automobile safety and realize autonomous driving, there is a need for image sensors to have the ability to identify objects in the surroundings (for example, in front). For example, in order to apply the brakes at the appropriate time for each detected object to avoid a collision, it is necessary to identify objects with different speeds, such as pedestrians and motorcycles, and perform brake control appropriate for each type. be.

A commonly used method for identifying the type of object is to use a classifier created by learning a large number of images of specific objects. However, since this method learns apparent feature patterns on images, it is difficult to accurately identify objects with similar shapes.

Due to these characteristics, conventional image recognition has difficulty in distinguishing between, for example, a person walking (hereinafter referred to as a pedestrian) and a person riding a two-wheeled vehicle such as a bicycle or motorcycle (hereinafter referred to as a two-wheeled vehicle).

For this reason, conventional methods have been used to identify the types of objects, such as pedestrians and two-wheeled vehicles, based on speeds calculated by optical flow.

For example, in the image recognition device disclosed in Patent Document 1, "a group of dictionaries having a plurality of dictionaries storing templates of subjects and an optical flow vector in an image are extracted, and the optical flow vectors are similar to each other. dictionary selection means for extracting a moving object region by extracting pixel points, and selecting a dictionary to be used for object recognition from the dictionary group based on the moving speed; and recognition means for recognizing the subject by comparing templates stored in the dictionary selected by the dictionary selection means.''

As mentioned above, this patent document 1 discloses a method of recognizing an object by calculating the speed of the object based on the optical flow calculated from the image.

Patent No. 4692344

However, with the conventional method described in Patent Document 1, etc., there are problems in that when the object is stationary, the speed cannot be determined and identification cannot be performed, and multiple frames are required to determine the speed. There was a problem that a time delay occurred in identification.

The present invention has been made in view of the above-mentioned circumstances, and its purpose is to be able to identify the type of object even when it is a stationary object, and to improve the performance of identifying objects that are similar in appearance. An object of the present invention is to provide an object identification device.

The features of the present invention for solving the above problems are as follows, for example.

The object identification device of the present invention is an object identification device that identifies the type of object from an image, and includes an object detection unit that detects an object region where an object exists from the image, and an object distance that calculates a distance distribution of the object region. The apparatus is characterized by comprising an information calculation section and an object identification section that identifies the type of the object based on the distance distribution.

According to the present invention, by using the distance distribution of the object region, it is possible to identify the type of the target object even if it is a stationary object, and it is possible to improve the performance of identifying similar-looking targets.

Problems, configurations, and effects other than those described above will be made clear by the description of the embodiments below.

FIG. 1 is a block diagram showing the configuration of a first embodiment of an object identification device according to the present invention. It is a figure showing an example of an image acquired from a camera. 3 is a diagram showing object regions of pedestrians and two-wheeled vehicles detected from the image of FIG. 2. FIG. It is a figure explaining the example of the distance information acquired with respect to the pedestrian's object area by the distance measurement part. FIG. 3 is a diagram illustrating an example of distance information acquired with respect to an object region of a two-wheeled vehicle by a distance measurement unit. FIG. 3 is a diagram illustrating from the side a situation in which distance information of a pedestrian area is measured. FIG. 3 is a diagram showing from the side a situation in which distance information in a two-wheeled vehicle area is measured. It is a diagram showing distance information of a pedestrian area as a frequency distribution with the horizontal axis representing distance. FIG. 3 is a diagram showing distance information in a two-wheeled vehicle area as a frequency distribution with distance as the horizontal axis. FIG. 7 is a diagram showing distance information obtained by extracting only the measured distance of the object itself from the distance information of a pedestrian area. FIG. 7 is a diagram showing distance information obtained by extracting only the measured distance of the object itself from the distance information of a two-wheeled vehicle area. It is a figure which shows the example (2nd Example) of the distance information acquired with respect to the pedestrian's object area by the distance measurement part. FIG. 7 is a diagram showing an example (second example) of distance information acquired with respect to an object region of a two-wheeled vehicle by a distance measurement unit. FIG. 3 is a block diagram showing the configuration of a third embodiment of an object identification device according to the present invention. It is a flowchart figure showing the flow of processing by an object identification device of a third example.

Embodiments of the present invention will be described below with reference to the drawings and the like. The following explanations show specific examples of the contents of the present invention, and the present invention is not limited to these explanations. Changes and modifications are possible. Further, in all the figures for explaining the present invention, parts having the same functions are given the same reference numerals, and repeated explanation thereof may be omitted.

[First example]
FIG. 1 is a block diagram showing the configuration of a first embodiment of an object identification device according to the present invention.

The object identification device 10 shown in FIG. 1 is mounted on, for example, a vehicle (not shown), and identifies an area where an object exists (hereinafter referred to as an object area) from an image (an image taken around the vehicle) acquired from a camera 9 serving as an image acquisition means. an object detection unit 1 that detects an object, a distance measurement unit 2 that acquires distance information of at least two points from the detected object area, and an object distance information that calculates statistical information such as distance distribution from the acquired distance information. It has a calculation unit 3 and an object identification unit 4 that identifies the type of object to be detected from the calculated distance distribution.

Note that the object identification device 10 is configured as a computer including a processor such as a CPU (Central Processing Unit), and memory such as a ROM (Read Only Memory), a RAM (Random Access Memory), and an HDD (Hard Disk Drive). . Each function of the object identification device 10 is realized by a processor executing a program stored in a ROM. The RAM stores data including intermediate data of calculations by programs executed by the processor.

The components of the first embodiment will be explained in order.

The object detection unit 1 is a means for detecting an object area from an image obtained from the camera 9. This process is a process of detecting a rigid body region from an image, and various means can be used. For example, when a stereo camera is used as a camera for acquiring images, the distance on the image can be determined based on parallax. The object area can be determined by grouping adjacent areas on the image that are close to each other.

The distance measuring section 2 is a means for measuring the distance of the object area determined by the object detecting section 1. For example, when using a stereo camera, the distance of an object area can be determined based on parallax.

The object distance information calculation unit 3 is a means for calculating distance information of an object area based on the distance measured by the distance measurement unit 2. This means is a means for calculating distance information, which serves as a criterion for determining the shape of the object in the depth direction, from distance information measured at a plurality of points acquired within the object region.

The distance information that serves as a criterion for determining the shape in the depth direction includes, for example, distance dispersion that indicates the distance distribution within the object region. When the distance dispersion is small, the unevenness of the shape of the object in the depth direction is small, and it can be said that the object region is planar. On the other hand, when the dispersion of distance is large, the shape of the object in the depth direction has large irregularities, and it can be said that the object region is not planar. In other words, a difference in the shape of the object to be detected in the depth direction can be detected depending on the magnitude of the dispersion of the distance of the object region.

The object identification unit 4 is a means for identifying the type of object based on the distance information of the object area calculated by the object distance information calculation unit 3, using a preset threshold. When distance dispersion is used as distance information, if the dispersion is larger than a preset threshold, the object is uneven in the depth direction, and if the dispersion is smaller than the preset threshold, it is a flat object with less unevenness in the depth direction. Let it be an object.

The actual processing flow of the object identification device 10 according to this embodiment will be explained using a specific example. In this embodiment, a stereo camera is used as an image acquisition means, and an example is shown in which a distance corresponding to the position of an acquired image can be acquired.

FIG. 2 is a diagram showing an example of an image acquired from a camera. In the image 21 of FIG. 2, a pedestrian 22 and a two-wheeled vehicle 23 are present.

The object detection unit 1 detects an object area from the image 21. There are various ways to implement the object detection unit 1. For example, when a stereo camera is used as the image acquisition means, distance information on the screen is acquired using the parallax that can be acquired from the stereo camera. Object regions can be extracted by grouping regions. FIG. 3 shows object regions 31 and 32 of a pedestrian 22 and a two-wheeled vehicle 23 detected from the image 21 in FIG. 2. Note that the object regions 31 and 32 may be rectangular as shown in FIG. 3, or may be irregular regions obtained from parallax or distance. To facilitate calculation processing, it is generally treated as a rectangle.

The distance measuring section 2 is a means for acquiring distance information within the object area detected by the object detecting section 1. FIG. 4 shows an example of the distance information acquired by the distance measuring unit 2 with respect to the object area 31 (hereinafter sometimes referred to as pedestrian area 31) of the pedestrian 22, and FIG. An example of distance information acquired for the object region 32 (hereinafter sometimes referred to as the two-wheeled vehicle region 32) is shown below. 41 in FIG. 4 is distance information for the pedestrian area 31 in FIG. 3, and 51 in FIG. 5 is distance information for the two-wheeled vehicle area 32 in FIG. The cells shown in 41 and 51 are pixels or groups of pixels in the image, and the numerical value in the cell indicates the distance of that position from the target camera. Here, distance values in meters are shown as an example. Distance information 41 corresponding to the pedestrian area 31 and distance information 51 corresponding to the two-wheeled vehicle area 32 each measure distances at 136 cell positions (17 cells vertically and 8 cells horizontally). Areas 43 and 44 in the distance information 41 correspond to areas where pedestrians exist in the pedestrian area 31, and distances such as 28.9 m are measured. On the other hand, since the area indicated by 47 in the distance information 41 is a background area, a distance such as 70.0 m, which is farther than the distance measured in 43, is measured. Similarly, the areas indicated by 52 and 53 in the distance information 51 correspond to the area where the two-wheeled vehicle exists in the two-wheeled vehicle area 32, and the distance in the area 52 of the upper body of a person riding the two-wheeled vehicle is approximately 28.9 m. A distance of approximately 28.0m was measured in 53 areas at the front of the lower two-wheeled vehicle. Further, the area indicated by 54 in the distance information 51 is a background area, and a distance such as 70.0 m is measured.

FIGS. 6A and 6B show a side view of a situation in which distance information 41 for the pedestrian area 31 and distance information 51 for the two-wheeled vehicle area 32 are measured. In FIG. 6A, 61 is a pedestrian, and 67 is the camera that photographed 31 and 41 in FIG. Here, 62 and 63 indicate the measured distances of the region 43 and the region 44 in FIG. 4, respectively. In the case of a pedestrian 61, there is a difference in the distance between the distance 62 near the face and the distance 63 of the feet pulled back while walking, but the area 69 of the feet pulled back is shown by the broken line in Figure 4. This is the range of six cells shown in area 49, and is 6÷136≈4.4% when viewed from the entire pedestrian area, which is a relatively narrow range.

On the other hand, in FIG. 6B, 64 is a two-wheeled vehicle, and 67 is the camera that photographed 32 and 51 in FIG. Here, 65 and 66 indicate the measured distances of areas 52 and 53 in FIG. 5, respectively. In the case of a two-wheeled vehicle 64, there is a difference in distance between the distance 65 that measures the area 52 near the face of the person riding the two-wheeled vehicle and the distance 66 that measures the area 53 that is the front wheel of the two-wheeled vehicle. The area 68 that is close to the camera 67, such as the wheel or cage of a motorcycle, is the area 58 indicated by the broken line in FIG. %, and areas with different distances are measured over a relatively wide range.

The object distance information calculation unit 3 is a means for calculating distance information indicating depth characteristics of the object region based on the distance information acquired by the distance measurement unit 2. As an example of distance information indicating depth characteristics, an example will be explained in which the distance variance of an object area is calculated.

FIG. 7 shows the distance information of the pedestrian area shown at 41 in FIG. 4 as a frequency distribution with the horizontal axis representing the distance. The vertical axis is the number of pixels. The distance distribution of the entire distance information 41 of the pedestrian area 31 is distributed over a wide range. This is because the distance information for the pedestrian area 41 in FIG. 4 includes a background area as shown in 47 and an area where the parallax cannot be obtained and the value is 0 as shown in 48. In the example of FIG. 7, the distances are distributed from 0 m to 70 m.

When calculating the variance, it is possible to calculate the variance using all the distance information for the pedestrian area shown in 41, but the distance information in 41 includes the background distance other than the object area, and the distance cannot be calculated accurately. Since invalid distances that were not found are included, by calculating the variance only for measured distances that are likely to be the distance of the target object (pedestrian in the case of 41) in the object area, more accurate depth distance information of the target object can be obtained. It is thought that it is possible to calculate In order to extract the measured distance of the object itself from the object region, for example, the following method is adopted.

First, a threshold value for the distance range of the object is set based on the depth of the object to be identified. The depth of pedestrians and two-wheeled vehicles targeted in this embodiment is approximately 2 m at maximum, and can be set to approximately 3 m even taking into account errors. Assuming that the range of 1.5m before and after the peak position of the distance distribution of objects in the object area is the distance range of the object, distances measured beyond that range should be treated as background distances or invalid distances due to failed measurements. I can do it. A method of calculating the variance based on the distance of the object itself will be explained using the distance distribution in the pedestrian area in FIG. 7 as an example. In the distance distribution shown in FIG. 7, the position of the distance with the maximum frequency is shown at 71. The peak distance in the distance information of pedestrian area 41 in Figure 4 is considered the representative distance of the pedestrian area, and the peak distance (representative distance) of 71 is used as the reference, and only distances included in a certain range before and after are considered. The variance shall be calculated as follows. Since the representative distance is 28.9m, only the measured values in the range of 27.3m to 30.4m, which is a range of 1.5m before and after it, are used to calculate the variance. In FIG. 7, the distance range shown at 74 is a distance range of 1.5 m in front and behind the peak distance 71, which is considered to be a pedestrian area measured. On the other hand, a distance range 72 of 74 or less and a distance range 73 of 74 or more are distance ranges that are considered to be background distances or invalid distances that cannot be measured. That is, the object distance information calculation unit 3 excludes distance information in distance ranges 72 and 73 with a large distance difference from the peak distance (representative distance) 71 of the object region, and calculates the variance of distances indicating the distance distribution.

Similarly, FIG. 8 shows the distance information of the two-wheeled vehicle area shown at 51 in FIG. 5 as a frequency distribution with the horizontal axis representing the distance. In the distance distribution shown in FIG. 8, the position of the distance with the maximum frequency is shown at 81. The peak distance in the distance information of the motorcycle area 51 in Figure 5 is considered the representative distance of the motorcycle area, and the peak distance (representative distance) of 81 is used as the reference, and the distribution is performed only on distances that are included in a certain range before and after. shall be calculated. Since the representative distance is 28.9m, only the measured values in the range of 27.3m to 30.4m, which is a range of 1.5m before and after it, are used to calculate the variance. In FIG. 8, the distance range shown at 85 is a distance range of 1.5 m in front and behind the peak distance 81, which is considered to be a two-wheeled vehicle area measured. On the other hand, a distance range 83 of 85 or less and a distance range 84 of 85 or more are distance ranges that are considered to be background distances or invalid distances that cannot be measured. That is, the object distance information calculation unit 3 excludes the distance information of the distance ranges 83 and 84 having a large distance difference from the peak distance (representative distance) 81 of the object region, and calculates the variance of the distances indicating the distance distribution.

9A and 9B are diagrams showing distance information obtained by extracting only the distance measured to the object itself from the distance information of the pedestrian area and the two-wheeled vehicle area based on the above method.

Distance information obtained by extracting only the distances included in the range 74 in FIG. 7 is shown at 91 in FIG. 9A, and distance information obtained by extracting only the distances included in the range 85 in FIG. 8 is shown in 92 in FIG. 9B. When the variances are calculated using only distance information excluding background information and invalid distance information, the variance of 91 in FIG. 9A is 0.0008, and the variance of 92 in FIG. 9B is 0.19. The reason why the variance of the pedestrian area is smaller than the variance of the motorcycle area is that the distance distribution of the 91 pedestrian area has a high peak as shown in 71 in Figure 7, and the pedestrian area has unevenness in the depth direction. (shape change) is small. On the other hand, the variance in the motorcycle area is larger than the variance in the pedestrian area.The reason for the distance distribution of the 92 motorcycle areas is that the peak shown at 81 in Figure 8 is lower than that for the pedestrian area, and the peak shown at 82 is lower than that for the pedestrian area. This is due to the fact that distances are measured even at close distances. This is because the front wheel and basket portion of the two-wheeled vehicle shown at 58 in FIG. 5 are measured closer to each other when viewed from the camera side than the body area of the person riding the two-wheeled vehicle.

The object identification unit 4 is a means for identifying the type of object from the distance information calculated by the object distance information calculation unit 3. Specifically, the value of the variance calculated by the object distance information calculation unit 3 is used as an index indicating the size of unevenness in the depth of the object area, and based on a preset threshold, the variance is larger than the threshold. If the variance is smaller than a threshold value, the vehicle is determined to be a two-wheeled vehicle with large irregularities in the depth direction, and if the variance is smaller than a threshold value, the vehicle is determined to be a pedestrian with small irregularities in the depth direction. The threshold value shall be a value appropriately determined through evaluation experiments, etc.

As an example, if we assume that the variance threshold is 0.01, those that exceed this are considered motorcycles, and those that do not exceed this are pedestrians, the variance of 91 pedestrians in Figure 9A is 0.0008, which does not exceed the threshold, so pedestrians , the variance of motorcycle 92 in FIG. 9B is 0.19, which exceeds the threshold, so it is determined that it is a motorcycle.

The above is the first embodiment of the object identification device 10 according to the present invention. As explained above, the object identification device 10 of the present embodiment includes an object detection unit 1 that detects an object region in which an object exists from an image acquired from a camera 9, and a distance dispersion indicating a distance distribution of the object region. and an object identification section 4 that identifies the type of the object based on the variance of distances indicating the distance distribution. Further, the object identification unit 4 detects a difference in the shape of the object, which is a detection target, in the depth direction based on the magnitude of the dispersion of the distance of the object area, and identifies the type of the object.

As a result, when the type of the target object cannot be identified from the outline alone, the type of the object can be identified with high accuracy by determining unevenness information in the depth direction of the object using distance information.

In this way, according to this embodiment, by using the distance distribution of the object region, the type of the target object can be identified even if it is a stationary object, and the performance of identifying similar-looking targets can be improved. .

[Second example]
In the second embodiment, in the object identification device 10 of FIG. 1, the distance information calculated by the object distance information calculation unit 3 is an example in which distance differences between a plurality of parts of the object, specifically, the upper and lower parts of the object are calculated. Explain.

In the second example, as shown in FIGS. 6A and 6B, the unevenness characteristics in the depth direction of pedestrians and motorcycles when viewed from the front and rear are as follows: in the case of a pedestrian, both the upper part (distance 62) and the lower part ( While there is no big difference in the distance 63), in the case of a motorcycle, the front wheel and basket part of the lower motorcycle (distance 66) is closer to the camera than the upper part (distance 65). Based on the distance distribution described above, the difference in distance between the top and bottom of the object area is calculated as distance information and used for identification.

Similarly to the first embodiment described above, the object detection section 1 detects an object area from the image acquired from the camera 9, and the distance measurement section 2 calculates the distance of the object area determined by the object detection section 1. measure.

10A and 10B show distance information measured by the distance measurement unit 2 for the pedestrian area 31 in FIG. 3 and the two-wheeled vehicle area 32 in FIG. 3. 101 in FIG. 10A is an example of distance information acquired for the pedestrian area 31, and 102 in FIG. 10B is an example of distance information acquired for the two-wheeled vehicle area 32.

The object distance information calculation section 3 calculates distance information indicating the depth characteristics of the object region (in other words, the depth direction of the object) based on the distance information (101 in FIG. 10A and 102 in FIG. 10B) acquired by the distance measurement section 2. However, in the object distance information calculation unit 3 of this embodiment, the distance of the object area 31 of the pedestrian shown at 101 in FIG. 10A is calculated as the distance information indicating the depth feature. The information (distance distribution) is divided into an upper half region 103 and a lower half region 104, and the average distance value is calculated for each region. In the example shown in FIG. 10A, of the 17 vertical cells, the area corresponding to 8 cells from the bottom cell is defined as the lower half area 104, and the area corresponding to the 8 cells above it (in other words, from 2nd to 9th cell from the top) (area for 8 cells) is the upper half area 103.

The object identification unit 4 identifies the type of object from the distance information calculated by the object distance information calculation unit 3. However, the object identification unit 4 of this embodiment uses the distance (average value) calculated by the object distance information calculation unit 3. ) is used as an index indicating the size of unevenness in the depth of the object region, and the difference (distance difference) between the upper half region 103 and the lower half region 104 is set in advance based on a preset threshold. If the threshold is exceeded, it is determined that the motorcycle has a motorcycle area at the bottom (large irregularities in the shape in the depth direction), and if the difference (difference in distance) does not exceed the preset threshold, the It is determined that there is no difference in the distance between and the lower part (the unevenness of the shape in the depth direction is small).

Similarly, the object distance information calculation unit 3 divides the distance information (distance distribution) of the object region 32 of the two-wheeled vehicle 102 in FIG. The average value is calculated, and the object identification unit 4 determines whether it is a two-wheeled vehicle or a pedestrian based on the difference between the distance average value in the upper half region 105 and the distance average value in the lower half region 106.

When the average values of the upper half area 103 and lower half area 104 of the distance information 101 of the pedestrian area 31 are calculated, they are 28.9 m and 28.9 m, respectively, and the difference in the average distance values of the upper half area 103 and the lower half area 104 is 0 m. becomes. Calculating the average values of the upper half area 105 and lower half area 106 of the distance information 102 of the motorcycle area 32 is 28.8 m and 28.5 m, respectively, and the difference between the average distance values of the upper half area 105 and the lower half area 106 is 0.3 m. It is. If the threshold for identifying two-wheeled vehicles and pedestrians in the object identification unit 4 is 0.2 m, the distance difference between the top and bottom of the distance information 101 of the pedestrian area 31 in FIG. 10A does not exceed the threshold, so pedestrians, pedestrians, Since the distance difference between the top and bottom of the distance information 102 of the two-wheeled vehicle area 32 in FIG. 10B exceeds the threshold value, it can be determined that the vehicle is a two-wheeled vehicle.

In the above example, distance information is obtained over the entire object area as shown in FIGS. 10A and 10B using the parallax of a stereo camera. In addition, although we have explained the example of calculating the difference in the average value of the distance between the upper half and the lower half, the invention is not limited to this. A similar effect can be obtained by calculating the difference in the average value from a part of the central area of the half. Further, it is not necessarily assumed that distance information can be obtained over the entire surface, and the present embodiment can be applied even when distance can be measured only at limited points on the object region using, for example, a laser sensor. For example, if it is possible to measure the distance of one or more places in the upper area 103 of the distance information 101 shown in FIG. 10A and the distance of one or more places in the lower area 104, only one place can be obtained. If not, by taking the difference between the top and bottom as a representative distance between the top and bottom, it is possible to determine whether it is a pedestrian or a two-wheeled vehicle. In other words, by taking the difference in distance (average value or representative distance) of multiple partial areas (areas containing one or more pixels) of a predetermined size that make up part of the object area, it is possible to determine whether a pedestrian or two-wheeled vehicle is It is possible to identify the type of object.

As explained above, in the object identification device 10 of the second embodiment, the object identification unit 4 detects the difference in distance between a plurality of partial regions forming a part of the object region, specifically, The type of the object is identified based on the difference in distance between the upper half region and the lower half region of the object region.

As a result, in this embodiment as well, similar to the first embodiment described above, by using the distance distribution of the object area, it is possible to identify the type of target object even if it is a stationary object, and Target identification performance can be improved.

[Third example]
FIG. 11 is a block diagram showing the configuration of a third embodiment of the object identification device according to the present invention. The third embodiment has a configuration in which an object recognition section 5 is added to the block diagram of FIG.

The object recognition unit 5 is a process that learns the characteristics of a plurality of types of objects in advance using machine learning or the like, and determines the type of the target object through recognition processing. Since machine learning learns apparent features such as the outline and texture of objects (features on two-dimensional images), it may be difficult to recognize and identify objects with similar shapes.

In the third embodiment, when the object recognition unit 5 identifies a type that is difficult to identify based only on the shape of the target object, an example will be described in which the object type is narrowed down using the object identification device 10 according to this embodiment. explain. As an example, an example will be described in which the object recognition unit 5 identifies four types of objects: vehicle, pedestrian, two-wheeled vehicle, and others (background). Among these four types of objects, pedestrians and motorcycles have similar appearance characteristics, making it difficult for the object recognition unit 5 to identify them. In the third embodiment, the object recognition unit 5 narrows down the object to three types: vehicle, pedestrian or two-wheeled vehicle, and background, and when the object is identified as a pedestrian or two-wheeled vehicle, the object identification device 10 according to this embodiment is used. to identify pedestrians and motorcycles. Further, this embodiment will be described using an example (see also the first embodiment) in which the distance variance of an object region is calculated as distance information.

FIG. 12 is a flowchart showing the flow of processing by the object identification device 10 of the third embodiment.

In S121, an image is acquired from the camera 9 as shown in FIG.

In S122, as shown in FIG. 3, the object detection unit 1 detects an object area from the image.

In S123, the reference counter i of the detected object (object region) is set to an initial value of 0.

In S124, the object recognition unit 5 performs recognition processing for the i-th object (hereinafter referred to as object i). In this S124, as described above, it is recognized from the apparent characteristics on the image which type of object i is among the four types created in advance: vehicle/pedestrian/two-wheeled vehicle/other (background).

In S125, it is determined whether the type of object i was recognized as a pedestrian or a two-wheeled vehicle in the recognition process of S124. If the type of object i is recognized as a pedestrian or a two-wheeled vehicle, the process proceeds to S127, where the object identification device 10 according to this embodiment performs identification processing. If the type of object i is recognized as a type other than a pedestrian or a two-wheeled vehicle, the process proceeds to S126.

In this embodiment, it is determined whether the type of object i is recognized as a pedestrian or a two-wheeled vehicle, but for example, if the type of object i is a vehicle (other than a pedestrian or a two-wheeled vehicle) or other (background). If the type of object i is recognized as a vehicle or other (background), the process proceeds to S126, and the type of object i is recognized as a type other than vehicle or other (background). If so, the process may proceed to S127, and the object identification device 10 according to this embodiment may perform identification processing.

In S126, if the type of object i is recognized as a vehicle or other (background) in the recognition process of S124, the recognized type (object i recognition result) is directly adopted as the type of object i and used in the process of S12b. move on.

In S127, the object distance information calculation unit 3 calculates the variance of the distance of the object area of the object i (pedestrian or two-wheeled vehicle) as object distance information based on the distance information acquired by the distance measurement unit 2.

In S128, the object identification unit 4 determines whether the value of the variance of the distance of the object area of the object i calculated in S127 is greater than or equal to a preset threshold. If the variance is greater than or equal to the threshold, proceed to S129; if the variance is less than the threshold, proceed to S12a.

In S129, if the value of the distance variance of object i is equal to or greater than the threshold value, object i is determined to be a two-wheeled vehicle because the shape of the object in the depth direction has large irregularities as described in the first embodiment. do.

In S12a, if the value of the distance variance of object i does not exceed the threshold value, as described in the first embodiment, the unevenness of the shape of the object in the depth direction is small, so object i is It is determined that

In S12b, the reference counter i of the object (object region) is incremented.

In S12c, it is determined whether the reference counter i has exceeded the maximum number of three-dimensional object candidate regions (the number of objects). If not, it is determined that there is an unprocessed three-dimensional object candidate region, and the process proceeds to S125. If it exceeds the limit, this process ends.

The above is the flow of processing by the object identification device 10 of the third embodiment. By combining recognition using machine learning and identification using the depth and distance features of objects according to this embodiment, we can identify gaits that are difficult to identify based on apparent features such as contours and textures, but that can be identified using depth information. This makes it possible to identify motorcyclists and motorcycles.

In addition, in this example (the same applies to the first and second examples described above), an example of a pedestrian and a two-wheeled vehicle is used, but as described above, although their apparent shapes from the front and rear are similar, , this embodiment can be similarly applied to objects that have different shapes in depth. For example, it is applicable not only to two-wheeled vehicles, but also to the identification of tricycles, rickshaws, and pedestrians. It can also be applied to distinguish between humans, cows, horses, etc. Further, the present embodiment can also be applied to identification by type (car model, etc.) of vehicles that have similar apparent characteristics.

As explained above, the object identification device 10 of the third embodiment includes an object detection unit 1 that detects an object region in which an object exists from an image acquired from a camera 9, and a an object recognition unit 5 that recognizes the type of the object from its characteristics, and a predetermined object (for example, a vehicle or background) that includes a predetermined object (for example, a pedestrian or a two-wheeled vehicle) or a predetermined object (for example, a vehicle or a background) among the objects recognized by the object recognition unit 5; an object distance information calculation unit 3 that calculates a distance distribution of the object area that does not include , and an object identification unit 4 that identifies the type of the object in the object area based on the distance distribution calculated by the object distance information calculation unit 3. and has.

With this, for example, by combining two-dimensional recognition using machine learning and three-dimensional identification using depth and distance features of the object, it is possible to quickly and accurately identify the type of object.

In this way, in this embodiment as well, as in the first and second embodiments described above, by using the distance distribution of the object area, it is possible to identify the type of object even if it is a stationary object. It is possible to improve the identification performance of objects that have similar appearance.

Note that the present invention is not limited to the above-described embodiments, and includes various modifications. For example, the embodiments described above are described in detail to explain the present invention in an easy-to-understand manner, and the present invention is not necessarily limited to having all the configurations described. Furthermore, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, delete, or replace a part of the configuration of each embodiment with the configuration of other embodiments.

Further, each of the above-mentioned configurations, functions, processing units, processing means, etc. may be partially or entirely realized in hardware by designing, for example, an integrated circuit. Furthermore, each of the above configurations, functions, etc. may be realized by software by a processor interpreting and executing a program for realizing each function. Information such as programs, tables, files, etc. that implement each function can be stored in a memory, a storage device such as a hard disk, an SSD (Solid State Drive), or a recording medium such as an IC card, an SD card, or a DVD.

Further, the control lines and information lines are shown to be necessary for explanation purposes, and not all control lines and information lines are necessarily shown in the product. In reality, almost all components may be considered to be interconnected.

1 Object detection section
2 Distance measurement section
3 Object distance information calculation unit
4 Object identification section
5 Object recognition unit (third embodiment)
10 Object identification device
31 Object area (pedestrian area)
32 Object area (motorcycle area)

Claims (6)

An object identification device that identifies the type of object from an image,
an object detection unit that detects an object region in which the object exists from the image;
a distance measuring unit that calculates distance information for each pixel occupying the object area or for each group of adjacent pixels in the image;
an object distance information calculation unit that calculates a distance distribution of the object area based on the distance information obtained by the distance measurement unit;
an object identification unit that identifies the type of the object based on the distance distribution ,
The object distance information calculation unit calculates the distance distribution based only on distance information included in a predetermined range, with a representative distance of the distance information calculated for the object region as a reference. Device. The object identification device according to claim 1,
an object recognition unit that recognizes the type of the object from the features on the image;
The object distance information calculation unit calculates a distance distribution of the object area that includes a predetermined object or does not include a predetermined object among the objects recognized by the object recognition unit,
The object identification device is characterized in that the object identification unit identifies the type of the object in the object area based on the distance distribution calculated by the object distance information calculation unit. The object identification device according to claim 1,
The object identification unit is characterized in that the object identification unit identifies the type of the object by detecting a difference in shape in the depth direction of the object, which is a detection target, based on the magnitude of distance dispersion of the object area. Device. The object identification device according to claim 1,
The object identification device is characterized in that the object identification unit identifies the type of the object based on a difference in distance between a plurality of partial regions forming part of the object region. The object identification device according to claim 4,
The object identification device is characterized in that the object identification unit identifies the type of the object based on a difference in distance between an upper half region and a lower half region of the object region. The object identification device according to claim 1,
The object identification device is characterized in that the object distance information calculation unit sets a distance with a maximum frequency among the distance information obtained for the object region as the representative distance.

Images