JP2020087037A - Identification apparatus and identification method - Google Patents

Abstract

To determine existence of an abnormal object based on an image with satisfactory accuracy.SOLUTION: The present invention is directed to an identification apparatus for determining existence of an abnormal object included in an image. The identification apparatus has a first type identifier for estimating a label of each of a plurality of images included in an input image based on a learning result of characteristics of pixels included in a first image which is an image having no abnormal object and for outputting a first probability distribution representing probability distribution of the plurality of labels for every pixel, a second type identifier for generating a second probability distribution from the first probability distribution based on the learning results of relationship of the labels at the surrounding of a plurality of pixels included in the first image, and a determining means for determining existence of the abnormal object in the second image based on discrepancy between the first probability distribution obtained by inputting the second image into the first type identifier and the second probability distribution obtained by inputting the first probability distribution into the second type identifier.SELECTED DRAWING: Figure 1A

Description

The present invention relates to an image-based abnormality determination technique.

As a method of detecting an obstacle on the road (an unknown object left on the road that hinders vehicle traveling), for example, a method using a millimeter wave radar as described in Non-Patent Document 1 has been proposed. There is. According to the technique described in Non-Patent Document 1, it is possible to stably detect an obstacle on the road regardless of the time zone and the weather and even at a long distance.

Further, as a method for detecting an obstacle on the road, for example, a method using a camera as described in Non-Patent Documents 2 and 3 has been proposed. In the technique described in Non-Patent Document 2, a road obstacle is detected by a discriminator learned based on an image. Further, in the technique described in Non-Patent Document 3, the phases in the time and space directions are matched between the past and present images, and a road obstacle is detected from the difference between them.

There is also an attempt to detect an obstacle on the road by combining a camera and a millimeter wave radar as described in Non-Patent Document 4.

Takayuki Inaba, Tetsuro Kirimoto, "In-vehicle millimeter-wave radar, Automotive Technology vol.64, no.2, pp.74-79, Feb.2010.89. Vehicle detection by 2-step AdaBoost using Joint HOG feature, 2009, Transactions of the Institute of Electronics, Information and Communication Engineers, Ozaki and Fujiyoshi et al. (Chubu University) A Study on Unspecified Obstacle Detection by Spatio-Temporal Difference from Past Vehicle Images, 2012, Symposium on Image Recognition and Understanding (MIRU2012), Kutoku and Murase et al. (Nagoya University) Integrating Millimeter Wave Radar with a Monocular Vision Sensor for On-Road Obstacle Detection Applications (Xi'an Jiaotong University), 2011

The method using the millimeter wave radar described in Non-Patent Document 1 has problems in detection accuracy and detection range. This is because the method using a millimeter wave radar generally has a low resolution and a narrow irradiation range of the radar, so that the size is small or it is difficult to detect an obstacle at the end of the road.

Further, in the detection of a road obstacle by machine learning as described in Non-Patent Document 2, there is a problem that an object that has not been learned in advance cannot be identified.
Moreover, the method using a camera image as described in Non-Patent Document 3 has a problem that a preceding vehicle, an oncoming vehicle, and other moving objects are erroneously determined as road obstacles.
The method described in Non-Patent Document 4 can detect an obstacle on the road with high accuracy, but since the millimeter-wave radar and the camera are used together, the cost for installing and operating the device increases.

The present invention has been made in consideration of the above problems, and an object of the present invention is to provide an identification device that can accurately determine the presence of an abnormal object based on an image.

An identification device according to one aspect of the present invention,
An identification device for determining the presence of an abnormal object included in an image, wherein a plurality of images included in an input image are obtained based on a result of learning characteristics of pixels included in a first image that does not include the abnormal object. A first type classifier that estimates a label for each of the pixels and outputs a first probability distribution that represents the probability distribution of a plurality of labels for each pixel, and a plurality of pixels included in the first image Based on the result of learning the relationship of labels in the vicinity of, from the first probability distribution, a second type discriminator that generates a second probability distribution, and a second image to the first type discriminator. Based on the dissociation degree of the first probability distribution obtained by input, and the second probability distribution obtained by inputting the first probability distribution to the second type discriminator, the Determination means for determining the presence of the abnormal object in the second image.

Further, the identification method according to the present invention,
An identification method for determining the presence of an abnormal object included in an image, wherein a plurality of images included in an input image are obtained based on a result of learning characteristics of pixels included in a first image that does not include the abnormal object. A first identification step of estimating a label for each of the pixels, and outputting a first probability distribution representing the probability distribution of a plurality of labels for each pixel; and a plurality of pixels included in the first image. Based on the result of learning the relationship of labels in the vicinity of, from the first probability distribution, a second identification step of generating a second probability distribution, and a second image in the first identification step. Based on the dissociation degree of the first probability distribution obtained by processing, and the second probability distribution obtained by processing the first probability distribution in the second identification step, the A determination step of determining that the abnormal object is present in the second image.

Further, an identification device according to another aspect of the present invention,
An identification device for determining the presence of an abnormal object included in an image, wherein a plurality of images included in an input image are obtained based on a result of learning characteristics of pixels included in a first image that does not include the abnormal object. A first type discriminator that estimates a label for each of the pixels and outputs a first probability distribution that represents the probability distribution of a plurality of labels for each pixel, and identifies the first image by the first type discriminator. Type that generates a second probability distribution from the first probability distribution based on the result of learning the tendency of the first probability distribution output by the first type discriminator when input to the device Discriminator, the first probability distribution obtained by inputting a second image to the first type discriminator, and obtained by inputting the first probability distribution to the second type discriminator Determination means for determining the presence of the abnormal object in the second image based on the dissociation degree of the second probability distribution.

According to the present invention, it is possible to provide an identification device that can accurately determine the presence of an abnormal object based on an image.

The functional block diagram of the identification device which concerns on 1st embodiment. The functional block diagram of the identification device which concerns on 1st embodiment. An example of the image acquired by the image acquisition unit. An example of an image in which a label is added to the input image. The figure explaining the data which a 1st discriminator outputs. The figure explaining the data which the second discriminator outputs. The figure explaining the decision tree by a random forest. The figure explaining the co-occurrence of the label which a 2nd discriminator learns. The flowchart of the process performed in 1st embodiment (learning mode). The flowchart of the process performed in 1st embodiment (evaluation mode). An example of a visual image of a road obstacle. The functional block diagram of the identification device which concerns on 2nd embodiment. The functional block diagram of the identification device which concerns on 2nd embodiment. The figure explaining the similarity of the label which a 3rd discriminator learns. The flowchart of the process performed in 2nd embodiment (learning mode). The flowchart of the process performed in 2nd embodiment (evaluation mode).

An identification device according to the present invention is based on an image obtained by capturing an image of the front of a vehicle, and a road obstacle (an unknown object that may hinder the running of the vehicle in front of the vehicle. It is also a device for identifying the presence or absence of an object. Specifically, machine learning is performed using an image obtained by capturing a situation where there is no road obstacle, and an image in which the presence or absence of a road obstacle is unknown is evaluated using the learning result.

An identification device according to the present invention estimates a label for each of a plurality of pixels included in an input image, based on a result of learning characteristics of pixels included in a first image that is an image that does not include an abnormal object. Then, a first type discriminator that outputs a first probability distribution that represents the probability distributions of a plurality of labels for each pixel is used.

On the other hand, as described above, there is a problem that the classifier that estimates the label based on the feature of the pixel cannot detect the unlearned object.
In order to solve this problem, the identification device according to the present invention uses (1) the first probability distribution based on the result of learning the relationship of labels around a plurality of pixels included in the first image. , A second type discriminator that generates a second probability distribution is also used, and (2) the first probability distribution obtained by inputting a second image to the first type discriminator, and the first It is determined that the abnormal object is present in the second image based on the dissociation degree of the second probability distribution obtained by inputting the probability distribution of the second type discriminator to the second type discriminator.

The relationship between the labels may be co-occurrence or similarity of the labels.

The second type discriminator is a discriminator that has learned label relationships (eg, label co-occurrence and similarity) around a pixel in an image that does not include an abnormal object (first image). is there. The second type classifier is configured to be able to output the second probability distribution to which the above learning result is applied, with the first probability distribution as an input.
The second probability distribution evaluated by the second type discriminator deviates from the first probability distribution, which means that the first type discriminator does not include an abnormal object in the image (first image). This means that a probability distribution that has a tendency different from the probability distribution that is output is output. Therefore, the existence likelihood of an abnormal object in the second image can be estimated based on the dissociation degree between the first probability distribution and the second probability distribution.

Further, the second type classifier may be a classifier that has learned the first probability distribution obtained by inputting the first image to the first type classifier as learning data. ..
Further, the second type discriminator, the first probability distribution obtained by inputting the first image to the first type discriminator as learning data, a plurality of pixels included in the first image. The discriminator may be a classifier that learns the similarity of the labels around the.

By learning the above-described label relationships using such learning data, the tendency of the probability distribution output by the first type classifier for the first image can be learned.

The second type discriminator uses the first label image in which the correct label is given to the first image as learning data, and the label co-occurrence around a plurality of pixels included in the first image. It may be characterized by being a classifier that has learned the sex.
The second type classifier may perform learning by using an image to which a correct answer label is added instead of the probability distribution. Even with such a configuration, the tendency of the probability distribution output by the first type classifier for the first image can be learned.

Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. The hardware configuration, module configuration, functional configuration, etc. described in each embodiment are not intended to limit the technical scope of the invention to these unless otherwise specified.

(First embodiment)
The first embodiment will be described with reference to the drawings. 1A and 1B are block diagrams showing a functional configuration of the identification device 10 according to the first embodiment.
The identification device 10 according to the present embodiment is a device that determines the presence or absence of a road obstacle included in the image based on the input image and outputs the determination result.

The classification device 10 has a plurality of classifiers and is configured to be switchable between a learning mode for learning the classifiers and an evaluation mode for evaluating an image based on the learning result. In the following description, the process of learning the classifier is referred to as a learning process, and the process of evaluating an image using the learned classifier is referred to as an evaluation process. An image used for learning is referred to as a learning image, and an image used for evaluation is referred to as an evaluation image. The learning image is an image acquired from the vehicle-mounted camera in the situation where there is no road obstacle, and the evaluation image is an image acquired from the vehicle-mounted camera in the situation where the existence of the road obstacle is unknown.
FIG. 1A is a diagram showing a data flow in the learning mode, and FIG. 1B is a diagram showing a data flow in the evaluation mode.

The identification device 10 according to the present embodiment can be realized by using a semiconductor integrated circuit (LSI). The identification device 10 includes an image acquisition unit 11, a labeling unit 12, an identification unit 13, and an abnormality determination unit 14. Further, the identification unit 13 is configured to include a first identification device 131 and a second identification device 132. These components are realized by software modules and correspond to the functions performed by the identification device 10, respectively.

The image acquisition unit 11 is a unit that acquires an image obtained by imaging the front of the vehicle. The image acquisition unit 11 acquires an image from, for example, a camera (vehicle-mounted camera) installed toward the front of the vehicle. FIG. 2 is an example of an image acquired by the image acquisition unit 11. In the present embodiment, the image acquired at a certain time t is expressed as I(t).

The label attaching unit 12 is a unit that attaches a label to the image acquired by the image acquiring unit 11.
Here, the label will be described. In the present embodiment, “what each of the plurality of pixels included in the image corresponds to” is represented by a label. FIG. 3 is an example in which a label is added to the illustrated image. The label assigning unit 12 defines M types of labels (also referred to as semantic labels) having a predetermined meaning in advance, and assigns the label m to the target image in pixel units.
For example, if M is 7, then m can be defined as:
1: Sky, 2: Nature, 3: Other vehicle, 4: Building, 5: White line, 6: Road, 7: Own vehicle In the present embodiment, the labeling unit 12 presents the input image to the user. , The label assignment result is obtained from the user. Hereinafter, the result of attaching a label to an image will be referred to as a label image. FIG. 3 is an example of the label image. Hereinafter, the label image corresponding to the image I(t) will be expressed as L(t).

Next, the identification unit 13 will be described. The discriminating unit 13 is configured to include a first discriminator 131 and a second discriminator 132.

The first discriminator 131 is the first type discriminator in the present invention, and is a discriminator that learns the relationship between the characteristics of the pixels included in the image and the plurality of labels described above. The first discriminator 131 includes a plurality of learning images (images that do not include road obstacles; hereinafter, also referred to as normal images) and a plurality of learning images that are obtained by assigning labels (correct labels) to the learning images. Label images are used for learning.

The first classifier 131 has a statistical distribution of features (color, edge strength, texture, etc.) around the coordinates (x, y) of the image I(t), and the semantic label m (m=1,...) At the coordinates. , M) by learning the relationship with them.
When the evaluation image is input, the first discriminator 131 outputs the probability distribution of semantic labels at the coordinates (x, y) of the image I(t). FIG. 4 is a diagram illustrating the data output by the first discriminator 131. FIG. 4 shows the probability distribution P _1,m of the semantic label of the image I(t) at the coordinates (x,y). In this example, the pixel at the coordinate (x, y) has the highest semantic label of 1, that is, is most likely to be empty.
The first discriminator 131 can be trained using a known learning method such as deep learning.

Note that the label image corresponding to the evaluation image can be obtained by selecting the label with the highest probability among the probability distributions output by the first discriminator 131 for each coordinate.

The second classifier 132 is the second type classifier in the present invention, and is a classifier that learns the co-occurrence of labels in a normal image. The second discriminator 132 is learned using a plurality of label images.

The second discriminator 132 calculates the semantic label m (m=1,..., M) at the coordinate (x, y) of the label image L(t) and the statistical distribution of the semantic label m around the coordinate. It is obtained by learning the relationship (that is, co-occurrence) of.
Upon inputting the probability distribution, the second classifier 132 converts the probability distribution into a label image L(t), and based on the learned co-occurrence of the label, coordinates (x, y) of the image L(t). ) Output the probability distribution of the semantic labels in. FIG. 5 is a diagram illustrating data output by the second discriminator 132. FIG. 5 shows the probability distribution P _2,m of the semantic label at the coordinates (x, y) of the label image L(t). In this example, with respect to the pixel at the coordinates (x, y), co-occurrence with surrounding pixels (for example, a pixel surrounded by pixels labeled with "empty" may be "empty") Is most likely to be empty", "Pixels in the vicinity of pixels labeled with "Sky" are likely to be "Sky" or "Natural"", etc.) Is represented.
The second classifier 132 can be trained using a known learning method such as random forest. A specific learning method will be described later.

The first discriminator 131 generates a probability distribution of labels according to the result of learning the characteristics of pixels, while the second discriminator 132 learns the co-occurrence of a plurality of labels around a certain coordinate. Generates a label probability distribution.
When the probability distribution is input by learning the second classifier 132, the probability distribution is converted into a label image, and the probability distribution of the semantic label as shown in FIG. 5 is calculated for each pixel included in the label image. An output discriminator can be obtained.

The abnormality determination unit 14 calculates the dissociation degree between the probability distribution output by the first discriminator 131 and the probability distribution output by the second discriminator, and based on the dissociation degree, road obstacles appear in the evaluation image. It is a means for calculating the degree of inclusion (abnormality).
The abnormality determination unit 14 calculates the likelihood S(x, y) indicating the likelihood of an obstacle at an arbitrary point p(x, y) of the input evaluation image I(t), and based on the calculation result, A road obstacle included in the evaluation image I(t) is detected. Specific processing contents will be described later.

Next, the process in which the second classifier 132 learns the co-occurrence of a plurality of labels included in the label image will be described in detail by exemplifying a random forest.
Random forest is a kind of group learning algorithm using a decision tree as a weak discriminator as shown in FIG. 6, and is composed of a plurality of nodes r(1,..., R) and links connecting the nodes. It The node in the highest layer is called a root node, the node in the lowest layer is a leaf node, and the other nodes are simply called nodes.

A determination condition Φ _r (r=1) for allocating a sample in the node (pixel p(x, y) randomly extracted from the semantic label image L(t)) to the left and right nodes by learning in each node. ,..., R) and the probability distribution P _2,m (r) (r=1,..., R, m=1,..., M) for the category m to be identified are stored.
The category m to be identified may be set in advance within a predetermined range (for example, 1,..., M (M=7)) as described above.

Candidates φ _c (c=1,..., C) for the judgment conditions necessary for learning the random forest are coordinates (
x, y) is set at random as two offset vectors (r _a , r _b ), and the semantic labels (m _a , m _b ) at the end points (a, b) of the vectors (ε 1, ε 1, , M) may be defined based on the co-occurrence.

A specific example will be described.
For example, as shown in FIG. _7A , it is assumed that the determination condition (r _a , r _b , 1, 2) is set at the coordinate p(x, y). In this case, the semantic label in the offset vector r _a is “empty” (m=1) and the semantic label in the offset vector r _b is “natural”.
Since it is established that (m=2), it transits to the node on the right side.
On the other hand, when the determination condition (r _a , r _b , 1, 1) is set at the coordinate p(x, y), the semantic label in the offset vector r _a is “empty” (m=1) and the offset is Since it is not established that the semantic label in the vector r _b is “empty” (m=1), the node transits to the node on the left side.

Similarly, as shown in FIG. 7 (B), the coordinates p (x, y) in the determination condition _{(r a, r b, 2,2} ) and setting the. In this case, it is established that the semantic label in the offset vector r _a is “natural” (m=2) and the semantic label in the offset vector r _b is “natural” (m=2). Transition to the node.
On the other hand, when the determination condition (r _a , r _b , 1, 2) is set at the coordinate p(x, y), the semantic label in the offset vector r _a is “empty” (m=1) and the offset is Since it is not established that the semantic label in the vector r _b is “natural” (m=2), the node transits to the left node.

Thus, candidate _{φ c (c = 1, ...} , C) of the determination condition Once obtained, after may be learned in accordance with known procedures. Learning here means the determination condition Φ _r (r=1,..., R) appropriate for each node r(1,..., R) and the probability distribution for the identification target category m (=1,..., M). P2 _,m (r) (r=1,..., R, m=1,..., M) is set.
Specifically, the determination condition Φ _r (rε1,..., R) at the r-th node is defined by the equation (1) among the determination condition candidates φ _c (c=1,..., C). The maximum reliability G(φ) may be set.

Here, Q _l (φ) is the number of p(x, y) that transits to the left side node under the determination condition φ, and Q _r (φ) is p(x, y) that transits to the right side node under the determination condition φ. The number of y), H(Q(φ)), represents the entropy for the identification target category at the predetermined node. In addition, H(Q _l (
φ)) represents the entropy for the identification target category of the sample that has transited to the node on the left side under the determination condition φ. Further, H(Q _r (φ)) represents the entropy for the identification target category of the sample that transits to the node on the right side under the determination condition φ.
Finally, the determination condition Φ _r (r at each node r(1,..., R) in the random forest
=1,..., R) and the probability distribution P _2,m (r) for the classification target category m are obtained.

The second discriminator 132 learns the “probability of the probability distribution output by the first discriminator 131 that has input a normal image” by learning the co-occurrence of labels in an image that does not include road obstacles. It can be said to be a vessel.

Next, the learning process performed by the identification device 10 according to the first embodiment will be described.
FIG. 8 is a flowchart showing the flow of processing for learning the first classifier 131 and the second classifier 132 using a plurality of images. Learning is performed using one or more learning images. The learning image is an image that does not include road obstacles. The process shown in FIG. 8 is repeatedly executed for each of the plurality of learning images. The plurality of learning images can be, for example, images I(t) acquired at a plurality of time steps.

First, in step S11, the image acquisition unit 11 acquires the learning image I(t).
Next, in step S12, the label assigning unit 12 acquires the label image L(t) corresponding to the learning image I(t). As mentioned above, the label image may be manually generated by the user of the device. The acquired label image L(t) is output to the first discriminator 131 and the second discriminator 132.
Next, in step S13, the first discriminator 131 performs learning by the above-described method based on the acquired learning image I(t) and label image L(t).
Next, in step S14, the second discriminator 132 performs learning by the method described above based on the acquired label image L(t).

Next, a method for determining the presence/absence of a road obstacle by using the learned identification unit 13 will be described.
FIG. 9 is a flowchart showing a flow of processing of evaluating the evaluation image and determining the presence of a road obstacle by using the learned identification unit 13. The evaluation image is an image in which the presence of an obstacle on the road is unknown. The evaluation image may be supplied in real time from, for example, a camera mounted on the vehicle.

First, in step S21, the image acquisition unit 11 acquires the evaluation image I′(t).
Next, in step S22, the first discriminator 131 generates a probability distribution P _1,m based on the input evaluation image I′(t).

The probability distribution P _1,m generated by the first discriminator 131 is the abnormality determination unit 14 and the second discriminator 13.
2 is output.
In step S23, first, the second discriminator 132 generates a label image L′(t) from the probability distribution P _1,m . A label m′ at an arbitrary coordinate in the label image L′(t) is a label having the maximum likelihood at the probability P _1,m . That is, it can be calculated by the equation (2).

Next, the second classifier 132 generates a label probability distribution P _2,m based on the generated label image L′(t). The probability distribution P _2,m generated by the second classifier 132 is output to the abnormality determination unit 14.

Next, in step S24, the abnormality determination unit 14 determines the first likelihood S ₁ based on the probability distribution P _1,m obtained in step S22 and the probability distribution P _2,m obtained in step S23. To calculate. The first likelihood S ₁ can be obtained by the sum of squared differences, as in Expression (3).

The first likelihood S ₁ (x, y) is the dissociation degree of the discrimination results of the first discriminator 131 and the second discriminator 132 at the coordinates (x, y) in the evaluation image I′(t). Show. The difference between the probability distribution output by the first discriminator 131 and the probability distribution output by the second discriminator 132 means that the result of discrimination based on the characteristics of the pixel and the label of the result. It means that the result of considering co-occurrence is different.
The fact that the identification result considering the label co-occurrence in the normal image deviates from the original identification result means that the probability distribution output by the first discriminator 131 is a probability distribution obtained when the normal image is input. Means having different tendencies. Therefore, the likelihood that an unknown object exists in the evaluation image can be estimated based on the magnitude of the first likelihood S ₁ .

The result obtained in this way is transmitted to a system that utilizes the detection result of a road obstacle, for example, a program in a vehicle driving support system.

In a system that utilizes the detection result of a road obstacle, for example, a region in which the first likelihood S ₁ is larger than a predetermined threshold value is specified and superimposed on the evaluation image, so that the road obstacle can be clearly indicated. it can. The predetermined threshold may be set, for example, to a value that minimizes the intra-class variance and maximizes the inter-class variance in the first likelihood S ₁ (discriminant analysis method).

With this configuration, as shown in FIG. 10A, it is possible to divide the area (white area) where the presence of a road obstacle is suspected and the other area (black area). Further, by using this result, as shown in FIG. 10B, it is possible to perform image processing that emphasizes the candidate area of the road obstacle.
Furthermore, the area detected as a road obstacle may be transmitted to another device (for example, a server device or a rear vehicle) via a network to call attention.

(Second embodiment)
In the first embodiment, the first discriminator that has learned the characteristics of pixels and the second discriminator that has learned the co-occurrence of labels around the target coordinates are used together to make a road obstacle determination. On the other hand, in the second embodiment, a third discriminator (second type discriminator in the present invention) that has learned the similarity of the feature vectors around the target coordinates is further used to determine the road obstacle. It is an embodiment.

11A and 11B are block diagrams showing the functional configuration of the identification device 10 according to the second embodiment. FIG. 11A is a diagram showing a data flow in the learning mode, and FIG. 11B is a diagram showing a data flow in the evaluation mode.

In the first embodiment, the second classifier 132 learned the co-occurrence of the semantic label in the normal image based on the label image obtained by assigning the correct label to the normal image. On the other hand, the third classifier 133 learns the degree of similarity of the semantic labels in the normal image based on the probability distribution obtained by inputting the normal image to the first classifier 131.

The third discriminator 133 is obtained by learning the relationship between the statistical distribution of the probability distribution P _1,m around the coordinates (x, y) and the semantic label m at the coordinates. That is, the third discriminator 133 can be said to be a discriminator that learns "what kind of semantic label is likely to appear around certain coordinates when a normal image is input to the first discriminator 131". .. In other words, the third classifier 133 can be said to be a classifier that learns the tendency of the probability distribution output by the first classifier 131 that has input a normal image.

Like the second classifier 132, the third classifier 133 can perform learning using a random forest.

This will be described with reference to FIG.
Candidate conditions φ _c (c=1,..., C) necessary for learning the random forest are coordinates (
Two offset vectors (r _a , r _b ) starting from (x, y) are randomly set, and probability distributions (P _1,m (a), P _1, ) at the end points (a, b) of the vectors are set _{. The} probability distribution vectors v _a and v _b are calculated based on _m (b)), and set based on the angle θ formed by v _a and v _b .
Here, v _a =(P _1,1 (a),..., P _1,M (a)) and v _b =(P _1,1 (b),..., P _1,M (b)) To do. Further, the threshold value of the angle θ is τ.

Here, it is assumed that the determination condition (r _a , r _b , τ) is set.
As shown in FIG. 12A, when the angle θ>τ formed by the probability distribution vector v _a at the point _a and the probability distribution vector v _b at the point b is satisfied, the node transits to the right node.
On the other hand, as shown in FIG. 12B, when the angle θ>τ formed by the probability distribution vector v _a at the point b and the probability distribution vector v _b at the point b is not satisfied, the node transits to the left node.
The angle θ formed by the probability distribution vector v _a and the probability distribution vector v _b can be calculated by the equation (4).

Here, when θ is close to 0, it is estimated that there is similarity between the probability distribution vector at the point a and the probability distribution vector at the point b. On the other hand, when θ is close to π/2, it is estimated that there is no similarity between the probability distribution vector at the point a and the probability distribution vector at the point b. That is, by setting an appropriate threshold value τ, the similarity between the probability distribution vector at the point a and the probability distribution vector at the point b can be determined.

When the probability distribution P _1,m output from the first classifier 131 is input by learning the third classifier 133, the probability distribution P _3,m of the semantic label is output for each pixel included in the image. It is possible to obtain a discriminator that

FIG. 13 is a flowchart showing a flow of processing for learning the first discriminator 131, the second discriminator 132, and the third discriminator 133 using a plurality of images in the second embodiment. Note that the same processing as that of the first embodiment is illustrated by the dotted line, and the description thereof will be omitted.

In the second embodiment, after step S14 is completed, the learning image is input again to the first discriminator 131, and evaluation is performed. The probability distribution P _1,m obtained as a result is input to the third discriminator 133, and learning is performed by the method described above.

FIG. 14 is a flowchart showing a flow of processing for evaluating the evaluation image and determining the presence of a road obstacle by using the learned identification unit 13.
In the second embodiment, after the step S23 is completed, the probability distribution P _1,m generated by the first classifier 131 is input to the third classifier 133 to obtain the probability distribution P _3,m (step S23A).

In the first embodiment, in step S24, the first likelihood S ₁ is calculated based on the dissociation degree of the probability distribution P _1,m and the probability distribution P _2,m .
On the other hand, in the second embodiment, the first likelihood S ₁ is calculated based on the dissociation degree of the probability distribution P _1,m and the probability distribution P _2,m , and the probability distribution P _1,m And the dissociation degree of the probability distribution P _3,m ,
The second likelihood S ₂ is calculated. Further, the first likelihood S ₁ and the second likelihood S ₂ are integrated to calculate the final likelihood (integrated likelihood S) (step S24A).
The second likelihood S ₂ can be obtained by the sum of squares of the differences, as in Expression (5).

In the second embodiment, the abnormality determination unit 14 calculates the integrated likelihood at the coordinates (x, y) of the evaluation image I′(t) based on the first likelihood S ₁ and the second likelihood S _2. The degree S(x,y) is calculated. The integrated likelihood S may be the sum of the first likelihood S ₁ and the second likelihood S ₂ as in Equation (6), or may be the first likelihood S ₁ as in Equation (7). It may be a product of the likelihood S ₁ and the second likelihood S ₂ .
S(x,y)=S ₁ (x,y)+S ₂ (x,y) (6)
S(x, y)=S ₁ (x, y)·S ₂ (x, y) (7)

(Advantageous effect)
The second discriminator 132 and the third discriminator 133 are both discriminators that learn "a tendency of a probability distribution obtained by inputting a normal image (an image that does not include road obstacles) to the first discriminator 131". It can be said that there is. By inputting the probability distribution output from the first classifier 131 to such a classifier and comparing the obtained probability distribution with the original probability distribution, “the first classifier is determined from the tendency at the time of learning. It is possible to detect that "the output is out of line". That is, even when the road obstacle cannot be correctly detected only by the first discriminator, it can be estimated that "the first discriminator is erroneously determined due to an unknown object".
Further, since the likelihood for a road obstacle can be calculated for each pixel, it is possible to clearly indicate to the user the area where the existence of a road obstacle is suspected.

(Modification)
The above-described embodiment is merely an example, and the present invention can be appropriately modified and implemented without departing from the scope of the invention.
For example, in the second embodiment, the first discriminator 131, the second discriminator 132, and the third discriminator 133 are used together, but only the first discriminator 131 and the third discriminator 133 may be used. In this case, the abnormality determination may be performed using only the second likelihood S ₂ .

The present invention can be understood as an identification device including at least a part of the above-mentioned means. Further, the present invention can be understood as an identification method that executes at least a part of the above processing.

The identification device 10 in the present invention is not limited to mounting by a semiconductor integrated circuit (LSI), and may be realized by a computer having a general-purpose microprocessor or memory executing a program.

Further, in the description of the embodiment, the configuration in which the obstacle on the road is detected has been described, but any object other than the obstacle on the road may be used as long as it detects an object that does not exist in the normal image. For example, the invention can be implemented as an identification device that detects that an image contains some abnormal object.

Further, in the description of the embodiment, the same device performs both the learning process and the evaluation process, but the device that performs the learning process and the device that performs the evaluation process may be separated. Further, means related to the learning process may be omitted from the identification device 10. For example, the learning device may be omitted from the illustrated identification device 10 and may be implemented as a device (identification device) that performs only the evaluation process.
In this case, the learned discriminator 13 (or the discriminator included in the discriminator 13) may be logically separated from the device so that it can be incorporated into another device. For example, the identification unit 13 learned by the identification device 10 according to the embodiment may be incorporated into a device that performs only the evaluation process.
On the contrary, the invention may be implemented as an apparatus that performs only the learning process.

10... Identification device 11... Image acquisition unit 12... Labeling unit 13... Identification unit 131... First discriminator 132... Second discriminator 14... Abnormality determination unit

Claims (8)

An identification device for determining the presence of an abnormal object included in an image,
Based on the result of learning the characteristics of the pixels included in the first image that is an image that does not include the abnormal object, the label is estimated for each of the plurality of pixels included in the input image, and the probability of the plurality of labels is estimated. A first type discriminator which outputs a first probability distribution representing a distribution for each pixel,
Based on the result of learning the relationship of labels in the vicinity of a plurality of pixels included in the first image, from the first probability distribution, a second type classifier that generates a second probability distribution,
The first probability distribution obtained by inputting a second image to the first type discriminator, and the second probability obtained by inputting the first probability distribution to the second type discriminator. Probability distribution, based on the dissociation degree of, the determination means for determining that the abnormal object is present in the second image,
And an identification device. The second type discriminator is a discriminator that has learned the co-occurrence or similarity of the labels as the relationship of the labels,
The identification device according to claim 1. The second type classifier is a classifier that has learned the first probability distribution obtained by inputting the first image to the first type classifier as learning data.
The identification device according to claim 1 or 2. The second type discriminator uses, as learning data, the first probability distribution obtained by inputting the first image to the first type discriminator, and around a plurality of pixels included in the first image. Is a classifier that has learned the similarity of the labels in
The identification device according to claim 1. The second type discriminator uses the first label image in which the correct label is given to the first image as learning data, and determines the co-occurrence of the label around a plurality of pixels included in the first image. Is a learned discriminator,
The identification device according to claim 1. An identification method for determining the presence of an abnormal object included in an image,
Based on the result of learning the characteristics of the pixels included in the first image that is an image that does not include the abnormal object, the label is estimated for each of the plurality of pixels included in the input image, and the probability of the plurality of labels is estimated. A first identification step of outputting a first probability distribution expressing a distribution for each pixel;
Based on the result of learning the relationship of labels in the vicinity of a plurality of pixels included in the first image, from the first probability distribution, a second identification step of generating a second probability distribution,
The first probability distribution obtained by processing the second image in the first identification step, and the second probability distribution obtained by processing the first probability distribution in the second identification step. Probability distribution, based on the dissociation degree of, the determination step of determining that the abnormal object is present in the second image,
Identification method, including. A program for causing a computer to execute the identification method according to claim 6. An identification device for determining the presence of an abnormal object included in an image,
Based on the result of learning the characteristics of the pixels included in the first image that is an image that does not include the abnormal object, the label is estimated for each of the plurality of pixels included in the input image, and the probability of the plurality of labels is estimated. A first type discriminator which outputs a first probability distribution representing a distribution for each pixel,
Based on the result of learning the tendency of the first probability distribution output by the first type discriminator when the first image is input to the first type discriminator, from the first probability distribution, A second type discriminator that generates a second probability distribution,
The first probability distribution obtained by inputting a second image to the first type discriminator, and the second probability obtained by inputting the first probability distribution to the second type discriminator. Probability distribution, based on the dissociation degree of, the determination means for determining that the abnormal object is present in the second image,
And an identification device.

Images