JP6994950B2 - How to learn image recognition system and neural network - Google Patents

Description

The present invention relates to an image recognition system using a neural network and a method for learning a neural network in an image recognition system.

Distance to the target object in a system that detects other vehicles and pedestrians using an in-vehicle camera and alerts the driver to the existence of those target objects, and a system that automatically controls according to the existence of those target objects. Accurately finding is an important elemental technology.

As a conventional technique, there is a method of performing pattern recognition using a neural network on a still image obtained by a monocular camera and estimating a physical quantity such as a distance to a target object.

However, the physical quantity may not be obtained accurately by the method by the prior art. In particular, the above method using a monocular camera has already matured to a considerable extent, and no significant improvement can be expected.

One of the reasons why the accuracy is not sufficient by the method by the conventional technique is that the neural network is optimized in consideration of only the appearance of one still image in the past. Although the physical quantities obtained by using such a neural network give appropriate results for still images, the obtained physical quantities are rarely different from the physical knowledge when grasped in chronological order. do not have. For example, it may indicate that the distance to the target object changes significantly in a short time.

The present invention has been made in view of such problems, and an object of the present invention is to provide an image recognition system capable of estimating physical quantities more accurately and a method for learning a neural network in an image recognition system.

According to one aspect of the present invention, a neural network in an image recognition system that estimates observations regarding the appearance of an object contained in each of a plurality of images using a neural network and calculates the physical quantity of the object from the estimated observations. The learning method of the neural network is provided, which optimizes the weight in the neural network by using the true value for the observation and the prior knowledge about the physical quantity.

It is desirable to optimize the weights in the neural network so that the sum of the first cost term indicating the estimation accuracy of the observation and the second cost term indicating the calculation accuracy of the physical quantity is minimized.

In this case, the second cost term may have a smaller value as the calculated physical quantity is closer to a value based on prior knowledge.
Alternatively, the second cost term may have a smaller value as the calculated distribution of the physical quantity is closer to the distribution based on prior knowledge.

The physical quantity may be jerk or jerk.

The image recognition system estimates a disturbance from each of the plurality of images, calculates the physical quantity of the object from the estimated observation and the disturbance, and uses the prior knowledge about the disturbance to calculate the weight in the neural network. It may be optimized.

Then, the sum of the first cost term indicating the estimation accuracy of the observation, the second cost term indicating the calculation accuracy of the physical quantity, and the third cost term indicating the estimation accuracy of the disturbance is minimized. It is desirable to optimize the weights in the neural network.

In this case, the third cost term may have a smaller value as the estimated disturbance is closer to a value based on prior knowledge.
Alternatively, the third cost term may have a smaller value as the estimated distribution of the disturbance is closer to the distribution based on prior knowledge.

The disturbance may include an inclination of the road surface, a deviation in the mounting height of the camera that captures each of the plurality of images, and at least one of the pitch angles of the camera.

The observation regarding the appearance of the object may be a rectangle surrounding the object in the plurality of images.

Further, according to another aspect of the present invention, an image recognition system that estimates observations regarding the appearance of an object contained in each of a plurality of images using a neural network, and calculates the physical quantity of the object from the estimated observations. Therefore, an image recognition system is provided that optimizes weights in the neural network by using the true value for the observation and the prior knowledge about the physical quantity.

The accuracy of estimating physical quantities is improved.

The figure which shows an example of an image recognition system schematically. The schematic diagram which shows the schematic structure of the image recognition system. The figure explaining the model of the observation, the disturbance and the physical quantity in 1st Embodiment. The block diagram which shows an example of the internal structure of the physical quantity estimation part 2 which concerns on 1st Embodiment. The figure explaining the physical quantity estimation part 2 at the time of learning. A graph comparing the acceleration estimation result by this method and the acceleration estimation result by the conventional method. A graph comparing the distance estimation result by this method and the distance estimation result by the conventional method. The figure explaining the model of the observation and the physical quantity in the 2nd Embodiment. The block diagram which shows an example of the internal structure of the physical quantity estimation part 2 which concerns on 2nd Embodiment.

Hereinafter, embodiments according to the present invention will be specifically described with reference to the drawings.

(First Embodiment)
FIG. 1 is a diagram schematically showing an example of an image recognition system. In this example, the physical quantity of the target vehicle 200 included in the image is estimated from the image obtained by the image taken by the camera 1 mounted on the own vehicle 100 by using a convolutional neural network that has been learned in advance. It is a thing. Collision risk judgment is performed from the output physical quantity.

The physical quantity is, for example, the size, distance, speed, acceleration, etc. of the target vehicle 200. These physical quantities are obtained by first estimating the position of the target vehicle 200 in the image and then performing geometric calculation based on a preset model for the estimation result.

Here, the position of the target vehicle 200 in the image can be supervised learning using a large amount of images whose true value is known in advance (for example, an image in which the true value is manually given), and high accuracy is achieved by the prior art. It can be estimated with.

On the other hand, it is not easy to obtain the true values of physical quantities such as the size, distance, speed, and acceleration of the target vehicle 200, and it is difficult to perform supervised learning in advance. Therefore, when these physical quantities are affected by disturbance, the error may become large. In particular, since the acceleration is obtained by differentiating the distance to the second order, the error tends to be particularly large even for a slight disturbance.

Therefore, in this embodiment, we present a learning method that can accurately detect physical quantities of dynamics such as magnitude, distance, velocity, and acceleration even when there is a disturbance. The "speed" is, to be exact, the "relative speed" of the target vehicle 200 with respect to the own vehicle 100, but in the present specification, it is simply referred to as a speed. The same applies to acceleration and the like.

FIG. 2 is a schematic diagram showing a schematic configuration of an image recognition system. The image recognition system includes a camera 1 which is an example of a sensor, a physical quantity estimation unit 2, and a time series filter 3.

As shown in FIG. 1, the camera 1 is a monocular camera mounted on the own vehicle 100, and captures a specific range in front of the camera 1. Although the captured image is a moving image, it is input to the physical quantity estimation unit 2 as a continuous still image at a plurality of times. In this embodiment, it is assumed that the "plurality of times" are consecutive 5 times (t-2 to t + 2).

The physical quantity estimation unit 2 estimates the physical quantity related to the target vehicle 200 from the images at a plurality of times by using a convolutional neural network. The physical quantities in the present embodiment are the size of the target vehicle 200 (more specifically, the vehicle height and the vehicle width), the distance to the target vehicle 200, the speed and acceleration of the target vehicle 200, and the like, and the details are shown in FIG. It will be described later using this. A specific configuration example of the physical quantity estimation unit 2 will be described later with reference to FIG. 4A.

The time-series filter 3 is, for example, a Kalman filter, and corrects the output from the physical quantity estimation unit 2. In the present embodiment, not only the distance but also the speed and the acceleration are input to the time series filter 3, so that the accuracy of the correction is improved.

FIG. 3 is a diagram illustrating a model of observation, disturbance, and physical quantity in the first embodiment. The optical axis of the camera 1 is the Z axis (rightward on the paper surface), the vertical direction is the Y axis (downward on the paper surface, the vertical downward direction is positive), and the directions orthogonal to the Z axis and the Y axis are the X axis (perpendicular to the paper surface). Further, the focal position (known) of the camera 1 is set as the origin. Then, when there is no disturbance, it is assumed that the road surface is parallel to the Z axis and the camera 1 is mounted at a height H (known) from the road surface. It is assumed that the road surface has no unevenness.

In this embodiment, the observations Γ _T , Γ _B , Γ _R , Γ _L , disturbance α, ΔH and physical quantities Dy, Dx, Z, dZ / dt, d ² Z / dt ² are defined as follows.

First, the observations Γ _T , Γ _B , Γ _R , and Γ _L , which are indicators related to the appearance of the target vehicle 200 included in the image, will be described. Γ _T is a ray showing a straight line passing through the camera 1 and the midpoint of the upper end of the rectangle when the target vehicle 200 is surrounded by a rectangle on the image. Similarly, Γ _B , Γ _R , and Γ _L are rays indicating straight lines passing through the camera 1 and the lower midpoint, the right midpoint, and the left midpoint of the rectangle, respectively. That is, the following equation holds.
Y = Γ _T * Z ・ ・ ・ (1)
Y = Γ _B * Z ・ ・ ・ (2)

Observations Γ _T , Γ _B , Γ _R , and Γ _L can be said to indicate a rectangle surrounding the target vehicle 200 in the image, in other words, indicate the position of the target vehicle 200 in the image. In the following, Γ _T , Γ _B , Γ _R , and Γ _L are collectively referred to as Γ.

Next, the disturbances α and ΔH will be described. The disturbance α is the inclination α of the road surface, and may be caused by the influence of picking of the own vehicle 100 or the like. The disturbance ΔH is a height deviation of the camera 1, more accurately, a difference from the mounting height H of the camera 1, and may be caused by the influence of the suspension in the own vehicle 100 or the like. In this case, the true road surface considering the disturbances α and ΔH is expressed as follows.
Y = (H + ΔH) + α * Z ・ ・ ・ (3)

In addition, the pitch angle may be considered as a disturbance. The pitch angle is the rotational movement of the camera 1 in the XZ plane, which is caused by the pitching of the own vehicle 100.

Next, the physical quantities Z, dZ / dt, d ² Z / dt ² , Dx, and Dy, which are the values to be finally obtained, will be described.

The physical quantity Z is the distance from the camera 1 to the target vehicle 200, and is calculated from the above equations (2) and (3).
Z = (H + ΔH) / (Γ _B －α) ・ ・ ・ (4)
The actual distance can be obtained by multiplying the distance Z in the above equation (4) by the focal length f (known internal parameter) of the camera 1.

The physical quantities dZ / dt and d ² Z / dt ² are the velocities and accelerations of the target vehicle 200, respectively, and are calculated by discretely differentiating the distance Z.

The physical quantity Dy is the height (vehicle height) of the target vehicle 200, and is calculated from the above equations (1), (2), and (4).
Dy = (Γ _B -Γ _T ) * (H + ΔH) / (Γ _B -α) ・ ・ ・ (5)

The physical quantity Dx is the width (vehicle width) of the target vehicle 200, and is calculated in the same manner as the vehicle height Dy by considering the pitch angle.

As described above, the physical quantities Dx, Dy, Z, dZ / dt, and d ² Z / dt ² in the present embodiment can all be calculated immediately based on the observed Γ and the disturbances α and ΔH.

FIG. 4A is a block diagram showing an example of the internal configuration of the physical quantity estimation unit 2 according to the first embodiment. The physical quantity estimation unit 2 includes an observation estimation unit 21, a disturbance estimation unit 22, and a physical quantity calculation unit 23. In FIG. 4A, the observation estimation unit 21 and the disturbance estimation unit 22 are divided for the sake of explanation, but a single convolutional neural network may be used. Further, a part or all of each part may be realized by the processor of the computer executing a predetermined program.

The observation estimation unit 21 uses a convolutional neural network (for example, a pattern recognizer) that has been pre-learned to be described later from each image at times t-2 to t + 2, and observes Γ (t-2) for 5 hours. It estimates ~ Γ (t + 2) and is a so-called black box.

The disturbance estimation unit 22 uses a convolutional neural network that has been pre-learned, which will be described later, from each image at times t-2 to t + 2, and disturbs α (t-2) to α (t + 2), ΔH for 5 hours. (T-2) to ΔH (t + 2) are estimated.

The physical quantity calculation unit 23 includes observations Γ (t-2) to Γ (t + 2) estimated by the observation estimation unit 21, and disturbances α (t-2) to α (t + 2), ΔH estimated by the disturbance estimation unit 22. From (t-2) to ΔH (t + 2), the above equations (4) and (5) are applied, and the physical quantities Z (t-2) to Z (t + 2), Dy (t-2) to 5 hours are applied. Dy (t + 2), Dx (t-2) to Dx (t + 2) are calculated. Further, the physical quantity calculation unit 23 discretely differentiates the distance Z to calculate the velocity dZ (t) / dt and the acceleration d ² Z (t) / dt ² . Since the velocity dZ (t) / dt and the acceleration d ² Z (t) / dt ² are obtained by discrete differentiation, they do not have five hours.

The physical quantity calculation unit 23 does not need to perform estimation using a convolutional neural network, and is a white box that “calculates” the physical quantity by geometric calculation.

Next, prior learning of the convolutional neural network in the observation estimation unit 21 and the disturbance estimation unit 22 will be described. A convolutional neural network is composed of a plurality of convolutional layers and a fully connected layer. The convolution layer applies a filter to the output from the convolution layer in the previous stage (in the convolution layer in the first stage, an input image at a plurality of times) to perform convolution. Weights are set in the filter, and pre-learning is to optimize these weights.

FIG. 4B is a diagram illustrating a physical quantity estimation unit 2 at the time of learning. In the present embodiment, the cost terms J1, J2, and J3 are set in each of the observation estimation unit 21, the disturbance estimation unit 22, and the physical quantity calculation unit 23. Then, the weights are optimized so that the cost function J, which is the sum of these, is minimized.

Since it is easy for the observation estimation unit 21 to obtain the true value of the rectangle (that is, the observation Γ) surrounding the target vehicle 200 in advance, it is desirable to perform supervised learning. Therefore, the image for 5 hours and the true value of the rectangular region (that is, the observation Γ) are input to the observation estimation unit 21 as teacher data for each of the images. Then, the cost term J1 indicating the estimation accuracy of the observation is defined. For example, the sum of the squares of the difference between the true value of the observed Γ and the estimated observed Γ can be the cost term J1.

An estimator (not shown) is provided in front of the observation estimation unit 21, and each of the images for 5 hours is regarded as a still image to estimate the rectangle (observation) of the target vehicle 200, and this is observed as the initial value Γ0. It may be input to the fully connected layer of the estimation unit 21. In this case, the observation estimation unit 21 corrects the initial value Γ0 and estimates the observation Γ, which can be applied even when a plurality of vehicles are included in the image. If such an estimator is not provided and there are a plurality of vehicles in the image, the true value may be set for each area of the image.

Since it is difficult for the disturbance estimation unit 22 to obtain the true values of the disturbances α and ΔH in advance, unsupervised learning is performed. Therefore, the teacher data is not input to the disturbance estimation unit 22. Instead, in this embodiment, the following prior knowledge about disturbances α and ΔH will be used.

The distribution of the slope α of the road surface shall follow the normal distribution with a mean value of 0 and a standard deviation of σ _α . The reason why the average value is set to 0 is that the pitching of the own vehicle 100 and the change in the road surface shape can appear on both the positive side and the negative side. It is obvious from experience that the standard deviation σ _α is about 1/100 order, which is appropriate considering the realistic slope of the road surface, and is not more than 1/10 order.

It is assumed that the distribution of the height deviation ΔH of the camera 1 follows a normal distribution with a mean value of 0 and a standard deviation of σ _H. The average value of 0 is set because ΔH is originally defined as a deviation from the mounting height of the camera 1. The standard deviation σ _H is a value that should be set reflecting the prior knowledge of the designer, and it is obvious from experience that about 1 cm order is appropriate and not more than 1 m order.

Then, the cost term J2 indicating the estimation accuracy of the disturbance is defined. For example, a cost term J2 is set in which the distributions of the estimated disturbances α and ΔH for 5 hours are smaller as they are closer to the normal distribution based on the above prior knowledge. More specifically, the sum of the logarithms of the Kullback-Leibler distance (KL distance) between the distributions of disturbances α and ΔH and the normal distribution based on the above prior knowledge can be set as the cost term J2. As another example, a cost term J2 may be set in which the closer the estimated 5 hours of disturbance α, ΔH is to the mean value (here, 0) based on prior knowledge, the smaller the value. More specifically, the sum of the squares of the estimated disturbances α and ΔH or the absolute value (because the average value is 0) may be the cost term J2.

Since it is difficult for the physical quantity calculation unit 23 to obtain the true value of the physical quantity in advance, unsupervised learning is performed. Therefore, the teacher data is not input to the physical quantity calculation unit 23. Instead, in this embodiment, the following prior knowledge about the physical quantities d ² Z / dt ² , Dy, and Dx is used.

The distribution of acceleration d ² Z / dt ² shall follow a normal distribution with mean value 0 and standard deviation σ _{Z 2} . The average value is set to 0 because acceleration and deceleration are considered to have the same frequency. It is self-evident from experience that the standard deviation σ _Z2 is appropriate on the order of 0.01 G (G is gravitational acceleration) and not more than 0.1 G, considering the realistic acceleration of the own vehicle 100 and the target vehicle 200. Is.

The distribution of the vehicle height Dy and the vehicle width Dx is assumed that the average value is a time-series average (the vehicle height and the vehicle width of the target vehicle 200) and follows the normal distribution of the standard deviations σ _Dy and σ _Dx . The standard deviations σ _Dy and σ _Dx are values that should be set reflecting the prior knowledge of the designer, and it is obvious from experience that about 1 cm order is appropriate and not more than 1 m order.

Then, a cost term J3 indicating the accuracy of calculating the physical quantity is defined. For example, a cost term J3 is set in which the calculated physical quantities d ² Z / dt ² and the distributions of the physical quantities Dy and Dx for 5 hours become smaller as they are closer to the normal distribution based on the above prior knowledge. More specifically, the sum of the logarithms of the Kullback-Leibler distance (KL distance) between the distribution of the physical quantities d ² Z / dt ² , Dy, and Dx and the normal distribution based on the above prior knowledge can be set as the cost term J3. can. As another example, consider the standard deviations σ _z2 , σ _Dy , and σ _Dx as 0, and the smaller the estimated values of the estimated physical quantities d ² Z / dt ² , Dy, and Dx are, the closer they are to the mean value based on prior knowledge. The cost term J3 to be taken may be set. More specifically, the sum of the squares or absolute values of the differences between the calculated physical quantities d ² Z / dt ² , Dy, and Dx and their respective average values may be used as the cost term J3. Alternatively, if the true value of acceleration can be obtained by measurement using high-precision PGS or millimeter-wave radar on a driving test course, supervised learning is performed for images with true values, and unsupervised learning is performed for images without true values. You may study.

Further, for the distance Z and the velocity dZ / dt, supervised learning may be performed if a true value is obtained, and unsupervised learning based on prior knowledge may be performed if the true value cannot be obtained.

The sum of the above cost terms J1 to J3 is used as the cost function, and the weight is optimized so that the cost function is minimized. Specifically, the mini-batch stochastic gradient descent method, which is a standard method in neural network learning, can be applied. A mini-batch refers to a plurality of randomly sampled learning samples at each iteration of optimization. In the present embodiment, images for five consecutive hours are one learning sample. The mini-batch size (the number of learning samples in one mini-batch) is large enough.

In the observation estimation unit 21, not only the estimated observation Γ approaches the true value, but also the internal weight is optimized so that the distribution of the physical quantity calculated by the physical quantity calculation unit 23 follows the prior knowledge. Similarly, in the disturbance estimation unit 22, not only the estimated disturbances α and ΔH follow the prior knowledge, but also the internal weights are optimized so that the distribution of the physical quantity calculated by the physical quantity calculation unit 23 follows the prior knowledge. ..

As a result, the observation estimation unit 21 can estimate the observation Γ that is close to the true value, and the disturbance estimation unit 22 can estimate the disturbances α and ΔH that appropriately follow the probability distribution based on the prior knowledge. The physical quantity calculation unit 23 can calculate physical quantities d ² Z / dt ² , Dy, and Dx that appropriately follow a probability distribution based on prior knowledge. This makes it difficult to obtain physically impossible estimation results such as the slope α of the road surface becoming about 1/10 and the acceleration d ² Z / dt ² becoming about 0.1 G, which is physically difficult. It means that you will be able to obtain meaningful results.

FIG. 5 is a graph comparing the acceleration estimation result by the present method and the acceleration estimation result by the conventional method. The horizontal axis is the true distance to the object, and the vertical axis is the root mean square value (Root Mean Square, unit is gravity acceleration G) of the estimated acceleration. Note that the observed Γ gives the true value, not the estimated value.

According to the conventional method, it can be seen that the absolute value of the acceleration is abnormally large (normal acceleration should be about 0.2 G at most), and clearly wrong values are frequently obtained. This indicates that even if the observed Γ is ideal, it does not necessarily give an accurate physical quantity.

On the other hand, according to this method, even when the distance to the target is long, the acceleration is sufficiently lower than that of the conventional method, and the result closer to reality is obtained.

FIG. 6 is a graph comparing the distance estimation result by the present method and the distance estimation result by the conventional method. The horizontal axis is the true distance to the object, and the vertical axis is the root mean square error of the error between the estimated distance and the true distance.

According to the conventional method, the error is large, and it can be seen that the error is particularly large as the distance to the target increases. On the other hand, according to this method, it can be seen that the distance is obtained more accurately than the conventional method.

As described above, in the first embodiment, the acceleration estimated from a plurality of continuous images is learned by using prior knowledge. Therefore, prior knowledge is reflected, and the obtained physical quantity becomes physically closer to reality and highly accurate. In addition, by learning using prior knowledge even for disturbances for which it is difficult to prepare true values in advance, the estimation accuracy is further improved.

(Second embodiment)
The second embodiment described below does not consider disturbance. Hereinafter, the differences from the first embodiment will be mainly described.

FIG. 7 is a diagram illustrating a model of observation and physical quantity in the second embodiment. The definition of coordinates is the same as in FIG. 3, but the inclination of the road surface is not taken into consideration.

In this embodiment, in addition to the observations Γ _T , Γ _B , Γ _R , and Γ _L of the first embodiment, the observation Γ _C is defined as follows. The observation Γ _C is a camera height ray that shows a straight line passing through the camera 1 and a point that moves vertically upward by the mounting height H of the camera 1 from the road surface in a rectangle surrounding the target vehicle 200 on the image. That is, the following equation holds.
Y = Γ _C * Z ・ ・ ・ (6)

Even in this case, the distance Z and the vehicle height Dy, which are physical quantities, are "calculated" as follows.
Z = H / (Γ _B -Γ _C ) ・ ・ ・ (7)
Dy = (Γ _B -Γ _T ) * H / (Γ _B -Γ _C ) ・ ・ ・ (8)
Similarly, the vehicle width Dx, the vehicle speed dZ / dt, and the acceleration d ² Z / dt ² are also calculated.

FIG. 8 is a block diagram showing an example of the internal configuration of the physical quantity estimation unit 2 according to the second embodiment. Since the disturbance is not considered in the present embodiment, the disturbance estimation unit 22 may not be provided as compared with FIG. The operations of the observation estimation unit 21 and the physical quantity calculation unit 23 are substantially the same as those of the first embodiment. That is, in the observation estimation unit 21, supervised learning is performed using the true value of the observation Γ _C. In the physical quantity calculation unit 23, unsupervised learning using prior knowledge is performed.

As described above, also in the second embodiment, the acceleration estimated from a plurality of continuous images is learned by using prior knowledge. Therefore, prior knowledge is reflected and the estimation accuracy is improved without considering the disturbance.

In each of the above-described embodiments, the physical quantity calculation unit 23 is not limited to the acceleration d ² Z / dt ² , but other physical quantities (size, distance Z, speed dZ / dt of the target vehicle 200 such as vehicle height Dy and vehicle width Dx). , Jerk d ³ Z / dt ³ (or higher-order discrete derivative)) is learned using prior knowledge to improve the estimation accuracy of physical quantities. However, it is particularly desirable to learn and estimate the acceleration d ² Z / dt ² which is useful for determining the danger of collision and tends to have a large estimation error by using prior knowledge.

The above-described embodiments have been described for the purpose of allowing a person having ordinary knowledge in the technical field to which the present invention belongs to carry out the present invention. Various modifications of the above embodiment can be naturally made by those skilled in the art, and the technical idea of the present invention can be applied to other embodiments. Accordingly, the invention is not limited to the described embodiments and should be the broadest scope according to the technical ideas defined by the claims.

1 Camera 2 Physical quantity estimation unit 3 Time series filter 21 Observation estimation unit 22 Disturbance estimation unit 23 Physical quantity calculation unit 100 Own vehicle 200 Target vehicle

Claims (12)

It is a learning method of a neural network in an image recognition system that estimates observations about the appearance of an object included in each of a plurality of images using a neural network and calculates the physical quantity of the object from the estimated observations.
Using the true value for the observation and the prior knowledge about the physical quantity, the sum of the first cost term indicating the estimation accuracy of the observation and the second cost term indicating the calculation accuracy of the physical quantity is minimized. , A neural network learning method that optimizes the weights in the neural network. The second cost term is the method for learning a neural network according to claim 1 , wherein the calculated physical quantity takes a smaller value as it is closer to a value based on prior knowledge. The second cost term is the method for learning a neural network according to claim 1 , wherein the calculated distribution of the physical quantity takes a smaller value as it is closer to the distribution based on prior knowledge. It is a learning method of a neural network in an image recognition system that estimates observations about the appearance of an object included in each of a plurality of images using a neural network and calculates the physical quantity of the object from the estimated observations.
The image recognition system estimates a disturbance from each of the plurality of images, calculates the physical quantity of the object from the estimated observation and the disturbance, and calculates the physical quantity of the object.
A method for learning a neural network that optimizes weights in the neural network by using the true value for the observation, the prior knowledge about the physical quantity, and the prior knowledge about the disturbance . The neural network so that the sum of the first cost term indicating the estimation accuracy of the observation, the second cost term indicating the calculation accuracy of the physical quantity, and the third cost term indicating the estimation accuracy of the disturbance is minimized. The method for learning a neural network according to claim 4 , wherein the weights in the above are optimized. The third cost term is the method for learning a neural network according to claim 5 , wherein the estimated disturbance takes a smaller value as it approaches a value based on prior knowledge. The third cost term is the method for learning a neural network according to claim 5 , wherein the estimated distribution of the disturbance takes a smaller value as it is closer to the distribution based on prior knowledge. 13. How to learn a neural network. The method for learning a neural network according to any one of claims 1 to 8, wherein the physical quantity is jerk or jerk. The method for learning a neural network according to any one of claims 1 to 9 , wherein the observation regarding the appearance of the object is a rectangle surrounding the object in the plurality of images. It is an image recognition system that estimates the observation of the appearance of an object contained in each of a plurality of images using a neural network and calculates the physical quantity of the object from the estimated observation.
Using the true value for the observation and the prior knowledge about the physical quantity, the sum of the first cost term indicating the estimation accuracy of the observation and the second cost term indicating the calculation accuracy of the physical quantity is minimized. , An image recognition system that optimizes weights in the neural network. It is an image recognition system that estimates the observation of the appearance of an object contained in each of a plurality of images using a neural network and calculates the physical quantity of the object from the estimated observation.
The image recognition system estimates a disturbance from each of the plurality of images, calculates the physical quantity of the object from the estimated observation and the disturbance, and calculates the physical quantity of the object.
An image recognition system that optimizes weights in a neural network using the true value for the observation, the prior knowledge about the physical quantity, and the prior knowledge about the disturbance.

Images