JP2934330B2 - Vehicle operation amount determination device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for directly determining a desired operation amount of a vehicle from road image information on a traveling road surface obtained while the vehicle is running, thereby realizing the vehicle in a short calculation time in real time. The present invention relates to an apparatus for determining an operation amount of a vehicle.

[0002]

2. Description of the Related Art Conventionally, a TV camera is mounted on a vehicle to image a travel route, and the image is processed to calculate an angle between a road and a traveling direction of the vehicle, a position of the vehicle, and the like. 2. Description of the Related Art There is known a device for determining an operation amount of a vehicle based on a control algorithm according to an operation amount of an accelerator, a brake, a steering, or the like, which is an operation amount of a vehicle, and a motion characteristic of the vehicle (Japanese Patent Laid-Open No. 62-140110) Gazette, JP 63-273917
JP-A-60-157611).

[0003]

However, in this prior art, a great deal of time is required for image processing for determining the position and orientation of the vehicle, and the control algorithm becomes complicated. There was a problem that a program could not be obtained and a large-scale system was required as an in-vehicle device.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problem, and an object of the present invention is to directly determine the operation amount of a vehicle from road image information, thereby realizing a real-time operation of the vehicle in a short calculation time. Is to determine the amount.

[0005]

In order to solve the above-mentioned problems, the present invention is directed to an image pickup apparatus for taking an image of a traveling road surface of a vehicle while the vehicle is traveling, and dividing road image information obtained by the imaging into a number of small areas. For each of the divided areas, road data creating means for creating road data indicating the existence information of the road portion in this divided area, and road data for each of the divided areas, corresponding to each of the divided areas A neural network having a plurality of input elements for inputting and an output element for outputting an operation value for determining a behavior of the vehicle is provided.

[0006]

In the road data creating means, the road surface of the vehicle is imaged while the vehicle is running, and the road image information obtained by the imaging is divided into a number of small areas. Then, in the road image information divided into the plurality of small regions, road data indicating existence information of a road portion in each divided region is created for each divided region.

Next, the road data for each divided area is
The input is input to the input element of the neural network corresponding to each of the divided areas. Then, from the output element of the neural network, an operation value for determining an optimum behavior of the vehicle according to the road image information obtained by the imaging is output.

Incidentally, the coupling coefficient of this neural network is determined by the road data obtained from various road image information, that is, the optimum data of the vehicle corresponding to the road environment such as the curvature of the various roads, the turning direction, and the inclination of the road. The operation value for determining the behavior is taught in advance as a teacher signal.

[0009]

DESCRIPTION OF THE PREFERRED EMBODIMENTS An image of a road viewed from the driver's seat of a vehicle changes depending on road conditions such as a straight road, a curve, an intersection, and the position and direction of a vehicle with respect to the road. That is, the road image represents the road condition and the relative position and orientation of the vehicle with respect to the road. Therefore, the relationship between the operation amounts of the travel control device such as the accelerator, the brake, and the steering of the vehicle with respect to various road images can be obtained, and the operation amounts can be directly determined from the features of the road images. At this time, the operation characteristics of the travel control device and the algorithm for obtaining the operation amount from the road image are not used.

Now, the operation amount when the road image is R, for example,
The amount of steering operation is SR. Further, the road image R is divided into n in the horizontal direction (the width direction of the vehicle) and m in the vertical direction (the traveling direction of the vehicle) to be divided into a number of small areas. This divided area is a _pq , the ratio of the road occupying the divided area a _pq is r _pq ,
By occupying the divided regions a _pq road, the divided region a _pq contributes proportion to the manipulation amount SR and g _pq. For the sake of simplicity, it is assumed that the operation amount SR is not affected by the speed of the vehicle, and that there is no interaction between the divided regions of the image. Under this condition, the manipulated variable SR is expressed by the following equation.

[0011]

[Number 1] _{SR = r 11 · g 11 +} r 12 · g 12 + ... + r pq · g pq + ... + r nm · g nm ... (1)

Formula (1) is created for a combination of road images R of various environments and an operation amount SR optimal for the road environment.
This equation is solved as an n × m-order simultaneous equation to obtain g _pq . That is, a table of the contribution rates g _pq to the operation amounts of the respective divided areas of the road image (hereinafter referred to as a contribution rate map) can be created.

According to this method, the number of divisions of the road image is n ×
The operation amount can be obtained directly and at high speed from the road image by m times of product-sum operations. However, in practice, since the operation amount is affected by the speed of the vehicle, a map of the contribution ratio for each speed is required. For this reason, a large-capacity memory for storing the map of the contribution ratio is required. In addition, when the map of the contribution ratio corresponding to the vehicle speed is selected, and when the map of the contribution ratio corresponding to the speed does not exist, the speed is determined. A large amount of processing time is required for the interpolation calculation for finding the contribution rate corresponding to.

Equation (1) does not consider the interaction between the divided areas. For example, regarding the divided area a ₁₁ and the divided area a ₁₂ , the operation amount obtained from the two divided areas is determined by r ₁₁ × g ₁₁ + r ₁₂ × g ₁₂ according to the equation (1). That is, equation (1) is determined by the values of r ₁₁ and r ₁₂ ,
A term (r ₁₁ + r ₁₂ ) × g representing the interaction between r ₁₁ and r _12
11.12 (g _11.12 is the contribution of r ₁₁ and r ₁₂ ) does not exist. In order to take this interaction into account, it is necessary to prepare a map of the contribution ratio representing the interaction between all the regions for each speed. Therefore, an enormous amount of memory is required to store the map of the contribution ratio, and much time is required to calculate the operation amount. Further, when a nonlinear element is included in the interaction, it is also difficult to solve the contribution ratio by a mathematical expression.

In order to solve these problems, the present inventors have introduced a neural network in which a road image or a road image and other elements such as a vehicle speed are used as input data, and a vehicle operation amount is used as output data. This neural network is learned in advance with respect to input data including various road images or road images and other elements such as vehicle speed, using an optimal operation amount as a teacher signal. As described above, the relationship between the road image and the operation amount, including the nonlinear relationship, is learned as the coupling coefficient of the neural network. By learning the coupling coefficient, even when the contribution rate cannot be solved as described above, it becomes possible to directly obtain the optimal operation amount for the road environment from the road image.

At an intersection, the driver's intention of which direction the vehicle should proceed is required. Therefore, it is not possible to uniquely determine the operation amount of the vehicle from the road environment. In this case, another means, for example, a known navigation device is used, or the traveling direction is directly designated at an intersection.

FIG. 1 is a block diagram showing the overall configuration of the apparatus according to the present embodiment. The CCD camera 10 is mounted on the ceiling of the vehicle 1 facing forward in the traveling direction so as to capture an image of a traveling path ahead in the traveling direction of the vehicle 1.
The CCD camera 10 is driven at a certain period by the image processing device 12, and captures an image of the traveling environment ahead of the vehicle in the traveling direction in real time with the traveling of the vehicle. CCD camera 10
Each pixel of one frame is scanned by the image processing device 12, and the gray level of each pixel is digitized. The digitized image data of one frame is stored in the frame memory 14.

Next, the image processing apparatus 12 repeatedly executes the processing shown in FIG. 2 at a predetermined imaging cycle. Step 1
At 00, one frame of image data is read from the frame memory 14. Then, in step 102,
A differential image is generated by a well-known gradient operation from the read image data, and the differential image is stored in the differential image memory 16. Next, in step 104, an edge line segment is extracted from the differential image generated in the differential image memory 16. Further, in step 106, the mutual distance and mutual angle between the extracted many edge lines are calculated, and the same long line segment and a plurality of edge line segments constituting a continuous curve having the same curvature are clustered. Thereby, a continuous straight line and a continuous curve can be extracted.

Next, in step 108, curb lines on both sides of the road and the like are specified by pattern matching of the curb lines of the road and both end lines of the road. Next, in step 110, a road portion is specified by pattern matching of the road shape in the obtained image.
The image of the road portion obtained as a result is an image as shown in FIG. The image of the road portion as shown in FIG. 3 is stored in the road image memory 18.

Next, in step 112, the captured image on the road image memory 18 is divided into small areas as shown in FIG. That is, the vehicle is divided into m parts in the traveling direction and n parts in the width direction of the vehicle perpendicular to the traveling direction. The image processing range for creating a road image viewed from the driver's seat is the range shown in FIG. That is, the height of the viewpoint is H (m), the visible distance is in the range from XO (m) to XL (m) forward from the driver's position, the left and right visible ranges at position XO are WX0 (m), and position XL. The visible range on the left and right is WXL. In addition, WXL
= XL / XO × WXO. At the position XL, the left and right visible ranges may be set to WXN (WXN ≦ WXL). In this way, unnecessary images at a distant place can be deleted.

This image processing range is in the horizontal direction (horizontal direction).
Is divided into n and m in the traveling direction (longitudinal direction) of the vehicle. The division in the left-right direction is divided into equal distances. The division in the traveling direction is performed by dividing an angle (viewing angle) θ between a line segment connecting the viewpoint and the position XO and a line segment connecting the viewpoint and the position XL at an equal angle. Specifically, the height of the viewpoint: H = 1.0 (m) Image processing range: XO = 2.0 (m), XL = 8.0 (m) WXO = 10.0 (m), WXL = 40.0 (m), WXN = 20.0
(m) Image resolution: n = 20, m = 6.

FIG. 5 shows an image processing range when a region divided into n × m small regions is a _pq and each divided region a _pq is represented by a square having the same area and the same shape. In step 114, a ratio r _{pq of} each divided area a _pq occupied by the road portion shown in FIG. 3 is calculated. Then, in step 116, the occupation ratio _rpq is stored in the road data memory 20 as road data. The road data r _pq may be a binary value “1” when a road exists at the center position of each divided area a _pq and “0” when no road exists. Each divided area a _pq indicates each area obtained by dividing the image processing range into n × m. In FIG. 5, the divided area indicated by “x” is an unnecessary image at a distant place from the range of the survey as described above. This is a divided region outside the excluded visible region. Therefore, the road data _rpq of the divided region outside the visible region is “0” regardless of the existence of the road portion.

When the road is constituted by a truck as shown in FIG. 6, road data _rpq obtained by imaging corresponding to the vehicle locations A, B, and C is shown in FIG. (A), (B) and (C) are obtained. In the drawing, the road data r _pq divided area a _pq of black "1", i.e., to mean the presence of a road section, the road data r _pq divided area a _pq of the open box "0", i.e., Means the absence of a road section. (A) shows road data obtained when the vehicle is traveling on a straight line portion of a truck, (B)
Indicates road data obtained when the vehicle approaches the corner of the truck, and (C) indicates road data obtained when the vehicle is traveling on the corner of the truck. The road data _rpq as shown in FIG. 7 obtained by the image processing device 12 in this way is stored in the road data memory 20 in step 116.
As described above, the image processing device 12 generates the road data in the road data memory 20 by executing the above-described calculation in synchronization with the predetermined imaging cycle.

On the other hand, a well-known vehicle navigation device 50 is used.
0 is a map data memory 52 that stores road map data as shown in FIG. 8 and a travel route data memory 56 that stores a command value of a traveling direction at an intersection of vehicles as shown in FIG.
Calculation device 5 for calculating the current position and direction of the vehicle
4. Then, the current position of the vehicle is specified on the map from the current position of the vehicle calculated every moment by the position calculation device 54 and the map data. For example, as shown in FIG. 8, the current position is the position a or the position b.
As specified on the map. The specific data of this position is output to the image processing device 30.

The image processing device 30 is a device for executing the processing shown in FIG. In step 200, the current road data formed in the road data memory 20 is input, and in step 202, it is determined whether or not the road data indicates an intersection. The determination based on the road data as to whether or not the road on which the vehicle is currently traveling has reached an intersection where the traveling direction of the vehicle diverges is performed based on whether or not the road is compared with a number of intersection patterns prepared in advance.

If the road data does not indicate an intersection, it is determined in step 204 whether or not the output of the position calculation device 54 indicates an intersection. If the determination result is NO, the process returns to step 200, and Similar determination is performed by inputting road data in synchronization with the next imaging cycle.
As described above, usually, whether or not the vehicle has approached the intersection is constantly monitored based on the road data generated in synchronization with the imaging cycle and the output result of the position calculation device 54.

If the image processing device 30 determines that the vehicle is approaching the intersection, the image processing device 3
In step 206, specific data for specifying the position of the vehicle on the map is input from the position calculation device 54 in step 206. Then, in step 208, the travel route data memory 56 is accessed based on the specific data, and a command value for the traveling direction at the specific position on the road is input.

Next, the image processing apparatus 30 executes step 21
0, based on the command value of the traveling direction, the road data currently created in the road data memory 20 leaves only the road data in the traveling direction of the vehicle, and sets the road data in the direction in which the vehicle does not travel to “0”. I do. For example, when a vehicle approaches an intersection like a position a and a position b on the road map of FIG. 8, the road data formed in the road data memory 20 is as shown in FIG. When the vehicle is located at the position a in FIG. 8, the command value of the traveling route at that time provided from the traveling route data memory 56 is “turn right”. Therefore, in this case, as shown in FIG. 11A, the image processing device 30 masks the road data in the non-traveling direction of the left direction and the straight traveling direction of the vehicle as “0”.

When the vehicle is approaching the intersection at the position b in FIG. 8, the command value of the traveling route at that time given from the traveling route data memory 56 is "straight ahead". Therefore, in this case, the image processing device 30
As shown in (B) of FIG. 1, road data in the non-traveling direction in the left-right direction of the vehicle is masked as “0”. The command value of the travel route may be stored in advance in the travel route data memory 56 or may be given by the driver when the vehicle approaches the intersection in real time with the travel of the vehicle.

FIG. 11 formed in the road data memory 30
Each divided area a _pq indicating the road shape in the traveling direction as shown in
Each road data r _pq of inputs to each input element of the input layer of the neural network 40. The current vehicle speed data of the vehicle measured by the vehicle speed detector 22 is input to one input element of the input layer of the neural network 40.

The neural network 40 includes an input layer 401 and an output layer 402, as shown in FIG. The input layer 401 stores the data points (1
20) and input the number of data points (1) of the vehicle speed data 12
The output layer 402 is composed of one input element, and the output layer 402 has three types of operation amounts Ac,
It is composed of three output elements that output Br and St.

The neural network has two layers as shown in FIG. 12 and an intermediate layer 40 as shown in FIG.
3 may be provided. Further, the intermediate layer may be composed of an arbitrary plurality of layers. A neural network is generally defined as a device that performs the following operations.
Output O ⁱ _j of the j-th element of the i-layer is calculated by the following equation. Here, i ≧ 2.

[0033]

[Equation 2] O ⁱ _j = f (I ⁱ _j ) (2)

[Equation 3] I ⁱ _j = ΣW ^i-1 _k, ⁱ _j · O ^i-1 _k + V ⁱ _j ... (3) ^k

[Equation 4] f (x) = 1 / {1 + exp (-x)}… (4)

[0034] However, V ⁱ _j bias of the j th processing element of the i-th layer, W ^i-1 _k, ⁱ _j is the first i-1 layer k-th element and the j of the i-th layer The coupling coefficient between the first element and O ¹ _j represents the output value of the j-th element in the first layer. That is, since it is the first layer, the input is output as it is without performing any operation,
It is also the input value of the j-th element in the input layer (first layer).
Therefore,

[0035]

Equation 5] ^{_{O 1 j = r pq ... (}} 5) i.e., O ¹ _j is equal to the road data r _pq of the divided regions a _pq.

Next, the calculation procedure of the neural network 41 having a three-layer structure shown in FIG. 13 will be described with reference to FIG. The number of elements in the intermediate layer of the neural network 41 is s. In step 300, the j-th element of the intermediate layer (second layer) receives the output value O ¹ _j = (r _pq ) from each element of the input layer (first layer), and obtains equation (3). Then, the product-sum function operation of the following equation is performed.

(Equation 6)₁₂₁  I^Two _j= ΣW¹ _k, ^Two _j・ O¹ _k+ V^Two _j … (6)^{k = 1} 

Next, in step 302, the output of each element of the intermediate layer (second layer) is calculated by the following equation using the sigmoid function of the product-sum function value of the input value expressed by equation (6). . The output value of the j-th element in the second layer is calculated by the following equation.

[0038]

O ² _j = f (I ² _j ) = 1 / ｛1 + exp (−I ² _j )… (7) The output value O ² _j is the value of each element of the output layer (third layer). Input value. Next, in step 304, the output layer (third
The product-sum operation of the input values of each element of the layer is executed.

[0039]

(Equation 8)_s  I^Three _j= ΣW^Two _k, ^Three _j・ O^Two _k+ V^Three _j … (8)^{k = 1}  Next, in step 306, similar to equation (7),
The output value of each element in the output layer is calculated using the Gmoid function.
It is. This output value O^Three _jAre the accelerator and blur, respectively.
Indicates the amount of steering and steering operation. That is, the operation amount O^Three
_jIs obtained by the following equation.

[0040]

[Equation 9] O ³ _j = f (I ³ _j ) = 1 / {1 + exp (-I ³ _j )} (9)

This neural network is similar to that shown in FIG.
The learning is performed according to the procedure shown in FIG. The coupling coefficient is performed by a well-known back propagation method. This learning is performed in advance by inputting various road data and using an optimal operation amount for the road data as a teacher signal in addition to an appropriate operation amount operated by a driver during actual driving of the vehicle as a teacher signal. You can also do it. In this case, as the device is used for a long time, a more appropriate operation amount is required.

In step 400 of FIG. 15, a learning signal δ ³ _j for each element in the output layer is calculated by the following equation.

Δ ³ _j = (T _j −O ³ _j ) · f ^′ (I ³ _j ) (10) where T _j is a teacher signal for the manipulated variable O ³ _{j as} an output, and is externally given. Is done. F ^′ (x) is a derivative of the sigmoid function.

Next, in step 402, the learning signal δ ² _j of the intermediate layer is calculated by the following equation.

[Equation 11] ₃ δ ² _j = f ^′ (I ² _j ) · Σδ ³ _k · W ² _j, ³ _k … (11) ^{k = 1}

Next, in step 404, each coupling coefficient of the output layer is corrected. The correction amount is obtained by the following equation.

Δω ² _i, ³ _j (t) = P · δ ³ _j · O ² _i + Q · Δω ² _i, ³ _j (t-1) (12) where Δω ² _i, ³ _j ( t) is the correction amount of the t-th calculation of the coupling coefficient between the j-th element of the output layer and the i-th element of the intermediate layer. Δω ² _i, ³ _j (t−1) is the previous correction amount of the coupling coefficient. P and Q are proportional constants. Therefore, the coupling coefficient is

[0045]

(13) The corrected coupling coefficient is obtained from W ² _i, ³ _j + Δω ² _i, ³ _j (t) → W ² _i, ³ _j (13) Next, the process proceeds to step 406, where the coupling coefficient of each element of the intermediate layer is corrected. The correction amount of the coupling coefficient is obtained by the following equation, as in the case of the output layer.

[0046]

Δω ¹ _i, ² _j (t) = P · δ ² _j · O ¹ _i + Q · Δω ¹ _i, ² _j (t−1) (14) Therefore, the coupling coefficient is

## EQU15 _{## The} corrected coupling coefficient is obtained from W ¹ _i, ² _j + Δω ¹ _i, ² _j (t) → W ¹ _i, ² _j (15).

Next, in step 408, it is determined whether or not the correction amount of the coupling coefficient has become a predetermined value or less, and it is determined whether or not the coupling coefficient has converged. If the coupling coefficient has not converged, the process returns to step 400, and the same operation is repeated using the newly corrected coupling coefficient, so that the coupling coefficient is corrected again. The learning is completed by repeating such calculations.

Learning of the above neural network is as follows.
It can be executed by traveling a truck or a general road many times as shown in FIG. 6 by a skilled driver, inputting the obtained road data, and using the operation amount by the driver as a teacher signal. .

The neural network 40 shown in FIG.
Are the operation amounts Ac, Br, and St of the accelerator, brake, and steering, but all three operation amounts may be used, or any two or one operation amount may be used. good. The coupling coefficient of the neural network 40 in FIG. 12 is a contribution ratio g _pq in the divided area a _pq . The coupling coefficient naturally includes the influence of the vehicle speed. or,
As shown in FIG. 13, when the neural network 41 having a multilayer structure is used, it is possible to obtain an operation amount in consideration of the interaction between the divided regions and the interaction between the divided region and the vehicle speed.

The operation amount obtained from the neural network at the time of actual traveling using the learned neural network is input to the traveling control device 24. The travel control device 24 drives the accelerator, brake, and steering of the vehicle 1 to optimize the behavior of the vehicle according to the road environment. On the other hand, when used as an alarm device, the output of the neural network 40 is input to the comparison device 26. The operation amount of the vehicle 1 operated by the driver is input to the comparison device 26, and the operation amount is compared with the operation amount output from the neural network 40. When the difference is equal to or more than the predetermined value, the alarm device 28 is driven to warn that the driver's operation is largely out of the operation amount by the neural network 40, thereby causing an operation error due to falling asleep or the like. Can be prevented.

Instead of the neural network 41 shown in FIG. 13, it is possible to use a neural network as shown in FIG. The neural network shown in FIG. 16 adds the amount of steering operation by the driver to the input value. Then, the actual operation amount of the steering, the interaction between the road data and the vehicle speed are taken into account, and the operation amount of the accelerator and the brake is determined.
As described above, since the actual operation amount of the steering is used as the input value of the neural network, when the driver determines the steering operation amount by himself / herself, the operation of the accelerator and the brake is more faithfully performed according to the driver's will. The amount can be determined.

The neural network may have a configuration as shown in FIG. In this case, the inputs of the neural network are road data, vehicle speed, and the operation amount of the accelerator, brake, and steering at the time of the previous control (before a minute time), and the output is the change ΔAc, of the operation amount of the accelerator, brake, and steering. ΔBr and ΔSt. This neural network learns the operation amount, road data, and vehicle speed at the time of the previous control as input data, and a change in the operation amount as a teacher signal. The change in the operation amount of the input data and the change in the operation amount of the output may be any one type or any two types of accelerator, brake, and steering operation amounts.

The neural network may be configured as shown in FIG. With this neural network, past history data can be input, and the output operation amount can be further optimized. That is, the input data of the neural network are the road data and the vehicle speed during the previous control and the road data and the vehicle speed during the current control.
The outputs of the neural network are accelerator, brake,
This is the steering operation amount. Since the road data at the previous control and the vehicle speed are considered including the interaction with those data at the current control, the continuity of the road data and the vehicle speed is taken into account, and a more optimal operation amount is reduced. Desired. This learning of the neural network can be executed by using past road data and vehicle speed as input data and using the operation amount at this time as a teacher signal. Further, in this neural network, the output may be a change in the operation amount from the operation amount in the previous control. In that case, a change in the operation amount is used as the teacher signal.

[0055]

As described above, according to the present invention, when a vehicle is traveling, a road surface is imaged in real time, and road data in which a road portion is specified for each divided region is generated from the captured image. The road data is used as an input value of the neural network, and an operation amount for determining the behavior of the vehicle is used as an output value. Therefore, since the operation amount for determining the behavior of the vehicle can be obtained directly from the road environment, the time for determining the operation amount can be extremely shortened. Further, since the operation amount is determined by the coupling coefficient of the neural network, the operation speed is improved and the storage device capacity is smaller than when the operation amount is determined by an algorithm. Further, since the coupling coefficient of the neural network can be learned in parallel with the actual running of the vehicle, an operation amount optimized by the road environment can be obtained.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of an operation amount determining device according to a specific embodiment of the present invention.

FIG. 2 is a flowchart showing a processing procedure of an image processing device 12 of the operation amount determining device.

FIG. 3 is an explanatory diagram showing an image in which a road portion is specified from an imaging screen.

FIG. 4 is an explanatory diagram showing division of an imaging screen.

FIG. 5 is an explanatory diagram showing a screen composed of a number of divided areas.

FIG. 6 is an explanatory diagram showing a road on which a vehicle travels.

FIG. 7 is an explanatory diagram showing road data generated from a road image.

FIG. 8 is an explanatory diagram showing a configuration of a road network and a command for a traveling direction.

FIG. 9 is a flowchart showing a processing procedure of the image processing apparatus 30.

FIG. 10 is an explanatory diagram showing road data obtained near an intersection.

FIG. 11 is an explanatory diagram showing road data that has been masked in accordance with a command value for a traveling direction.

FIG. 12 is a configuration diagram showing a configuration of a neural network.

FIG. 13 is a configuration diagram showing a configuration of a neural network.

FIG. 14 is a flowchart showing a calculation procedure in the neural network.

FIG. 15 is a flowchart showing a learning procedure of the neural network.

FIG. 16 is a configuration diagram showing a configuration of a neural network of an operation amount determination device according to another embodiment.

FIG. 17 is a configuration diagram showing a configuration of a neural network of an operation amount determination device according to another embodiment.

FIG. 18 is a configuration diagram showing a configuration of a neural network of an operation amount determination device according to another embodiment.

[Explanation of symbols]

Reference Signs List 10 CCD camera 12 Image processing device 20 Road data memory 30 Image processing device 40, 41 Neural network

────────────────────────────────────────────────── ─── Continued on the front page (51) Int.Cl. ⁶ Identification code FI G08G 1/09 G08G 1/09 V H04N 7/18 H04N 7/18 C (73) Patent holder 000000011 Aisin Seiki Co., Ltd. Kariya, Aichi 2-1-1, Asahi-cho, Yokohama-shi (72) Inventor Fumio Matsunari 41 41, Chuchu-Yokomichi, Oku-cho, Nagakute-cho, Aichi-gun, Aichi Prefecture Inside Toyota Central Research Laboratory Co., Ltd. No. 41, Chochu-Yokomichi, Toyota Central Research Institute, Inc. (72) Inventor Hiroyuki Yoshida 41, Nagakute-cho, Aichi-gun, Aichi Prefecture, Japan. No. 41, Toyota Chuo Research Institute, Inc. (72) Inventor Kazuharu Ota Aichi Prefecture 2-1-1 Toyota-cho, Kariya City Inside Toyota Industries Corporation (72) Inventor Go Ishimoto 1-Toyota-cho, Toyota-cho, Toyota-shi, Aichi Prefecture Toyota (72) Inventor Setsuo 1 Toyota Town, Toyota City, Aichi Prefecture Toyota Motor Corporation (72) Inventor Yasutoshi Kato 1-1-1 Showacho, Kariya City, Aichi Prefecture Japan Denso Co., Ltd. (72 Inventor Kenji Asano 2-1-1 Asahi-cho, Kariya-shi, Aichi Aisin Seiki Co., Ltd. (56) References JP-A-63-174199 (JP, A) JP-A-4-12144 (JP, A) JP-A-4-12143 (JP, A) JP-A-4-12142 (JP, A) JP-A-4-11522 (JP, A) JP-A-3-282708 (JP, A) JP-A-3-268110 (JP JP-A-3-235723 (JP, A) JP-A-4-252308 (JP, A) JP-A-2-238600 (JP, A) JP-A-2-238599 (JP, A) (58) Surveyed field (Int.Cl. ⁶ , DB name) B60K 41/28 F02D 29/02 F02D 45/00 370 G06F 15/18 520 G08G 1/04 G08G 1/09

Claims (1)

(57) [Claims] An image of a traveling road surface of a vehicle is taken while the vehicle is traveling, road image information obtained by the imaging is divided into a number of small regions, and a road portion in each of the divided regions is divided. Road data generating means for generating road data indicating presence information of the vehicle, a plurality of input elements for inputting each road data for each of the divided areas corresponding to each of the divided areas, and determining a behavior of the vehicle. And a neural network having an output element that outputs an operation value to be operated.