JP6839642B2 - Vehicle control devices, vehicle control methods, and programs - Google Patents

Description

The present invention relates to vehicle control devices, vehicle control methods, and programs.

A device that is mounted on a vehicle such as an automobile and recognizes the traveling environment of the vehicle by image processing is known (see, for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2004-265432

However, in the conventional technique, the timing of recognizing the driving environment may be delayed.

The present invention has been made in consideration of such circumstances, and one of the objects of the present invention is to provide a vehicle control device, a vehicle control method, and a program capable of recognizing a driving environment at an earlier timing. ..

(1): Whether or not the vehicle control device is in a congested state in the image pickup unit that images the periphery of the vehicle and the sky area corresponding to the space above a predetermined altitude among the areas of the image imaged by the image pickup unit. Is determined, and when it is determined that the sky area is in a congested state, an estimation unit for estimating that a traffic control point where the traffic of the vehicle or a pedestrian is controlled exists in front of the vehicle is provided. The traffic control points include intersections, confluences, branch points, or railroad crossings .

(2): When the estimation unit determines that the sky area is in a congested state, the vehicle control device according to (1) is placed at a position on the road corresponding to the position of the sky area in the congested state. , It is estimated that the traffic control point exists.

(3): In the vehicle control device according to (1) or (2), the sky region is based on the spatial frequency obtained by the estimation unit from the change in the pixel value of the sky region by the two-dimensional Fourier transform. It determines whether or not it is in a congested state.

(4): The vehicle control device according to (3) determines that the sky region is in a congested state when the estimation unit determines that the intensity of a component of the spatial frequency equal to or higher than a predetermined frequency is equal to or higher than a threshold value. To do.

(5): In the vehicle control device according to any one of (1) to (4), the estimation unit further corresponds to the road space in the area of the image captured by the imaging unit. When it is determined whether or not the road surface area is congested and the sky area is congested and the road surface area is congested, one or both of the sky area and the road surface area are determined. It is estimated that the traffic control point exists with a higher degree of certainty than when it is determined that the vehicle is not in a congested state.

(6): The vehicle control device according to any one of (1) to (5) further includes a recognition unit that recognizes a road sign indicating the existence of the traffic control point from the road on which the vehicle is located. When the estimation unit determines that the sky area is in a congested state, the recognition unit recognizes the road sign, as compared with the case where the recognition unit does not recognize the road sign. It is estimated that the traffic control point exists with a high degree of certainty.

(7): The vehicle control device according to any one of (1) to (6) further includes a recognition unit that recognizes a road marking indicating the existence of the traffic control point from the road on which the vehicle is located. When the road marking is recognized by the recognition unit, the estimation unit starts a process of determining whether or not the sky area is in a congested state.

(8): The vehicle control device according to any one of (1) to (5) further includes a recognition unit that recognizes a structure indicating the existence of the traffic control point, and the recognition unit is the recognition unit. When it is estimated by the estimation unit that the traffic control point exists, the recognition process of the structure is started.

(9): The vehicle control device according to any one of (1) to (8) further includes a driving control unit that controls at least acceleration / deceleration of the vehicle, and the driving control unit is the estimation unit. When it is estimated that the traffic control point exists, the vehicle is decelerated.

(10): Whether or not the imaging unit images the periphery of the vehicle, and the estimation unit is in a congested state in the sky region corresponding to the space above a predetermined altitude in the region of the image captured by the imaging unit. When it is determined that the sky area is congested, it is estimated that there is a traffic control point in front of the vehicle in which the traffic of the vehicle or pedestrian is controlled, and the traffic control point is located at the traffic control point. A vehicle control method that includes an intersection, a confluence, a fork, or a railroad crossing.

(11): The computer mounted on the vehicle equipped with the image pickup unit that captures the periphery of the vehicle is in a congested state in the sky area corresponding to the space above a predetermined altitude among the areas of the image captured by the image pickup unit. whether to determine, when the sky region was determined to be congested, the front of the vehicle, it is estimated that the traffic control point vehicle or pedestrian traffic is controlled is present, the A program that includes intersections, confluences, junctions, or railroad crossings as traffic control points.

According to (1) to (11), the traveling environment can be recognized at an earlier timing.

It is a block diagram of the vehicle system 1 using the vehicle control device of 1st Embodiment. It is a functional block diagram of the 1st control unit 120 and the 2nd control unit 160. It is a figure which shows an example of the result of two-dimensional Fourier transform for an image. It is a figure which shows how the target track is generated based on the recommended lane. It is a flowchart which shows an example of the process executed by the automatic operation control device 100 of 1st Embodiment. It is a figure which shows typically the image taken by the camera 10. It is a figure which shows an example of the setting position of a target area. It is a figure which shows an example of the setting position of a target area. It is a figure which shows an example of the setting position of a target area. It is a figure which shows an example of the setting position of a target area. It is a flowchart which shows the other example of the process executed by the automatic operation control apparatus 100 of 1st Embodiment. It is a figure which shows an example of the spatial frequency extracted as a feature of an image from a target area. It is a figure which shows an example of the spatial frequency extracted as a feature of an image from a target area. It is a figure which shows an example of the predetermined road marking. It is a figure which shows another example of the setting position of a target area. It is a flowchart which shows an example of the process executed by the automatic operation control device 100 of 2nd Embodiment. It is a figure which shows an example of the contrast between a texture and a power spectrum image. It is a figure which shows an example of the contrast between a texture and a power spectrum image. It is a figure which shows an example of the contrast between a texture and a power spectrum image. It is a figure for demonstrating the setting method of the threshold value which allows the power spectrum value to be equal to _{or more than a spectrum threshold value STH.} It is a figure for demonstrating the setting method of the threshold value which allows the power spectrum value to be equal to _{or more than a spectrum threshold value STH.} It is a figure which shows an example of the hardware composition of the automatic operation control device 100 of an embodiment.

Hereinafter, embodiments of the vehicle control device, vehicle control method, and program of the present invention will be described with reference to the drawings. In the following embodiment, the vehicle control device will be described as being applied to a vehicle capable of automatic driving (autonomous driving). The automatic driving is, for example, a mode in which the vehicle is driven by controlling one or both of the steering and acceleration / deceleration of the vehicle without depending on the operation of the occupant on the vehicle. The automatic driving may include driving assistance such as ACC (Adaptive Cruse Control) and LKAS (Lane Keeping Assist).

<First Embodiment>
[overall structure]
FIG. 1 is a configuration diagram of a vehicle system 1 using the vehicle control device of the first embodiment. The vehicle on which the vehicle system 1 is mounted (hereinafter referred to as the own vehicle M) is, for example, a vehicle such as a two-wheeled vehicle, a three-wheeled vehicle, or a four-wheeled vehicle, and its drive source is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or the like. Alternatively, it is a combination of these. When the electric motor is provided, the electric motor operates by using the electric power generated by the generator connected to the internal combustion engine or the electric power generated by the secondary battery or the fuel cell.

The vehicle system 1 includes, for example, a camera 10, a radar device 12, a finder 14, an object recognition device 16, a communication device 20, an HMI (Human Machine Interface) 30, a vehicle sensor 40, a navigation device 50, and the like. It includes an MPU (Map Positioning Unit) 60, a driving operator 80, an automatic driving control device 100, a traveling driving force output device 200, a braking device 210, and a steering device 220. These devices and devices are connected to each other by a multiplex communication line such as a CAN (Controller Area Network) communication line, a serial communication line, a wireless communication network, or the like. The configuration shown in FIG. 1 is merely an example, and a part of the configuration may be omitted or another configuration may be added.

The camera 10 is, for example, a digital camera using a solid-state image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). One or a plurality of cameras 10 are attached to an arbitrary position of a vehicle (hereinafter, referred to as own vehicle M) on which the vehicle system 1 is mounted. When photographing the front, the camera 10 is attached to the upper part of the front windshield, the back surface of the rearview mirror, and the like. The camera 10 periodically and repeatedly images the periphery of the own vehicle M, for example. The camera 10 may be a stereo camera. The camera 10 is an example of an “imaging unit”.

The radar device 12 radiates radio waves such as millimeter waves around the own vehicle M, and detects radio waves (reflected waves) reflected by the object to detect at least the position (distance and orientation) of the object. One or a plurality of radar devices 12 may be attached to any position of the own vehicle M. The radar device 12 may detect the position and velocity of the object by the FM-CW (Frequency Modulated Continuous Wave) method.

The finder 14 is a LIDAR (Light Detection and Ranging). The finder 14 irradiates the periphery of the own vehicle M with light and measures the scattered light. The finder 14 detects the distance to the target based on the time from light emission to light reception. The emitted light is, for example, a pulsed laser beam. One or a plurality of finder 14s may be attached to any position of the own vehicle M.

The object recognition device 16 performs sensor fusion processing on the detection results of a part or all of the camera 10, the radar device 12, and the finder 14, and recognizes the position, type, speed, and the like of the object. The object recognition device 16 outputs the recognition result to the automatic operation control device 100. Further, the object recognition device 16 may output the detection results of the camera 10, the radar device 12, and the finder 14 to the automatic driving control device 100 as they are, if necessary.

The communication device 20 communicates with another vehicle m existing in the vicinity of the own vehicle M by using, for example, a cellular network, a Wi-Fi network, Bluetooth (registered trademark), DSRC (Dedicated Short Range Communication), or the like. Communicates with various server devices via a wireless base station. Like the own vehicle M, the other vehicle m may be a vehicle that is automatically driven or a vehicle that is manually driven, and there are no particular restrictions. The manual operation is different from the automatic operation described above, and means that the acceleration / deceleration and steering of the own vehicle M are controlled according to the operation of the occupant with respect to the driving operator 80.

The HMI 30 presents various information to the occupants of the own vehicle M and accepts input operations by the occupants. The HMI 30 includes various display devices, speakers, buzzers, touch panels, switches, keys and the like.

The vehicle sensor 40 includes a vehicle speed sensor that detects the speed of the own vehicle M, an acceleration sensor that detects the acceleration, a yaw rate sensor that detects the angular velocity around the vertical axis, an orientation sensor that detects the direction of the own vehicle M, and the like.

The navigation device 50 includes, for example, a GNSS (Global Navigation Satellite System) receiver 51, a navigation HMI 52, and a routing unit 53, and the first map information 54 is stored in a storage device such as an HDD (Hard Disk Drive) or a flash memory. Holds. The GNSS receiver 51 identifies the position of the own vehicle M based on the signal received from the GNSS satellite. The position of the own vehicle M may be specified or complemented by an INS (Inertial Navigation System) using the output of the vehicle sensor 40. The navigation HMI 52 includes a display device, a speaker, a touch panel, keys, and the like. The navigation HMI 52 may be partially or wholly shared with the above-mentioned HMI 30. The route determination unit 53, for example, has a route from the position of the own vehicle M (or an arbitrary position input) specified by the GNSS receiver 51 to the destination input by the occupant using the navigation HMI 52 (hereinafter, hereafter). The route on the map) is determined with reference to the first map information 54. The first map information 54 is, for example, information in which a road shape is expressed by a link indicating a road and a node connected by the link. The first map information 54 may include road curvature, POI (Point Of Interest) information, and the like. The map route determined by the route determination unit 53 is output to the MPU 60. Further, the navigation device 50 may perform route guidance using the navigation HMI 52 based on the map route determined by the route determination unit 53. The navigation device 50 may be realized by, for example, the function of a terminal device such as a smartphone or a tablet terminal owned by an occupant. Further, the navigation device 50 may transmit the current position and the destination to the navigation server via the communication device 20 and acquire the route on the map returned from the navigation server.

The MPU 60 functions as, for example, a recommended lane determination unit 61, and holds the second map information 62 in a storage device such as an HDD or a flash memory. The recommended lane determination unit 61 divides the route provided by the navigation device 50 into a plurality of blocks (for example, divides the route every 100 [m] with respect to the vehicle traveling direction), and refers to the second map information 62 for each block. Determine the recommended lane. The recommended lane determination unit 61 determines which lane to drive from the left. The recommended lane determination unit 61 determines the recommended lane so that the own vehicle M can travel on a reasonable route to proceed to the branch destination when there is a branch point, a merging point, or the like on the route.

The second map information 62 is more accurate map information than the first map information 54. The second map information 62 includes, for example, information on the center of the lane, information on the boundary of the lane, and the like. Further, the second map information 62 may include road information, traffic regulation information, address information (address / zip code), facility information, telephone number information, and the like. The second map information 62 may be updated at any time by accessing another device using the communication device 20.

The driving controller 80 includes, for example, an accelerator pedal, a brake pedal, a shift lever, a steering wheel, a deformed steering wheel, a joystick, and other controls. A sensor for detecting the amount of operation or the presence or absence of operation is attached to the operation operator 80, and the detection result is the automatic operation control device 100, or the traveling driving force output device 200, the brake device 210, and the steering device. It is output to a part or all of 220.

The automatic operation control device 100 includes, for example, a first control unit 120 and a second control unit 160. The first control unit 120 includes a feature extraction unit 132 and an estimation unit 134. Each component of the first control unit 120 and the second control unit 160 is realized by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). In addition, some or all of these components are hardware (circuits) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), and GPU (Graphics Processing Unit). It may be realized by the part; including circuitry), or it may be realized by the cooperation of software and hardware.

FIG. 2 is a functional configuration diagram of the first control unit 120 and the second control unit 160. The first control unit 120 includes, for example, a recognition unit 130 and an action plan generation unit 140. The first control unit 120, for example, realizes a function by AI (Artificial Intelligence) and a function by a model given in advance in parallel. For example, the function of "recognizing an intersection" is executed in parallel with the recognition of an intersection by deep learning or the like and the recognition based on predetermined conditions (there are signals that can be pattern matched, road markings, etc.), and both are executed. It is realized by scoring and comprehensively evaluating. This ensures the reliability of autonomous driving.

The recognition unit 130 determines the position, speed, acceleration, and other states of objects around the own vehicle M based on the information input from the camera 10, the radar device 12, and the finder 14 via the object recognition device 16. recognize. Objects include other vehicles m and stationary obstacles. The position of the object is recognized as, for example, a position on absolute coordinates with the representative point (center of gravity, center of drive axis, etc.) of the own vehicle M as the origin, and is used for control. The position of the object may be represented by a representative point such as the center of gravity or a corner of the object, or may be represented by a represented area. The "state" of an object may include acceleration or jerk of the object, or "behavioral state" (eg, whether or not the vehicle is changing lanes or is about to change lanes). Further, the recognition unit 130 recognizes the shape of the curve that the own vehicle M is about to pass based on the image captured by the camera 10. The recognition unit 130 converts the shape of the curve from the image captured by the camera 10 into a real plane, and for example, two-dimensional point sequence information or information expressed using a model equivalent thereto is information indicating the shape of the curve. Is output to the action plan generation unit 140.

Further, the recognition unit 130 recognizes, for example, the lane (traveling lane) in which the own vehicle M is traveling. For example, the recognition unit 130 has a road marking line pattern (for example, an arrangement of a solid line and a broken line) obtained from the second map information 62 and a road marking line around the own vehicle M recognized from the image captured by the camera 10. By comparing with the pattern of, the driving lane is recognized. The recognition unit 130 may recognize the traveling lane by recognizing not only the road marking line but also the running road boundary (road boundary) including the road marking line, the shoulder, the curb, the median strip, the guardrail, and the like. .. In this recognition, the position of the own vehicle M acquired from the navigation device 50 and the processing result by the INS may be added. In addition, the recognition unit 130 recognizes road markings drawn on the road surface such as a temporary stop line, road signs, obstacles, red lights, tollhouses, and other road events.

When recognizing the traveling lane, the recognition unit 130 recognizes the position and posture of the own vehicle M with respect to the traveling lane. The recognition unit 130 determines, for example, the deviation of the reference point of the own vehicle M from the center of the lane and the angle formed by the center of the lane in the traveling direction of the own vehicle M with respect to the relative position of the own vehicle M with respect to the traveling lane. And may be recognized as a posture. Further, instead of this, the recognition unit 130 sets the position of the reference point of the own vehicle M with respect to any side end portion (road division line or road boundary) of the traveling lane, and the relative position of the own vehicle M with respect to the traveling lane. May be recognized as.

The feature extraction unit 132 of the recognition unit 130 extracts features from an image (digital image) captured by the camera 10. Features include, for example, the spatial frequency of an image, which is a two-dimensional space, the edge density of luminance or color, the variance of a histogram of luminance or color, and the like. For example, the feature extraction unit 132 performs a two-dimensional Fourier transform such as FFT (Fast Fourier Transform) on the image, and changes the pixel value in the horizontal direction of the image (periodicity of the change in the pixel value) and the vertical direction. Each of the changes in the pixel value (periodicity of the change in the pixel value) is converted into an angular frequency. Hereinafter, the angular frequency in the horizontal direction is expressed as μ, and the angular frequency in the vertical direction is expressed as ν.

For example, the feature extraction unit 132 sets an area (hereinafter, referred to as a detection window) of a predetermined size (a predetermined aspect ratio and a predetermined area) with respect to the image, and changes the position of the set detection window. However, the spatial frequency of the image area that overlaps the detection window is extracted by the two-dimensional Fourier transform. For example, the predetermined size is set to a size smaller than the target area described later.

FIG. 3 is a diagram showing an example of the result of the two-dimensional Fourier transform on the image. In the illustrated example, the power spectrum, which is the absolute value of the squares of the spectra of the angular frequency μ and the angular frequency ν, is represented by a grayscale image. The center O of the image represents the origin where both the angular frequency μ and the angular frequency ν are zero. The closer to this point O, the more the power spectrum of the low frequency component, and the farther it is, the more the power spectrum of the high frequency component. .. In the power spectrum image, the larger the power spectrum, the whiter the image, and the smaller the power spectrum, the blacker the image.

The estimation unit 134 of the recognition unit 130 estimates that the traffic control point exists in front of the own vehicle M based on the features extracted by the feature extraction unit 132. The traffic control point is a point where traffic such as a vehicle or a pedestrian is controlled, and includes, for example, an intersection, a confluence point, a branch point, a railroad crossing, and the like. Further, the estimation unit 134 may estimate that a structure exists in the vicinity of the traffic control point. The structure around the traffic control point may be a three-dimensional physical object such as a traffic light, a vehicle detector, a traffic information board, a road sign, a guard rail, a street light, or a curved mirror. It may be an object with a two-dimensional substance such as a road sign.

Further, the estimation unit 134 may derive the certainty of the estimation when estimating the existence of the traffic control point. The degree of certainty is, for example, an index showing the degree of probability that a traffic control point exists.

In principle, the action plan generation unit 140 travels in the recommended lane determined by the recommended lane determination unit 61, and further determines events to be sequentially executed in automatic driving so as to be able to respond to the surrounding conditions of the own vehicle M. To do. Events include, for example, a constant-speed driving event in which the vehicle travels in the same lane at a constant speed, a follow-up driving event in which the vehicle travels at a constant speed while following the vehicle in front, an overtaking event in which the vehicle in front is overtaken, and avoiding approaching an obstacle. Avoidance events for braking and / or steering, curve driving events, passing events passing through traffic control points such as intersections, pedestrian crossings, and crossings, lane change events, merging events, branching events, automatic stops There are events, takeover events for ending automatic operation and switching to manual operation. Follow-up travel is, for example, the relative distance (inter-vehicle distance) between the own vehicle and the preceding vehicle by accelerating or decelerating the own vehicle within a predetermined set vehicle speed (for example, 50 to 100 [km / h]). ) Is a running mode that is kept constant.

The action plan generation unit 140 generates a target trajectory on which the own vehicle M will travel in the future in response to the activated event. Details of each functional unit will be described later. The target trajectory includes, for example, a velocity element. For example, the target track is expressed as a sequence of points (track points) to be reached by the own vehicle M. The track point is a point to be reached by the own vehicle M for each predetermined mileage (for example, about several [m]) along the road, and separately, for a predetermined sampling time (for example, about 0 comma [sec]). ) Target velocity and target acceleration are generated as part of the target trajectory. Further, the track point may be a position to be reached by the own vehicle M at the sampling time for each predetermined sampling time. In this case, the information of the target velocity and the target acceleration is expressed by the interval of the orbital points.

FIG. 4 is a diagram showing how a target trajectory is generated based on the recommended lane. As shown, the recommended lanes are set to be convenient for driving along the route to the destination. The action plan generation unit 140 activates a passing event, a lane change event, a branching event, a merging event, and the like when approaching a predetermined distance (which may be determined according to the type of event) of the recommended lane switching point. If it becomes necessary to avoid an obstacle during the execution of each event, an avoidance trajectory is generated as shown in the figure.

The second control unit 160 sets the traveling driving force output device 200, the brake device 210, and the steering device 220 so that the own vehicle M passes the target trajectory generated by the action plan generation unit 140 at the scheduled time. Control.

Returning to FIG. 2, the second control unit 160 includes, for example, an acquisition unit 162, a speed control unit 164, and a steering control unit 166. The combination of the action plan generation unit 140, the speed control unit 164, and the steering control unit 166 is an example of the "operation control unit".

The acquisition unit 162 acquires the information of the target trajectory (orbit point) generated by the action plan generation unit 140 and stores it in a memory (not shown).

The speed control unit 164 controls the traveling driving force output device 200 or the braking device 210 based on the speed element associated with the target trajectory stored in the memory.

The steering control unit 166 controls the steering device 220 according to the degree of bending of the target trajectory stored in the memory. The processing of the speed control unit 164 and the steering control unit 166 is realized by, for example, a combination of feedforward control and feedback control. As an example, the steering control unit 166 executes a combination of feedforward control according to the curvature of the road in front of the own vehicle M and feedback control based on the deviation from the target trajectory.

The traveling driving force output device 200 outputs a traveling driving force (torque) for traveling the vehicle to the drive wheels. The traveling driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, a transmission, and the like, and an ECU (Electronic Control Unit) that controls them. The ECU controls the above configuration according to the information input from the second control unit 160 or the information input from the operation operator 80.

The brake device 210 includes, for example, a brake caliper, a cylinder that transmits flood pressure to the brake caliper, an electric motor that generates flood pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor according to the information input from the second control unit 160 or the information input from the operation operator 80 so that the brake torque corresponding to the braking operation is output to each wheel. The brake device 210 may include, as a backup, a mechanism for transmitting the oil pressure generated by the operation of the brake pedal included in the operation operator 80 to the cylinder via the master cylinder. The brake device 210 is not limited to the configuration described above, and is an electronically controlled hydraulic brake device that controls an actuator according to information input from the second control unit 160 to transmit the oil pressure of the master cylinder to the cylinder. May be good.

The steering device 220 includes, for example, a steering ECU and an electric motor. The electric motor, for example, applies a force to the rack and pinion mechanism to change the direction of the steering wheel. The steering ECU drives the electric motor according to the information input from the second control unit 160 or the information input from the operation controller 80, and changes the direction of the steering wheel.

[Feature extraction processing flow]
FIG. 5 is a flowchart showing an example of processing executed by the automatic operation control device 100 of the first embodiment. The processing of this flowchart is, for example, repeated at a predetermined cycle.

First, the feature extraction unit 132 sets a detection window for a certain target area included in the image captured by the camera 10 (step S100).

FIG. 6 is a diagram schematically showing an image captured by the camera 10. For example, when the road on which the own vehicle M travels is a straight road, a plurality of lane markings (LN1, LN2, etc.) that divide the lanes of the road intersect near the vanishing point P of the image. In such a case, if there is a traffic light, a traffic information guide plate, or the like at a distant position in front of the vehicle M in the traveling direction, the traffic light or the traffic light is located in a very narrow image area (for example, the area indicated by R in the figure) near the vanishing point P. An image area showing a traffic information information board or the like will be included. Therefore, the region near the vanishing point P, where these structures are likely to exist, is determined as the target region so that the traffic signal, the traffic information guide plate, and the like can be recognized with high accuracy.

7 to 10 are views showing an example of the setting position of the target area. For example, as illustrated in FIG. 7, the feature extraction unit 132 regards the point where the lane markings LN1 and LN2 that partition the own vehicle M intersect with each other as the vanishing point P, and from the horizontal line HOR passing through the vanishing point P. Also determines the image area on the upper side in the vertical direction as the target area. As a result, the image area for which the detection window is set becomes smaller, so that the processing load of feature extraction can be reduced. Hereinafter, the image region above the horizontal line HOR in the vertical direction will be referred to as an “sky region” and will be described.

Further, as illustrated in FIG. 8, the feature extraction unit 132 is above the horizontal line HOR passing through the vanishing point P in the vertical direction, and the length (width) in the horizontal direction is equal to or less than _{the first predetermined distance D HOR.} An image area that satisfies the conditions may be determined as the target area. In the illustrated example, an image region included in a range of _{equidistant 1 / 2D HOR on} each of the left and right sides is determined as a target region with reference to the vertical line VERT passing through the vanishing point P. If the condition that the length (width) in the horizontal direction is equal to _{or less than the first predetermined distance D HOR} is satisfied, the right side of the vertical line VERT is 2 / 3D _HOR , the left side is 1 / 3D _HOR , and so on. The target area may be biased to the left or right with respect to the line VERT. As a result, the image area for which the detection window is set can be made smaller, and the processing load of feature extraction can be further reduced.

Further, as illustrated in FIG. 9, the feature extraction unit 132 _{intersects a point P # having a second predetermined distance D VERT in} the vertical direction from the horizontal line HOR passing through the vanishing point P, the horizontal line HOR, and the image end portion. A triangle formed by a side connecting a total of three points with two points is derived, and the image area of the derived triangle is removed from the image area perpendicular to the horizontal line HOR passing through the vanishing point P, and the remaining image. The area may be determined as the target area. As a result, the detection window can be set in consideration of the height of the traffic light or the like in the three-dimensional space (real space) to be imaged by the camera 10.

Further, as illustrated in FIG. 10, the feature extraction unit 132 is an image area excluding the triangular image area, is vertically above the horizontal line HOR passing through the vanishing point P, and has a horizontal length. An image region satisfying the condition that the (width) is equal to or less than the first predetermined distance D _HOR may be determined as the target region. As a result, the height of the traffic light or the like can be taken into consideration in the real space, and the image area to be set by the detection window can be reduced. As a result, the processing load of feature extraction can be further reduced.

Next, the feature extraction unit 132 cuts out an image area that overlaps with the detection window set for the target area, and extracts features from the cut out image area (step S102). For example, the feature extraction unit 132 performs a two-dimensional Fourier transform on the cut out image region to extract the spatial frequency. Further, the feature extraction unit 132 may extract the edge density of the luminance or the color, the variance of the histogram of the luminance or the color, and the like.

Next, the feature extraction unit 132 determines whether or not the detection window is set for the entire area of the target area (step S104), and determines that the detection window is not set for the entire area of the target area. If so, the process returns to S100, and the detection window is reset to a different position from the previously set position of the detection window. The different positions are, for example, positions where the detection window set last time and the detection window set this time do not overlap each other.

On the other hand, when the feature extraction unit 132 determines that the detection window is set for the entire region of the target region, that is, when the spatial frequency of the entire region of the target region is derived, the processing of this flowchart ends.

[Estimation processing flow for the existence of traffic control points]
FIG. 11 is a flowchart showing another example of the process executed by the automatic operation control device 100 of the first embodiment. The processing of this flowchart may be repeated, for example, at a predetermined cycle.

First, the estimation unit 134 determines whether or not a predetermined feature has been extracted from the sky region determined as the target region by the feature extraction unit 132 (step S200). The predetermined feature is, for example, a spatial frequency in which the power spectrum value or the spectral intensity is equal to or higher than a certain threshold value in a high frequency region. Further, the predetermined feature may be an edge having a density equal to or higher than the threshold value, a histogram of brightness or color having a variance equal to or higher than the threshold value, or another feature, or a combination thereof. It may be.

For example, as shown in FIG. 6 described above, when a traffic light is present in front of the own vehicle M, a pole (telegraph pillar or the like) that supports the traffic light is used to set the target area in the sky area above the vanishing point P. ) And multiple electric wires that supply power to traffic lights are likely to be included in the target area. Artificial objects such as these poles and electric wires generally have many linear components and have a certain direction with respect to the image, whereas natural objects such as trees, soil, and the sky have few linear components. It tends to have no direction. Therefore, when a natural object such as a tree, soil, or the sky exists as a background of an artificial object such as a pole or an electric wire, the boundary of the artificial object with respect to the background tends to be clear. Therefore, the spatial frequency extracted as a feature of the image from the target area contains a large amount of high-frequency components, and when the search window is set in the target area, a predetermined feature can be easily extracted.

12 and 13 are diagrams showing an example of the spatial frequency extracted as a feature of the image from the target region. In the figure, the horizontal axis represents the angular frequency μ which is the horizontal spatial frequency of the image or the angular frequency ν which is the vertical spatial frequency of the image, and the vertical axis represents the power spectrum. In the example of FIG. 12, in the low frequency region below a _{certain frequency threshold FTH} , the power spectrum is equal to _{or higher than a certain threshold STH (hereinafter referred to as spectrum threshold STH} ), and in the high frequency region above the _{frequency threshold FTH.} , The power spectrum is less than _{the spectrum threshold STH.} In such a case, the estimation unit 134 determines that a predetermined feature has not been extracted from the target region by the feature extraction unit 132.

On the other hand, in the example of FIG. 13 _{, the power spectrum is equal to} or higher than the spectral threshold STH in both the low frequency region below a _{certain frequency threshold FTH} and the high frequency region above the _{frequency threshold FTH.} In such a case, the estimation unit 134 determines that a predetermined feature has been extracted from the target region by the feature extraction unit 132.

In the above example, a spectrum threshold value _STH having the same magnitude is uniformly provided for the power spectrum of _{each frequency, and it is determined whether or not the power spectrum is equal to or higher than the spectrum threshold value STH} , but the present invention is limited to this. Absent. For example, the estimation unit 134 may make the spectrum threshold value _STH to be compared smaller as the frequency becomes higher (high frequency power spectrum).

When the estimation unit 134 determines that the predetermined feature is not extracted from the target area, it estimates that the traffic control point does not exist in front of the own vehicle M (step S202).

On the other hand, when the estimation unit 134 determines that a predetermined feature has been extracted from the target region, the estimation unit 134 determines whether or not there are a predetermined number or more of image regions (search windows) from which the predetermined feature has been extracted (step S204). ).

For example, when there are a predetermined number or more of image regions from which a predetermined feature is extracted, the target region contains a large amount of spatial frequencies of high frequency components. As described above, the higher the spatial frequency, the higher the probability that multiple poles will be installed or multiple electric wires will be stretched in the three-dimensional space corresponding to the target area of the image. In many cases, the landscape feels cluttered (crowded) when the space is visually recognized. That is, it is determined that the image area is in a congested state. When structures are overflowing above the road so that humans feel cluttered, there is a high possibility that there are structures such as traffic lights, traffic information boards, and road signs. Therefore, the estimation unit 134 determines that the traffic control point exists in front of the own vehicle M when there are a predetermined number or more of image regions from which a predetermined feature is extracted, that is, when the target region is in a congested state. Estimate (step S206).

For example, the estimation unit 134 estimates that the traffic control point exists at a position in the three-dimensional space (imaging space of the camera 10) corresponding to the position of the target area in the congested state. The "position in the three-dimensional space corresponding to the position of the target area" is, for example, a position when the depth direction of the image is converted into the traveling direction of the own vehicle M. Therefore, when the own vehicle M is traveling on the road, the estimation unit 134 estimates that the traffic control point exists around the road on the front side when viewed from the traveling direction of the own vehicle M.

Next, when the automatic driving control device 100 determines that the traffic control point exists in front of the own vehicle M, the automatic driving control device 100 performs a predetermined process for preparing for the own vehicle M to relatively approach the traffic control point. Start (step S208). This ends the processing of this flowchart.

The predetermined process is a part of various processes such as recognizing a two-dimensional object such as a road marking, recognizing a three-dimensional object such as a traffic light, and decelerating the own vehicle M. Or include all.

For example, when the predetermined process is to recognize a two-dimensional object, the recognition unit 130 starts the process of recognizing the predetermined road marking from the road surface of the own lane. For example, the recognition unit 130 recognizes symbols and characters having features such as a predetermined shape and a predetermined hue as road markings by image processing such as pattern matching.

FIG. 14 is a diagram showing an example of a predetermined road marking. For example, certain road markings include various markings such as the arrow symbols MK1 and MK2 indicating the direction in which the vehicle can travel, and the diamond symbol MK3 indicating that there is a pedestrian crossing or a bicycle crossing zone ahead. For example, when a two-dimensional object having the same shape as these illustrated symbols exists in the image region corresponding to the road surface, the recognition unit 130 recognizes this as a predetermined road marking.

When the predetermined process is to recognize a three-dimensional object, the recognition unit 130 starts a process of recognizing an object such as a traffic light, a traffic information guide plate, or a road sign. For example, the recognition unit 130 recognizes the various objects described above by image processing such as pattern matching.

Further, when the predetermined process is to decelerate the own vehicle M, the action plan generation unit 140 reduces the target speed and the target acceleration included as speed elements in the target trajectory. As a result, the speed control unit 164 controls the traveling driving force output device 200 or the brake device 210 based on the speed element associated with the target trajectory, so that the own vehicle M decelerates.

The processing of the flowchart shown in FIG. 11 has been described as being repeatedly performed at predetermined intervals, but the present invention is not limited to this, and may be started when a predetermined road marking is recognized by the recognition unit 130. As a result, the processing load can be reduced because the existence of the traffic control point is estimated based on the spatial frequency only when it can be determined that there is a high probability that the traffic control point exists.

According to the first embodiment described above, whether or not the camera 10 that captures the periphery of the own vehicle M and the sky area corresponding to the space above a predetermined altitude among the image areas of the camera 10 are in a congested state. When it is determined that the sky area is congested, the driving environment is recognized at an earlier timing by providing the estimation unit 134 that estimates that the traffic control point exists in front of the own vehicle M. be able to.

In general, when trying to recognize a structure (traffic light, etc.) that exists far away from an image captured in front, an object that exists at a distant position becomes a subject that is relatively small compared to a nearby object. In the image of the subject, the image area is relatively narrow (the area is small) with respect to the entire image area. Therefore, the number of pixels occupying the subject in the image area to be processed tends to decrease, and when recognizing an object located far away from the own vehicle M from the image, it is determined that some object exists at that position. Even if it can be recognized, it tends to be difficult to recognize what kind or shape the object is. Further, when the resolution of an image is increased in order to increase the number of pixels, the load of the recognition process increases as the resolution increases, so that there is a concern that the recognition of a structure such as a traffic light may be delayed.

On the other hand, in the first embodiment, the structure located far away from the own vehicle M is not recognized from the shape or the like, but the signal is based on the spatial frequency obtained by the two-dimensional Fourier transform of the image. Since it is estimated that such a structure exists, the driving environment can be recognized at an earlier timing. In addition, since it is estimated that a structure such as a signal is present based on the spatial frequency, if there is a high probability that the structure will exist even if the structure such as a signal is not actually recognized, image processing such as pattern matching is performed. Since the process of recognizing the structure is started by the above, the driving environment can be recognized while reducing the image processing load. Further, if it is estimated that a structure such as a traffic light exists based on the spatial frequency and it can be determined that there is a high probability that the structure exists, the own vehicle M is started to be decelerated as a predetermined process. When the speed of the own vehicle M is lower, the shape of a structure such as a traffic light can be recognized. As a result, the traffic control point can be recognized more accurately.

<Second Embodiment>
Hereinafter, the second embodiment will be described. In the above-described first embodiment, it has been described that the target area is set in the sky area above the horizon. On the other hand, the second embodiment is different from the above-described first embodiment in that the target area is set in the area below the horizon in addition to or instead of the sky area above the horizon. .. Hereinafter, the differences from the first embodiment will be mainly described, and the description of the functions and the like common to the first embodiment will be omitted.

FIG. 15 is a diagram showing another example of the setting position of the target area. For example, as illustrated in FIG. 15, the feature extraction unit 132 regards the point where the lane markings LN1 and LN2 that partition the own vehicle M intersect with each other as the vanishing point P, and from the horizontal line HOR passing through the vanishing point P. The target area is both the upper sky area in the vertical direction and the image area of the own lane (hereinafter referred to as the road surface area) which is below the horizontal line HOR in the vertical direction and is surrounded by the lane markings LN1 and LN2. decide. As a result, the spatial frequency can be extracted as a feature of the image from both the sky region and the road surface region.

FIG. 16 is a flowchart showing an example of processing executed by the automatic operation control device 100 of the second embodiment. The processing of this flowchart may be repeated, for example, at a predetermined cycle.

First, the estimation unit 134 determines whether or not a predetermined feature has been extracted from the sky region determined as the target region by the feature extraction unit 132 (step S300). When the estimation unit 134 determines that a predetermined feature has not been extracted from the sky region, it estimates that the traffic control point does not exist in front of the own vehicle M (step S302).

On the other hand, when it is determined that a predetermined feature is extracted from the sky region, the estimation unit 134 determines whether or not there are a predetermined number or more of image regions (search windows) from which the predetermined feature is extracted in the sky region ( Step S304). When the number of image areas from which predetermined features are extracted is less than a predetermined number and the sky area is not congested, the estimation unit 134 proceeds to the process of S302 and estimates that the traffic control point does not exist in front of the own vehicle M. ..

On the other hand, in the estimation unit 134, when the number of image regions from which a predetermined feature is extracted is a predetermined number or more and the sky region is in a congested state, the estimation unit 134 further determines from the road surface region determined as the target region by the feature extraction unit 132. It is determined whether or not the feature has been extracted (step S306). When the estimation unit 134 determines that the predetermined feature has not been extracted from the road surface region, the estimation unit 134 proceeds to the process of S302 and estimates that the traffic control point does not exist in front of the own vehicle M.

On the other hand, when it is determined that the predetermined feature is extracted from the road surface region, the estimation unit 134 determines whether or not there are a predetermined number or more of the image regions from which the predetermined feature is extracted in the road surface region (step S308). When the number of image regions from which predetermined features are extracted is less than a predetermined number and the road surface region is not in a congested state, the estimation unit 134 proceeds to the process of S302 and estimates that there is no traffic control point in front of the own vehicle M. ..

On the other hand, the estimation unit 134 estimates that a traffic control point exists in front of the own vehicle M when the number of image regions from which predetermined features are extracted is a predetermined number or more and the road surface region is in a congested state (). Step S310).

Next, when the automatic driving control device 100 determines that the traffic control point exists in front of the own vehicle M, the automatic driving control device 100 performs a predetermined process for preparing for the own vehicle M to relatively approach the traffic control point. Start (step S312). This ends the processing of this flowchart.

By such processing, it is possible to evaluate whether or not a three-dimensional structure such as a traffic light exists in the sky area, and also evaluate whether or not a two-dimensional object such as a road marking exists in the road surface area. can do.

In the flowchart shown in FIG. 16 described above, the processing order of S300 and S304 and the processing of S306 and S308 may be opposite to each other.

Further, in the flowchart shown in FIG. 16 described above, when there are a predetermined number or more of image regions from which predetermined features are extracted in both the sky region and the road surface region, a traffic control point exists in front of the own vehicle M. If the predetermined feature is not extracted from any of the regions, or if the predetermined feature is extracted but the image region from which the predetermined feature is extracted is considered to be congested. The explanation is made assuming that the traffic control point does not exist in front of the own vehicle M when the number (predetermined number) has not been reached, but the present invention is not limited to this.

For example, the estimation unit 134 derives as a score how much a predetermined feature is extracted in both the sky region and the road surface region, and based on the simple average or the weighted average of the score, is in front of the own vehicle M. Estimate that a traffic control point exists. For example, the estimation unit 134 derives the number of image regions from which a predetermined feature is extracted from the sky region as a score A, and the number of image regions from which a predetermined feature is extracted from the road surface region as a score B. .. Then, the estimation unit 134 obtains an average such as {(1-α) A + αB} / 2 (α is an arbitrary constant), and uses this average value as the certainty of estimation. As a result, the estimation unit 134 controls traffic in front of the own vehicle M with the highest certainty when, for example, in both the sky region and the road surface region, the number of image regions from which predetermined features are extracted is a predetermined number or more. When it is estimated that a point exists and the number of image regions from which predetermined features are extracted is equal to or greater than the predetermined number in only one of the regions, the predetermined features are extracted in both the sky region and the road surface region. It is presumed that the traffic control point exists in front of the own vehicle M with a lower certainty than the case where the image area is equal to or larger than a predetermined number.

For example, when the predetermined process is to decelerate the own vehicle M, the action plan generation unit 140 increases the certainty when the traffic control point is estimated by the estimation unit 134, the greater the certainty of the own vehicle. A target trajectory that slows down M may be generated.

Further, for example, when the predetermined process is to recognize a two-dimensional or three-dimensional object, the recognition unit 130 may shorten the cycle of repeating the process of recognizing the object by image processing such as pattern matching.

Further, the flowchart shown in FIG. 16 described above may be started when a predetermined road marking is recognized by the recognition unit 130, as in the first embodiment described above. In this case, the estimation unit 134 determines when the number of image areas from which the predetermined features are extracted exceeds a predetermined number in one or both of the sky area and the road surface area while the predetermined road marking is recognized. In the state where the road markings are not recognized, the own vehicle has a higher degree of certainty than the case where the number of image areas from which predetermined features are extracted exceeds a predetermined number in one or both of the sky area and the road surface area. It may be estimated that there is a traffic control point in front of M.

According to the second embodiment described above, in the image region of the camera 10, the self is comprehensively based on the spatial frequencies of the sky region above the horizon and the road surface region below the horizon. Since it is estimated whether or not a traffic control point exists in front of the vehicle M, the traveling environment can be recognized more accurately.

<Third Embodiment>
Hereinafter, the third embodiment will be described. In the first and second embodiments described above, as an example, a spatial frequency having a power spectrum value or a spectral intensity equal to or higher than a certain threshold value is extracted as a predetermined feature in a high frequency region. On the other hand, the third embodiment is different from the first and second embodiments described above in that a predetermined texture is extracted from the image of the camera 10 as a predetermined feature. The predetermined texture is, for example, a texture obtained from an image when a traffic light, a vehicle detector, a traffic information information board, a road sign, a guardrail, a street light, a curved mirror, or the like is imaged. Hereinafter, the differences from the first and second embodiments will be mainly described, and the description of the functions and the like common to the first and second embodiments will be omitted.

17 to 19 are diagrams showing an example of comparison between the texture and the power spectrum image. (A) of each figure represents an image area cut out by a certain search window, and (b) shows a power spectrum image obtained by converting an image area cut out by a search window into a spectrum by a two-dimensional Fourier transform. .. As illustrated in FIGS. 17 and 18, when the pixel value changes strongly in a certain direction, the power spectrum becomes strong in the frequency domain in the direction in which the pixel value changes, and a linear distribution tends to appear. On the other hand, as illustrated in FIG. 19, when the pixel value changes to some extent in any direction, the power spectrum becomes strong in both the horizontal and vertical frequency regions. As described above, since a structure such as a traffic light is an object having a certain directionality such as a pole or an electric wire, a power spectrum image of an image region cut out by a search window is illustrated in FIGS. 17 and 18. In the case of such a power spectrum image, it can be determined that the image area cut out by the search window includes a directional object.

Therefore, the estimation unit 134 sets a threshold that allows the _{power spectrum value to be equal to or higher than the spectral threshold STH} in one or both of the high frequency region of the vertical spatial frequency and the high frequency region of the horizontal spatial frequency. Then, when the power spectrum value is equal to _{or higher than the spectrum threshold STH in} the frequency region below the set threshold, it is determined that the image region that is the source of the power spectrum image is a predetermined texture.

20 and 21 are diagrams for explaining a method of setting a threshold value that allows the power spectrum value to be equal to _{or higher than the spectrum threshold value STH.} For example, as shown in FIG. 20, the estimation unit 134 allows the spectrum distribution to spread in the horizontal frequency domain as the frequency domain increases in the vertical frequency domain, so that the spectrum threshold value S is set _{for the region RPS.} set the _TH, the other region excluding the region R _PS sets the spectral threshold S _TH greater threshold than (e.g. the order of magnitude of the threshold causing regarded as infinite). Further, as shown in FIG. 21, the estimation unit 134 allows the spectrum distribution to spread to the vertical frequency region as the frequency region increases in the horizontal frequency region, so that the spectrum threshold value S is set _{for the region RPS.} set _TH, sets a greater threshold than the spectral threshold S _TH for other regions except for the region R _PS. As a result, for the power spectrum image as illustrated in FIGS. 17 and 18, the power spectrum value becomes equal to _{or higher than the spectrum threshold value STH,} and it is determined that the image region cut out by the detection window is a predetermined texture. When the image of the camera 10 has a predetermined texture, the estimation unit 134 estimates that the traffic control point exists in front of the own vehicle M.

According to the third embodiment described above, in order to estimate the existence of a traffic control point based on the power spectrum image obtained by two-dimensional Fourier transforming the image of the camera 10, the first or second embodiment Similarly, the driving environment can be recognized at an earlier timing.

[Hardware configuration]
The automatic operation control device 100 of the above-described embodiment is realized by, for example, a hardware configuration as shown in FIG. FIG. 22 is a diagram showing an example of the hardware configuration of the automatic operation control device 100 of the embodiment.

The automatic operation control device 100 of the embodiment includes a communication controller 100-1, a CPU 100-2, a RAM (Random Access Memory) 100-3, a ROM (Read Only Memory) 100-4, a flash memory, an HDD (Hard Disc Drive), or the like. The secondary storage device 100-5 and the drive device 100-6 are connected to each other by an internal bus or a dedicated communication line. A portable storage medium such as an optical disk is mounted on the drive device 100-6. The program 100-5a stored in the secondary storage device 100-5 is expanded into the RAM 100-3 by a DMA controller (not shown) or the like, and executed by the CPU 100-2 to control the first control unit 120 and the second control unit 120. Part 160 is realized. Further, the program referred to by the CPU 100-2 may be stored in the portable storage medium mounted on the drive device 100-6, or may be downloaded from another device via the network NW.

The above embodiment can be expressed as follows.
A camera that captures the surroundings of the vehicle and
Storage to store information and
A processor that executes a program stored in the storage is provided.
The processor executes the program.
Of the areas of the image captured by the camera, it is determined whether or not the sky area corresponding to the space above a predetermined altitude is in a congested state, and when it is determined that the sky area is in a congested state, the vehicle It was configured to presume that there was a traffic control point ahead,
Vehicle control device.

Although the embodiments for carrying out the present invention have been described above using the embodiments, the present invention is not limited to these embodiments, and various modifications and substitutions are made without departing from the gist of the present invention. Can be added.

1 ... Vehicle system, 10 ... Camera, 12 ... Radar device, 14 ... Finder, 16 ... Object recognition device, 20 ... Communication device, 30 ... HMI, 40 ... Vehicle sensor, 50 ... Navigation device, 60 ... MPU, 80 ... Driving Operator, 100 ... automatic operation control device, 120 ... first control unit, 130 ... recognition unit, 132 ... feature extraction unit, 134 ... estimation unit, 140 ... action plan generation unit, 160 ... second control unit, 162 ... acquisition 164 ... Speed control unit, 166 ... Steering control unit, 200 ... Travel driving force output device, 210 ... Brake device, 220 ... Steering device, M ... Own vehicle, m ... Other vehicle

Claims (11)

An imaging unit that captures the surroundings of the vehicle and
When it is determined whether or not the sky region corresponding to the space above a predetermined altitude is in a congested state among the regions of the image captured by the imaging unit, and when it is determined that the sky region is in a congested state, the vehicle In front of, an estimation unit that estimates that there is a traffic control point where the traffic of vehicles or pedestrians is controlled is provided.
The traffic control points include intersections, confluences, branch points, or railroad crossings.
Car both control device. When the estimation unit determines that the sky area is in a congested state, the estimation unit estimates that the traffic control point exists at a position on the road corresponding to the position of the sky area in the congested state. The vehicle control device according to 1. The estimation unit determines whether or not the sky region is in a congested state based on the spatial frequency obtained from the change in the pixel value of the sky region by the two-dimensional Fourier transform.
The vehicle control device according to claim 1 or 2. The estimation unit determines that the sky region is in a congested state when the intensity of a component of the spatial frequency equal to or higher than a predetermined frequency is equal to or higher than a threshold value.
The vehicle control device according to claim 3. The estimation unit further
Of the areas of the image captured by the imaging unit, it is determined whether or not the road surface area corresponding to the road space is in a congested state.
When it is determined that the sky area is in a congested state and the road surface area is in a congested state, the certainty is higher than when it is determined that one or both of the sky area and the road surface area is not in a congested state. Estimate the existence of the traffic control point,
The vehicle control device according to any one of claims 1 to 4. Further provided with a recognition unit that recognizes a road marking indicating the existence of the traffic control point from the road on which the vehicle is located.
The estimation unit is higher when the recognition unit recognizes the road marking when it is determined that the sky area is in a congested state, as compared with the case where the recognition unit does not recognize the road marking. Estimate the existence of the traffic control point with certainty,
The vehicle control device according to any one of claims 1 to 5. Further provided with a recognition unit that recognizes a road marking indicating the existence of the traffic control point from the road on which the vehicle is located.
When the road marking is recognized by the recognition unit, the estimation unit starts a process of determining whether or not the sky area is in a congested state.
The vehicle control device according to any one of claims 1 to 6. A recognition unit that recognizes a structure indicating the existence of the traffic control point is further provided.
When it is estimated by the estimation unit that the traffic control point exists, the recognition unit starts the recognition process of the structure.
The vehicle control device according to any one of claims 1 to 5. A driving control unit that controls at least acceleration / deceleration of the vehicle is further provided.
The driving control unit decelerates the vehicle when the estimation unit estimates that the traffic control point exists.
The vehicle control device according to any one of claims 1 to 8. The image pickup unit captures the surroundings of the vehicle and
The estimation unit determines whether or not the sky region corresponding to the space above a predetermined altitude is in a congested state among the regions of the image captured by the imaging unit, and determines that the sky region is in a congested state. In this case, it is estimated that there is a traffic control point in front of the vehicle in which the traffic of the vehicle or pedestrian is controlled.
The traffic control points include intersections, confluences, branch points, or railroad crossings.
Vehicle control method. A computer mounted on a vehicle equipped with an imaging unit that captures the surroundings of the vehicle
Of the areas of the image captured by the imaging unit, it is determined whether or not the sky area corresponding to the space above the predetermined altitude is in a congested state.
When it is determined that the sky area is in a congested state, it is estimated that there is a traffic control point in front of the vehicle in which the traffic of the vehicle or pedestrian is controlled .
The traffic control points include intersections, confluences, branch points, or railroad crossings.
program.