JP2021064056A - Zebra zone recognition device, vehicle control device, zebra zone recognition method, and program - Google Patents

Abstract

To provide a zebra zone recognition device, a vehicle control device, a zebra zone recognition method, and programs capable of appropriately recognizing zebra zones with a simple processing.SOLUTION: A zebra zone recognition device comprises a generating part, a setting part, and a specification part, where the generating part is mounted on a vehicle and generates a plane image showing the road color and the luminance distribution in the traveling direction of the vehicle based on the information acquired from the recognition device to recognize the surrounding situation of the vehicle, the setting part sets multiple first detecting lines parallel to each other on the generated plane image by the generating part and multiple second detecting lines orthogonal to each of the first detecting lines, and the specification part determines whether or not each of the plurality of the first detecting lines and of the plurality of the second detecting lines intersects with the outline of the road sign in the plane image and identifies the road sign as the zebra zone based on the determined result.SELECTED DRAWING: Figure 2

Description

The present invention relates to a zebra zone recognition device, a vehicle control device, a zebra zone recognition method, and a program.

Conventionally, in order to recognize the degree of approach to the vehicle inflow port, a recognition device technology that identifies line marks such as continuous lines, entrance / exit dashed lines, and headrace zones (striped patterns), and sets the alert level according to the identification result. Is known (see, for example, Patent Document 1).

JP-A-7-117594

However, in the conventional technique, the study on recognizing the zebra zone in a suitable manner has been insufficient.

The present invention has been made in consideration of such circumstances, and provides a zebra zone recognition device, a vehicle control device, a zebra zone recognition method, and a program capable of preferably recognizing a zebra zone by a simple process. One of the purposes is to do.

The zebra zone recognition device, the vehicle control device, the zebra zone recognition method, and the program according to the present invention have adopted the following configurations.
(1): The zebra zone recognition device according to one aspect of the present invention is mounted on a vehicle and is a road surface in the traveling direction of the vehicle based on information acquired from a recognition device for recognizing the surrounding conditions of the vehicle. A generator that generates a planar image that represents the distribution of color or brightness,
A setting unit for setting a plurality of first detection lines parallel to each other and a plurality of second detection lines orthogonal to each of the first detection lines in the plane image generated by the generation unit.
It is determined whether or not each of the plurality of first detection lines and the plurality of second detection lines intersects the contour of the road marking in the plane image, and the road marking is specified as a zebra zone based on the result of the determination. It is a zebra zone recognition device including a specific part.

(2): In the aspect of (1) above, the specific unit sets the plurality of first detection lines or the plurality of second detection lines to be parallel to the longitudinal direction of the lane in which the vehicle is located. Is what you do.

(3): In the embodiment (1) or (2), the setting unit sets the first detection line and the second detection line in a monitoring area of a predetermined size, and the specific unit monitors the monitoring. When all of the first detection lines in the area intersect the contour of the road marking and all of the second detection lines in the monitoring area intersect the contour of the road marking, the monitoring area is a part of the zebra zone. If not (at least part of the first detection line in the monitoring area does not intersect the contour of the road marking, or at least part of the second detection line in the monitoring area is road marking. If it does not intersect the contour), the detected road marking line is not specified as the zebra zone.

(4): In the aspect of (3) above, the monitoring area includes a predetermined number of the first detection lines or the second detection lines adjacent to the first detection line or the second detection line of interest. This is the area that contains it.

(5): The zebra zone recognition device according to another aspect of the present invention is mounted on a vehicle and has a vanishing line from an arbitrary point toward the vanishing point in an captured image acquired from a camera that captures the surrounding situation of the vehicle. A setting unit for setting a plurality of third detection lines along the line and a plurality of fourth detection lines along the lateral direction of the captured image, and the plurality of third detection lines and the plurality of fourth detection lines are described above. It is provided with a specific portion that determines whether or not it intersects with the contour of the road marking in the captured image and identifies the road marking as a zebra zone based on the result of the determination.

(6): In the aspect of (5) above, the setting unit sets the third detection line and the fourth detection line in the monitoring area, and the specific unit sets the third detection line in the monitoring area. When all of the lines intersect the contour of the road marking in the captured image and all of the fourth detection lines in the monitoring area intersect the contour of the road marking in the captured image, the monitoring area is said to be said. It is determined that the monitoring area is not a part of the zebra zone, and if not, it is determined that the monitoring area is not a part of the zebra zone.

(7): In the embodiment (5) or (6), the monitoring area is an area including a predetermined number of the third detection line or the fourth detection line.

(8): In the above aspects (1) to (7), the specific unit periodically repeats the process of specifying the zebra zone, and at least the state of the vehicle direction indicator and the yaw rate. Based on the detection result of the sensor and a part or all of the detection result of the gyro sensor, it is determined whether or not the vehicle is turning, and when it is determined that the vehicle is turning, the zebra zone is determined. Is not specified, and the specific result before determining that the vehicle is turning is maintained.

(9): The vehicle control device according to one aspect of the present invention is the vehicle based on the zebra zone recognition device according to any one of (1) to (8) above and the processing result by the zebra zone recognition device. It is provided with an operation control unit for controlling the steering and speed of the vehicle.

(10): In the aspect of (9) above, an object recognition unit that recognizes an object existing in the vicinity of the vehicle and estimates the risk due to the object is further provided, and the object recognition unit is specified by the specific unit. When the zebra zone is on the side of the vehicle, it is determined whether or not the second lane, which is located on the side of the zebra zone and is different from the first lane in which the vehicle is located, can be further recognized. When it is determined that two lanes can be recognized and the traveling direction of the second lane intersects the traveling direction of the first lane, the second lane is recognized as an adjacent lane of the first lane, and the second lane is recognized. It estimates the risk of an object in.

(11): In the zebra zone recognition method according to another aspect of the present invention, the progress of the vehicle is based on information acquired from a recognition device mounted on the vehicle and for recognizing the surrounding situation of the vehicle. A plane image representing the distribution of the color or brightness of the road in the direction is generated, and the generated plane image has a plurality of first detection lines parallel to each other and a plurality of second detection lines orthogonal to each of the first detection lines. A detection line is set, and it is determined whether or not each of the plurality of first detection lines and the plurality of second detection lines intersects the contour of the road marking in the plane image, and the road is determined based on the result of the determination. It identifies the marking as a zebra zone.

(12): The program according to another aspect of the present invention is mounted on a computer and is a road in the traveling direction of the vehicle based on information acquired from a recognition device for recognizing the surrounding situation of the vehicle. A plane image representing the distribution of color or brightness of the above is generated, and the generated plane image includes a plurality of first detection lines parallel to each other and a plurality of second detection lines orthogonal to each of the first detection lines. Is set, and it is determined whether or not each of the plurality of first detection lines and the plurality of second detection lines intersects the contour of the road marking in the plane image, and the road marking is zebra based on the determination result. It is intended to be identified as a zone.

(13): In the zebra zone recognition method according to another aspect of the present invention, a computer is mounted on a vehicle and the captured image acquired from a camera that captures the surrounding situation of the vehicle is directed from an arbitrary point to a vanishing point. A plurality of third detection lines along the vanishing line and a plurality of fourth detection lines along the lateral direction of the image are set, and each of the plurality of third detection lines and the plurality of fourth detection lines captures the image. It is determined whether or not it intersects with the contour of the road marking in the image, and the road marking is specified as the zebra zone based on the result of the determination.

(14): The program according to another aspect of the present invention is a vanishing line from an arbitrary point toward a vanishing point in an captured image acquired from a camera mounted on a vehicle and capturing the surrounding conditions of the vehicle on a computer. A plurality of third detection lines along the line and a plurality of fourth detection lines along the lateral direction of the image are set, and each of the plurality of third detection lines and the plurality of fourth detection lines is a road marking in the image. It is made to judge whether or not it intersects with the contour of, and the road marking is specified as a zebra zone based on the result of the judgment.

According to the above aspects (1) to (14), the zebra zone can be suitably recognized by a simple process.

Further, according to the above aspect (9) or (10), the vehicle can be suitably driven based on the specific result of the zebra zone.

It is a block diagram of the vehicle system using the zebra zone recognition device of 1st Embodiment. It is a functional block diagram of the vehicle control device of 1st Embodiment. It is a figure which shows an example of the image taken by the camera. It is a figure which shows an example of the plane image generated by the plane image generation part based on the image shown in FIG. It is a figure for demonstrating the monitoring area set by the detection line setting part. It is a figure for demonstrating the 1st detection line and the 2nd detection line set by the detection line setting part. It is a figure which illustrated the relationship between a pedestrian crossing and a detection line. It is a figure which shows an example of a specific scene. It is a flowchart which shows an example of the processing flow by a zebra zone recognition part. It is a functional block diagram of the vehicle control device which concerns on 2nd Embodiment. It is a figure for demonstrating the process of the detection line setting part of 2nd Embodiment. It is a figure which shows an example of the hardware composition of the vehicle control device of an embodiment.

Hereinafter, the zebra zone recognition device, the vehicle control device, the zebra zone recognition method, and the embodiment of the program of the present invention will be described with reference to the drawings.

<First Embodiment>
[overall structure]
FIG. 1 is a configuration diagram of a vehicle system 1 using the zebra zone recognition device of the first embodiment. The vehicle on which the vehicle system 1 is mounted is, for example, a vehicle such as a two-wheeled vehicle, a three-wheeled vehicle, or a four-wheeled vehicle, and the drive source thereof is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination thereof. The electric motor operates by using the power generated by the generator connected to the internal combustion engine or the discharge power of the secondary battery or the fuel cell.

The vehicle system 1 includes, for example, a camera 10, a radar device 12, a LIDAR (Light Detection and Ranging) 14, an object recognition device 16, a driving controller 80, a vehicle control device 100, and a traveling driving force output device 200. And a braking device 210 and a steering device 220. These devices and devices are connected to each other by a multiplex communication line such as a CAN (Controller Area Network) communication line, a serial communication line, a wireless communication network, or the like. The configuration shown in FIG. 1 is merely an example, and a part of the configuration may be omitted or another configuration may be added.

The camera 10 is, for example, a digital camera using a solid-state image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 10 is attached to an arbitrary position of the vehicle on which the vehicle system 1 is mounted (hereinafter, the own vehicle M). When photographing the front, the camera 10 is attached to the upper part of the front windshield, the back surface of the rearview mirror, and the like. The camera 10 periodically and repeatedly images the periphery of the own vehicle M, for example. The camera 10 may be a stereo camera.

The radar device 12 radiates radio waves such as millimeter waves around the own vehicle M, and detects radio waves (reflected waves) reflected by the object to detect at least the position (distance and direction) of the object. The radar device 12 is attached to an arbitrary position of the own vehicle M. The radar device 12 may detect the position and velocity of the object by the FM-CW (Frequency Modulated Continuous Wave) method.

The LIDAR 14 irradiates the periphery of the own vehicle M with light (or an electromagnetic wave having a wavelength close to that of light) and measures the scattered light. The LIDAR 14 detects the distance to the target based on the time from light emission to light reception. The emitted light is, for example, a pulsed laser beam. The LIDAR 14 is attached to an arbitrary position on the own vehicle M.

The object recognition device 16 performs sensor fusion processing on the detection results of a part or all of the camera 10, the radar device 12, and the LIDAR 14, and recognizes the position, type, speed, and the like of the object. The object recognition device 16 outputs the recognition result to the vehicle control device 100. The object recognition device 16 may output the detection results of the camera 10, the radar device 12, and the LIDAR 14 to the vehicle control device 100 as they are. The object recognition device 16 may be omitted from the vehicle system 1.

The communication device 20 communicates with another vehicle existing in the vicinity of the own vehicle M by using, for example, a cellular network, a Wi-Fi network, Bluetooth (registered trademark), DSRC (Dedicated Short Range Communication), or wirelessly. Communicates with various server devices via the base station.

The HMI 30 presents various information to the occupants of the own vehicle M and accepts input operations by the occupants. The HMI 30 includes various display devices, speakers, buzzers, touch panels, switches, keys and the like.

The vehicle sensor 40 includes a vehicle speed sensor that detects the speed of the own vehicle M, an acceleration sensor that detects acceleration, a yaw rate sensor that detects an angular velocity around the vertical axis, a gyro sensor that detects the direction of the own vehicle M, and an operation of a direction indicating device. Includes a blinker switch that detects the state.

The navigation device 50 includes, for example, a GNSS receiver 51, a navigation HMI 52, and a route determination unit 53. The navigation device 50 holds the first map information 54 in a storage device such as an HDD or a flash memory. The GNSS receiver 51 identifies the position of the own vehicle M based on the signal received from the GNSS satellite. The position of the own vehicle M may be specified or complemented by an INS (Inertial Navigation System) using the output of the vehicle sensor 40. The navigation HMI 52 includes a display device, a speaker, a touch panel, keys, and the like. The navigation HMI 52 may be partially or wholly shared with the above-mentioned HMI 30. The route determination unit 53, for example, has a route from the position of the own vehicle M (or an arbitrary position input) specified by the GNSS receiver 51 to the destination input by the occupant using the navigation HMI 52 (hereinafter, hereafter). The route on the map) is determined with reference to the first map information 54. The first map information 54 is, for example, information in which a road shape is expressed by a link indicating a road and a node connected by the link. The first map information 54 may include road curvature, POI (Point Of Interest) information, and the like. The route on the map is output to MPU60. The navigation device 50 may provide route guidance using the navigation HMI 52 based on the route on the map. The navigation device 50 may be realized by, for example, the function of a terminal device such as a smartphone or a tablet terminal owned by an occupant. The navigation device 50 may transmit the current position and the destination to the navigation server via the communication device 20 and acquire a route equivalent to the route on the map from the navigation server.

The MPU 60 includes, for example, a recommended lane determination unit 61, and holds the second map information 62 in a storage device such as an HDD or a flash memory. The recommended lane determination unit 61 divides the route on the map provided by the navigation device 50 into a plurality of blocks (for example, divides the route into a plurality of blocks (for example, every 100 [m] with respect to the vehicle traveling direction), and refers to the second map information 62. Determine the recommended lane for each block. The recommended lane determination unit 61 determines which lane to drive from the left. When a branch point exists on the route on the map, the recommended lane determination unit 61 determines the recommended lane so that the own vehicle M can travel on a reasonable route to proceed to the branch destination.

The second map information 62 is more accurate map information than the first map information 54. The second map information 62 includes, for example, information on the center of the lane, information on the boundary of the lane, and the like. Further, the second map information 62 may include road information, traffic regulation information, address information (address / zip code), facility information, telephone number information, and the like. The second map information 62 may be updated at any time by the communication device 20 communicating with another device.

The driving controller 80 includes, for example, an accelerator pedal, a brake pedal, a shift lever, a steering wheel, a deformed steering wheel, a joystick, and other controls. A sensor for detecting the amount of operation or the presence or absence of operation is attached to the driving operator 80, and the detection result is the vehicle control device 100 or the traveling driving force output device 200, the braking device 210, and the steering device 220. It is output to some or all of them.

The vehicle control device 100 includes, for example, a first control unit 120 and a second control unit 160. The first control unit 120 and the second control unit 160 are realized by, for example, a hardware processor such as a CPU executing a program (software), respectively. Further, some or all of these components may be realized by hardware such as LSI, ASIC, FPGA, GPU (including circuit section; circuitry), or realized by collaboration between software and hardware. May be done. The program may be stored in advance in a storage device (a storage device including a non-transient storage medium) such as an HDD or a flash memory of the vehicle control device 100, or a removable storage such as a DVD or a CD-ROM. It is stored in a medium, and the storage medium (non-transient storage medium) may be installed in the HDD or flash memory of the vehicle control device 100 by being attached to the drive device.

FIG. 2 is a functional configuration diagram of the vehicle control device 100. The vehicle control device 100 includes, for example, a first control unit 120 and a second control unit 160. The first control unit 120 includes, for example, a recognition unit 130 and an action plan generation unit 140. The first control unit 120, for example, realizes a function by AI (Artificial Intelligence) and a function by a model given in advance in parallel. For example, the function of "recognizing an intersection" is executed in parallel with the recognition of an intersection by deep learning or the like and the recognition based on predetermined conditions (there are signals that can be pattern matched, road markings, etc.), and both are executed. It may be realized by scoring and comprehensively evaluating. This ensures the reliability of autonomous driving. The recognition unit 130 is an example of a “zebra zone recognition device”. The combination of the action plan generation unit 140 and the second control unit 160 is an example of the “operation control unit”.

The recognition unit 130 recognizes the periphery of the own vehicle M. The recognition unit 130 includes, for example, a vehicle position recognition unit 131, an object recognition unit 132, and a zebra zone recognition unit 133. The zebra zone recognition unit 133 includes, for example, a plan view generation unit 133A, a detection line setting unit 133B, and a zebra zone identification unit 133C. An example of the zebra zone recognition device is one provided with at least the zebra zone recognition unit 133.

The own vehicle position recognition unit 131 recognizes the lane (traveling lane) in which the own vehicle M is traveling based on the information input from the camera 10, the radar device 12, and the LIDAR 14 via the object recognition device 16. For example, the own vehicle position recognition unit 131 is imaged by the pattern of the road marking line (for example, the arrangement of solid lines and broken lines, the thickness of lines, the spacing between lines, etc.) obtained from the second map information 62, and the camera 10. The traveling lane is recognized by comparing with the pattern of the road lane marking around the own vehicle M recognized from the image. The own vehicle position recognition unit 131 recognizes the traveling lane by recognizing not only the road lane marking but also the lane marking (road boundary) including the road lane marking, the shoulder, the curb, the median strip, the guardrail, and the like. You may. In this recognition, the position of the own vehicle M acquired from the navigation device 50 and the processing result by the INS may be added.

When recognizing the traveling lane, the own vehicle position recognition unit 131 recognizes the position and posture of the own vehicle M with respect to the traveling lane. The own vehicle position recognition unit 131 determines, for example, the deviation of the reference point of the own vehicle M from the center of the lane and the angle formed by the own vehicle M with respect to the line connecting the center of the lane in the traveling direction of the own vehicle M with respect to the traveling lane. It may be recognized as the relative position and orientation of. Instead of this, the own vehicle position recognition unit 131 sets the position of the reference point of the own vehicle M with respect to any side end portion (road lane marking or road boundary) of the traveling lane, etc., relative to the traveling lane. It may be recognized as a position.

The object recognition unit 132 is an object (including a vehicle in front and an oncoming vehicle) around the own vehicle M based on the information input from the camera 10, the radar device 12, and the LIDAR 14 via the object recognition device 16. Recognize the position, speed, acceleration, and other states. The position of the object is recognized as, for example, a position on absolute coordinates with the representative point (center of gravity of the vehicle, the center of the drive shaft, etc.) of the own vehicle M as the origin, and is used for control. The position of the object may be represented by a representative point such as the center of gravity or a corner of the object, or may be represented by a represented area. The "state" of an object may include acceleration or jerk of the object, or "behavioral state" (eg, whether or not the vehicle is changing lanes or is about to change lanes).

In addition, the object recognition unit 132 estimates the risk (risk area) due to the existence of the object in the vicinity of the own vehicle M. The risk is, for example, an index value in which the value decreases as the center point of the object is set as the maximum value and spreads to the periphery, and becomes zero when the object is sufficiently separated. An example of applying the risk in this embodiment will be described later.

The processing of the zebra zone recognition unit 133 will be described later.

In principle, the action plan generation unit 140 travels in the recommended lane determined by the recommended lane determination unit 61, and further, it is possible to execute automatic driving corresponding to the surrounding conditions of the own vehicle M (for example, possible). As long as the vehicle M travels on a low-risk route estimated by the object recognition unit 132 within the recommended lane), the target track on which the own vehicle M will travel in the future is generated. The target trajectory includes, for example, a velocity element. For example, the target track is expressed as a sequence of points (track points) to be reached by the own vehicle M. The track point is a point to be reached by the own vehicle M for each predetermined mileage (for example, about several [m]) along the road, and separately, for a predetermined sampling time (for example, about 0 comma [sec]). ) Target velocity and target acceleration are generated as part of the target trajectory.

The second control unit 160 sets the traveling driving force output device 200, the brake device 210, and the steering device 220 so that the own vehicle M passes the target trajectory generated by the action plan generation unit 140 at the scheduled time. Control.

Returning to FIG. 1, the second control unit 160 includes, for example, an acquisition unit 162, a speed control unit 164, and a steering control unit 166. The acquisition unit 162 acquires the information of the target trajectory (orbit point) generated by the action plan generation unit 140 and stores it in a memory (not shown). The speed control unit 164 controls the traveling driving force output device 200 or the braking device 210 based on the speed element associated with the target trajectory stored in the memory. The steering control unit 166 controls the steering device 220 according to the degree of bending of the target trajectory stored in the memory. The processing of the speed control unit 164 and the steering control unit 166 is realized by, for example, a combination of feedforward control and feedback control. As an example, the steering control unit 166 executes a combination of feedforward control according to the curvature of the road in front of the own vehicle M and feedback control based on the deviation from the target trajectory.

The traveling driving force output device 200 outputs a traveling driving force (torque) for traveling the vehicle to the drive wheels. The traveling driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, a transmission, and the like, and an ECU that controls them. The ECU controls the above configuration according to the information input from the second control unit 160 or the information input from the operation operator 80.

The brake device 210 includes, for example, a brake caliper, a cylinder that transmits flood pressure to the brake caliper, an electric motor that generates flood pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor according to the information input from the second control unit 160 or the information input from the operation operator 80 so that the brake torque corresponding to the braking operation is output to each wheel. The brake device 210 may include, as a backup, a mechanism for transmitting the oil pressure generated by the operation of the brake pedal included in the operation operator 80 to the cylinder via the master cylinder. The brake device 210 is not limited to the configuration described above, and is an electronically controlled hydraulic brake device that controls an actuator according to information input from the second control unit 160 to transmit the oil pressure of the master cylinder to the cylinder. May be good.

The steering device 220 includes, for example, a steering ECU and an electric motor. The electric motor, for example, applies a force to the rack and pinion mechanism to change the direction of the steering wheel. The steering ECU drives the electric motor according to the information input from the second control unit 160 or the information input from the driving operator 80 to change the direction of the steering wheel.

[Zebra zone recognition]
Hereinafter, the processing of the zebra zone recognition unit 133 will be described. The zebra zone recognition unit 133 recognizes the zebra zone existing around the own vehicle M. The zebra zone (guide zone) is one of the road markings drawn on the road, and is, for example, a striped pattern diagonal to the road and drawn in white. The zebra zone is an area where passage within the area is permitted by law, but stopping is not permitted.

The plane image generation unit 133A of the zebra zone recognition unit 133 uses the color or brightness of the road in the traveling direction of the own vehicle M based on the image captured by the camera 10 or the information of the reflection intensity of the target detected by the LIDAR 14. Generates a planar image showing the distribution of. The plane image may be a binary image (black and white image) or a color image. The plane image is an image when the viewpoint is virtually placed in the sky. In the following description, it is assumed that the plane image generation unit 133A generates a plane image from the image captured by the camera 10.

FIG. 3 is a diagram showing an example of an image captured by the camera 10, and FIG. 4 is a diagram showing an example of a planar image IM1 generated by the planar image generation unit 133A based on the image shown in FIG. .. The plane image generation unit 133A generates a plane image by applying a conversion table or a conversion function for converting an image plane to a real plane image to the image captured by the camera 10.

The plane image generation unit 133A periodically performs a process of generating a plane image, for example. Further, the plane image generation unit 133A recognizes that the own vehicle M is approaching the intersection, or recognizes that the own vehicle M is approaching the confluence, and the like. A plane image may be generated only when the condition is satisfied. When a plane image is generated based on the reflection intensity of the target detected by LIDAR 14, the plane image generation unit 133A uses the information of the reflection point detected by LIDAR (for example, {vertical irradiation angle, left-right direction). The plane image is generated by removing the information about the height from the irradiation angle, distance, and reflection intensity}.

The detection line setting unit 133B sets a plurality of (predetermined number of) first detection lines and a plurality of (predetermined number of) second detection lines in the monitoring area in the plane image IM1 generated by the plane image generation unit 133A. Set each. The monitoring area is, for example, a rectangle (for example,) having a side length slightly larger than the width of one line (hereinafter referred to as “line mark”) constituting a road marking line, a pedestrian crossing, a zebra zone, or the like. It is set to a rectangle of several tens [cm] x several tens [cm]). A line mark is a white or yellow line drawn on a road in the form of a solid line or a broken line.

For example, in the plane image IM1, the detection line setting unit 133B roughly searches the region around the own vehicle M where the line mark is likely to exist, and sets the region where the edge points exist at a certain density as the initial monitoring region. Then, the detection line setting unit 133B comprehensively sets the monitoring area in the initial monitoring area by a method such as raster scan. FIG. 5 is a diagram for explaining a monitoring area set by the detection line setting unit 133B. In the figure, iMA is the initial monitoring area and MA is the monitoring area. Further, CR1 is a pedestrian crossing, and ZZ1 and ZZ2 are zebra zones.

Then, the detection line setting unit 133B sets a plurality of first detection lines and a plurality of second detection lines in each monitoring area MA. FIG. 6 is a diagram for explaining a first detection line and a second detection line set by the detection line setting unit 133B. In the figure, LM is a line mark, DL1 (1), ... DL1 (n) are n first detection lines, and DL2 (1), ... DL2 (m) are m second detection lines. is there. The detection line setting unit 133B sets the first detection line DL1 in a direction parallel to the longitudinal direction of the lane in which the own vehicle M is traveling, for example. The longitudinal direction of the lane is obtained from, for example, the extending direction of the road lane marking recognized by the own vehicle position recognition unit 131. The detection line setting unit 133B sets the second detection line DL2 in a direction orthogonal to the first detection line DL1, for example. The interval between the first detection line DL1 and the second detection line DL2 is such that a predetermined number (for example, about 3 to 5) of the first detection line DL1 or the second detection line DL2 fits in the monitoring area MA, for example. Is set to. It is desirable that this interval is slightly narrower than the interval of line marks on pedestrian crossings and the like.

The zebra zone identification unit 133C determines whether or not each of the plurality of first detection lines DL1 and the plurality of second detection lines DL2 intersects the contour of the road marking (hereinafter referred to as the contour of the line mark) in the plane image IM1. Judgment is made, and the road marking is identified as the zebra zone based on the result of the judgment. The zebra zone identification unit 133C is used when the feature quantities such as brightness and color element (RGB) of each pixel on the first detection line DL1 or the second detection line DL2 change more than the reference value between the pixels and the adjacent pixels. , It is determined that "it intersects with the outline of the line mark". More specifically, in the zebra zone identification unit 133C, all of the first detection line DL1 in the monitoring area MA intersects the contour of the line mark, and all of the second detection line DL2 in the monitoring area MA are line marks. If it intersects with the contour of, it is determined that the monitoring area MA is a part of the zebra zone, and if not, it is determined that the monitoring area MA is not a part of the zebra zone.

Generally, the line mark of the zebra zone is drawn diagonally with respect to the extending direction of the lane. Therefore, each of the first detection line DL1 set in the direction parallel to the longitudinal direction of the lane and the second detection line DL2 set in the direction orthogonal to the first detection line DL1 is the line mark of the zebra zone. It is expected to intersect with either contour. Also in FIG. 6, all of the first detection line DL1 in the monitoring area MA intersects with the contour of the line mark LM, and all of the second detection line DL2 in the monitoring area MA intersects with the contour LM of the line mark. There is. In addition, there are pedestrian crossings where line marks are drawn diagonally with respect to the road, such as scrambled intersections, but the zebra zone identification section 133C performs processing such as excluding this in advance with reference to map information. You may go.

On the contrary, line marks such as pedestrian crossings and stop lines are generally drawn so as to be parallel or orthogonal to the extending direction of the lane. FIG. 7 is a diagram illustrating the relationship between the pedestrian crossing and the detection line. In the example of FIG. 7, the first detection line DL1 (3) fits between the line mark LM and does not intersect the contour of the line mark LM. Therefore, the zebra zone identification unit 133C determines that the monitoring area MA is not a part of the zebra zone.

[Zebra zone recognition during turning]
The zebra zone recognition unit 133 repeatedly executes the above-mentioned process. During that time, the zebra zone recognition unit 133 determines whether or not the own vehicle M is turning, and when it is determined that the vehicle M is turning, it does not recognize the zebra zone and determines that the vehicle is turning. The recognition result before (for example, immediately before) the timing at which the determination is reached is maintained. When the own vehicle M is turning, it may be difficult to measure the angle of the vehicle body of the own vehicle M with respect to the lane in real time and synchronize the timing in relation to the time required for the image analysis processing. Therefore, when the own vehicle M is turning, the first detection line DL1 that should have been "set in a direction parallel to the longitudinal direction of the lane" may not actually be parallel to the longitudinal direction of the lane. Because there is sex. Therefore, in the embodiment, when it is determined that the own vehicle M is turning, the recognition result of the zebra zone is not updated, and the recognition result before the previous time is maintained. This makes it possible to reduce the possibility of false positives.

The zebra zone recognition unit 133 refers to the detection result of the vehicle sensor 40, and based on at least a part or all of the state of the direction indicator of the own vehicle M, the detection result of the yaw rate sensor, and the detection result of the gyro sensor. It is determined whether or not the own vehicle M is turning. For example, the zebra zone recognition unit 133 indicates that the direction indicator is operating in either direction, the detection result of the yaw rate sensor indicates a value equal to or higher than the threshold value, the detection result of the gyro sensor and the extending direction of the lane. When two or more of the angles formed by the vehicle changing at a speed equal to or greater than the threshold value are satisfied, it is determined that the own vehicle M is turning.

When the zebra zone is specified by the zebra zone identification unit 133C, the action plan generation unit 140 generates a target trajectory so as not to stop in the zebra zone, and controls the steering and speed of the own vehicle M.

[Application example]
Hereinafter, the processing of the action plan generation unit 140 in the specific scene will be described. As described above, the object recognition unit 132 estimates the risk (risk area) due to the existence of the object around the own vehicle M. The objects for which the object recognition unit 132 estimates the risk of an object are mainly the lane in which the own vehicle M is traveling (hereinafter referred to as the own lane), the adjacent lane adjacent to the own lane, the oncoming lane facing the own lane, and the own lane. It is an object that exists in each of the crossing lanes that intersect the lane. By lowering the priority of recognition or not recognizing an object existing in the other region by the object recognition unit 132, it is possible to reduce the processing load and suppress erroneous detection.

By the way, according to the recognition rules, the lane recognized as an adjacent lane is "a lane adjacent to the own lane across a road marking line", and regarding a lane having a zebra zone between the own lane. Is not recognized as an adjacent lane. However, in the merging scene, it is assumed that there is a merging lane with a zebra zone in between the lane and the merging lane, and the merging lane will eventually merge into the own lane. The object recognition unit 132 is located in front of the zebra zone ZZ when the zebra zone ZZ specified by the zebra zone identification unit 133C is on the side of the own vehicle M, and is in the own lane (first lane) in which the own vehicle M is located. When it is determined whether or not the second lane different from) can be further recognized, it is determined that the second lane can be recognized, and the traveling direction of the second lane intersects with the traveling direction of the own lane (first lane). , Recognize the second lane as an adjacent lane and estimate the risk of objects in the second lane.

FIG. 8 is a diagram showing an example of a specific scene. In the figure, L1 is the own lane (first lane), and L2 is the merging lane (second lane). _DL1 is the traveling direction of the own lane, and _DL2 is the traveling direction of the merging lane. The "direction of travel of a lane" means the direction of travel of a vehicle traveling in the normal direction in that lane. "Side" means "front" the range opened by a predetermined angle (for example, about 15 degrees) to the left and right centering on the front of the own vehicle M, and "front" the range opened by a predetermined angle centered on the rear of the own vehicle M. When defined as "backward", it is the area other than those. Strictly speaking, it is "sideways" and "within a predetermined distance (for example, several tens [m])". Further, m in the figure is another vehicle m which is an example of an object existing in the merging lane (second lane), and Rm shows a risk distribution estimated by the object recognition unit 132 for the other vehicle m. In the figure, the dark part indicates that the risk (index value) is high. For example, the object recognition unit 132 derives the risk centering on the representative point of the other vehicle m so that the closer to the representative point, the higher the risk, and the farther away from the representative point, the lower the risk. Then, the action plan generation unit 140 generates a target trajectory with one of the guidelines to avoid a high-risk position. As a result, the risk of the vehicle merging into the own lane can be appropriately evaluated, and unnecessary approach to the object can be suppressed.

[Processing flow]
FIG. 9 is a flowchart showing an example of the processing flow by the zebra zone recognition unit 133. First, the zebra zone recognition unit 133 determines whether or not the own vehicle M is turning (step S100). If it is determined that the own vehicle M is turning, the current cycle process is not performed and the previous recognition result is maintained (step S114). That is, the specific unit 133C of the zebra zone recognition unit 133 does not specify the cycle this time and maintains the previous result.

When it is determined that the own vehicle M is not turning, the plane image generation unit 133A acquires a camera image (step S102) and generates a plane image (step S104).

Next, the processes of steps S106 to S112 are executed for each monitoring area. The detection line setting unit 133B sets the first detection line and the second detection line (step S106). The zebra zone identification unit 133C determines whether or not all of the first detection line and the second detection line intersect the contour of the line mark (step S108). When all of the first detection line and the second detection line intersect the contour of the line mark, the zebra zone identification unit 133C identifies the monitoring area as the zebra zone (step S110). If even a part of the first detection line and the second detection line does not intersect the contour of the line mark, the zebra zone identification unit 133C does not specify the monitoring area as the zebra zone (step S112).

As described above, according to the first embodiment, the vehicle progresses based on the information mounted on the vehicle (own vehicle M) and acquired from the recognition device (camera 10) for recognizing the surrounding conditions of the vehicle. A generation unit (plane image generation unit 133A) that generates a plane image representing the distribution of the color or brightness of the road in the direction, a plurality of first detection lines parallel to each other on the plane image generated by the generation unit, and a first A setting unit (detection line setting unit 133B) for setting a plurality of second detection lines orthogonal to each of the detection lines, and each of the plurality of first detection lines and the plurality of second detection lines are road markings in a plane image. By providing a specific unit (Zebra zone identification unit 133C) that determines whether or not it intersects the contour and identifies the road marking as a zebra zone based on the result of the determination, the zebra zone is suitable for simple processing. Can be recognized. Further, according to the first embodiment, since the action plan generation unit 140 generates a target trajectory reflecting the specific result of the zebra zone, the own vehicle M is suitably driven based on the specific result of the zebra zone. Can be done.

<Second Embodiment>
Hereinafter, the second embodiment will be described. FIG. 10 is a functional configuration diagram of the vehicle control device 100 # according to the second embodiment. Hereinafter, "#" will be added to the reference numerals for the configurations peculiar to the second embodiment.

The zebra zone recognition unit 133 # according to the second embodiment includes, for example, a detection line setting unit 133 # B and a zebra zone identification unit 133C.

FIG. 11 is a diagram for explaining the processing of the detection line setting unit 133 # B of the second embodiment. As shown in the figure, the detection line setting unit 133 # B disappears from an arbitrary point in the image in a predetermined area (area corresponding to the road in front of the own vehicle M) in the captured image (camera image) acquired from the camera 10. A plurality of third detection line DL3s along the vanishing point toward the point VP and a plurality of fourth detection lines DL4 extending in the lateral direction of the image are set. The detection line setting unit 133 # B may shorten the interval of the third detection line DL3 as the distance from the own vehicle M increases. FIG. 11 is a diagram for conceptual explanation only, and in reality, the third detection line DL3 and the fourth detection line DL4 are set at a higher density than that shown in FIG. 11 in the monitoring area set in a rectangular shape or the like. Rectangle. Then, in the zebra zone identification unit 133 # C, as in the first embodiment, all of the third detection line DL3 in the monitoring area intersects the contour of the line mark, and the fourth detection line DL4 in the monitoring area If everything intersects the contour of the line mark, it is determined that the monitoring area is part of the zebra zone, otherwise it is determined that the monitoring area is not part of the zebra zone.

Since the vanishing line in the camera image is theoretically almost parallel to the road marking line, setting the vanishing line as the third detection line DL3 is the first along the extending direction of the lane after converting to a flat image. 1 It is the same as setting the detection line DL1 in principle. Further, since the line along the horizontal direction of the image becomes a line orthogonal to the extending direction of the lane when converted to a flat image, setting the fourth detection line DL4 along the horizontal direction of the image is a flat image. In principle, it is the same as setting the second detection line DL2 in the direction orthogonal to the first detection line DL1. Therefore, according to the second embodiment, the same effect as that of the first embodiment can be obtained. Since it is assumed that the vanishing line does not match the extending direction of the lane due to a change in the height of the road surface, the first embodiment is superior in terms of accuracy, but the process of converting to a flat image is performed. Since it is unnecessary, the second embodiment has a merit that the processing load can be reduced.

The flowchart of FIG. 9 can be used for the flow of processing executed by the zebra zone recognition unit 133 # of the second embodiment, except that step S104 is omitted and the processing content of step S106 is the same as that of the first embodiment. different.

According to the second embodiment described above, in the captured image (camera image) mounted on the vehicle (own vehicle M) and acquired from the camera (10) that captures the surrounding situation of the vehicle, a vanishing point (camera image) is obtained from an arbitrary point. Setting unit (detection line setting unit 133 # B) for setting a plurality of third detection lines (DL3) along the vanishing point toward VP) and a plurality of fourth detection lines (DL4) along the horizontal direction of the plane image. And, it is determined whether or not each of the plurality of third detection lines and the plurality of fourth detection lines intersects the contour of the road sign in the plane image, and the road sign is specified as the zebra zone based on the result of the determination. By providing the unit (zebra zone specific unit 133 # C), the above-mentioned effect can be obtained.

[Hardware configuration]
FIG. 12 is a diagram showing an example of the hardware configuration of the vehicle control device of the embodiment. As shown in the figure, the vehicle control device 100 of the first embodiment (or the vehicle control device 100 # of the second embodiment) includes a communication controller 100-1, a CPU 100-2, a RAM 100-3 used as a working memory, and a boot. The ROM 100-4 for storing programs, the storage devices 100-5 such as flash memory and HDD, the drive device 100-6, and the like are connected to each other by an internal bus or a dedicated communication line. The communication controller 100-1 communicates with a component other than the vehicle control device 100. The storage device 100-5 stores a program 100-5a executed by the CPU 100-2. This program is expanded into RAM 100-3 by a DMA (Direct Memory Access) controller (not shown) or the like, and is executed by CPU 100-2. As a result, a part or all of the first control unit 120 and the second control unit 160 is realized.

The embodiment described above can be expressed as follows.
A storage device that stores programs and
With a hardware processor,
The hardware processor executes a program stored in the storage device by executing the program.
Based on the information mounted on the vehicle and acquired from the recognition device for recognizing the surrounding situation of the vehicle, a plane image showing the distribution of the color or brightness of the road in the traveling direction of the vehicle is generated.
A plurality of first detection lines parallel to each other and a plurality of second detection lines orthogonal to each of the first detection lines are set in the generated plane image.
It is determined whether or not each of the plurality of first detection lines and the plurality of second detection lines intersects the contour of the road marking in the plane image.
Identify the road marking as a zebra zone based on the result of the judgment,
A zebra zone recognizer that is configured to.

The embodiment described above can also be expressed as follows.
A storage device that stores programs and
With a hardware processor,
The hardware processor executes a program stored in the storage device by executing the program.
In the captured image acquired from the camera mounted on the vehicle and capturing the surrounding situation of the vehicle, a plurality of third detection lines along the vanishing point from an arbitrary point to the vanishing point and a plurality of the plurality of detection lines along the lateral direction of the plane image. Set with the 4th detection line of
It is determined whether or not each of the plurality of third detection lines and the plurality of fourth detection lines intersects the contour of the road marking in the plane image.
Identify the road marking as a zebra zone based on the result of the judgment,
A zebra zone recognizer that is configured to.

Although the embodiments for carrying out the present invention have been described above using the embodiments, the present invention is not limited to these embodiments, and various modifications and substitutions are made without departing from the gist of the present invention. Can be added.

1 Vehicle system 10 Camera 12 Radar device 14 LIDAR
16 Object recognition device 20 Communication device 40 Vehicle sensor 50 Navigation device 100 Vehicle control device 120 First control unit 130 Recognition unit 131 Own vehicle position recognition unit 132 Object recognition unit 133 Zebra zone recognition unit 133A Plane image recognition unit 133B Detection line setting unit 133C Zebra zone identification unit 140 Action plan generation unit 160 Second control unit 100 # Vehicle control device 130 # Recognition unit 133 # Zebra zone recognition unit 133 # B Detection line setting unit 133 # C Zebra zone identification unit

Claims (14)

A generator that is mounted on the vehicle and generates a plane image showing the distribution of the color or brightness of the road in the traveling direction of the vehicle based on the information acquired from the recognition device for recognizing the surrounding situation of the vehicle.
A setting unit for setting a plurality of first detection lines parallel to each other and a plurality of second detection lines orthogonal to each of the first detection lines in the plane image generated by the generation unit.
It is determined whether or not each of the plurality of first detection lines and the plurality of second detection lines intersects the contour of the road marking in the plane image, and the road marking is identified as the zebra zone based on the result of the determination. With a specific part to do
A zebra zone recognition device equipped with. The specific unit sets the plurality of first detection lines or the plurality of second detection lines so as to be parallel to the longitudinal direction of the lane in which the vehicle is located.
The zebra zone recognition device according to claim 1. The setting unit sets the first detection line and the second detection line in the monitoring area, and sets the first detection line and the second detection line.
In the specific unit, all of the first detection lines in the monitoring area intersect with the contour of the road marking in the plane image, and all of the second detection lines in the monitoring area are roads in the plane image. If it intersects the contour of the marking, it is determined that the monitoring area is part of the zebra zone, otherwise it is determined that the monitoring area is not part of the zebra zone.
The zebra zone recognition device according to claim 1 or 2. The monitoring area is an area including a predetermined number of the first detection lines or the second detection line.
The zebra zone recognition device according to claim 3. In the captured image acquired from the camera mounted on the vehicle and capturing the surrounding situation of the vehicle, a plurality of third detection lines along the vanishing point from an arbitrary point to the vanishing point and a plurality of the captured images along the lateral direction of the captured image. The setting unit that sets the 4th detection line of
It is determined whether or not each of the plurality of third detection lines and the plurality of fourth detection lines intersects the contour of the road marking in the captured image, and the road marking is identified as the zebra zone based on the result of the determination. With a specific part to do
A zebra zone recognition device equipped with. The setting unit sets the third detection line and the fourth detection line in the monitoring area, and sets the third detection line and the fourth detection line.
In the specific unit, all of the third detection lines in the monitoring area intersect with the contour of the road marking in the captured image, and all of the fourth detection lines in the monitoring area are roads in the captured image. If it intersects the contour of the marking, it is determined that the monitoring area is part of the zebra zone, otherwise it is determined that the monitoring area is not part of the zebra zone.
The zebra zone recognition device according to claim 5. The monitoring area is an area including a predetermined number of the third detection line or the fourth detection line.
The zebra zone recognition device according to claim 5 or 6. The specific unit periodically repeats the process of specifying the zebra zone.
Whether or not the vehicle is turning is determined based on at least a part or all of the state of the vehicle direction indicator, the detection result of the yaw rate sensor, and the detection result of the gyro sensor.
When it is determined that the vehicle is turning, the zebra zone is not specified, and the specific result before determining that the vehicle is turning is maintained.
The zebra zone recognition device according to any one of claims 1 to 7. The zebra zone recognition device according to any one of claims 1 to 8.
A driving control unit that controls the steering and speed of the vehicle based on the processing result of the zebra zone recognition device.
A vehicle control device. Further provided with an object recognition unit that recognizes an object existing around the vehicle and estimates the risk due to the object.
When the zebra zone specified by the specific unit is on the side of the vehicle, the object recognition unit is located in front of the zebra zone and is in a second lane different from the first lane in which the vehicle is located. Further, it is determined whether or not the vehicle can be recognized, and when it is determined that the second lane can be recognized and the traveling direction of the second lane intersects the traveling direction of the first lane, the second lane is referred to as the first lane. Recognize as an adjacent lane to the lane and estimate the risk of an object in the second lane.
The vehicle control device according to claim 9. The computer
Based on the information mounted on the vehicle and acquired from the recognition device for recognizing the surrounding situation of the vehicle, a plane image showing the distribution of the color or brightness of the road in the traveling direction of the vehicle is generated.
A plurality of first detection lines parallel to each other and a plurality of second detection lines orthogonal to each of the first detection lines are set in the generated plane image.
It is determined whether or not each of the plurality of first detection lines and the plurality of second detection lines intersects the contour of the road marking in the plane image.
Identify the road marking as a zebra zone based on the result of the judgment,
Zebra zone recognition method. On the computer
Based on the information mounted on the vehicle and acquired from the recognition device for recognizing the surrounding situation of the vehicle, a plane image showing the distribution of the color or brightness of the road in the traveling direction of the vehicle is generated.
A plurality of first detection lines parallel to each other and a plurality of second detection lines orthogonal to each of the first detection lines are set in the generated plane image.
It is determined whether or not each of the plurality of first detection lines and the plurality of second detection lines intersects the contour of the road marking in the plane image.
Based on the result of the judgment, the road marking is identified as a zebra zone.
program. The computer
In the captured image acquired from the camera mounted on the vehicle and capturing the surrounding situation of the vehicle, a plurality of third detection lines along the vanishing point from an arbitrary point to the vanishing point and a plurality of the captured images along the lateral direction of the captured image. Set with the 4th detection line of
It is determined whether or not each of the plurality of third detection lines and the plurality of fourth detection lines intersects the contour of the road marking in the captured image.
Identify the road marking as a zebra zone based on the result of the judgment,
Zebra zone recognition method. On the computer
In the captured image acquired from the camera mounted on the vehicle and capturing the surrounding situation of the vehicle, a plurality of third detection lines along the vanishing point from an arbitrary point to the vanishing point and a plurality of the captured images along the lateral direction of the captured image. Set the 4th detection line of
It is determined whether or not each of the plurality of third detection lines and the plurality of fourth detection lines intersects the contour of the road marking in the captured image.
Based on the result of the judgment, the road marking is identified as a zebra zone.
program.

Images