JP2020135091A - Recognition device and recognition method - Google Patents

Abstract

To improve recognition accuracy of a traveling compartment line.SOLUTION: Recognition devices 105, 105a for recognizing a traveling compartment line pair that partition a traveling lane LN1 of vehicles 10, 10a includes: a candidate extraction unit 111 for extracting a plurality of compartment line candidates using an image; a selection execution unit 112 for calculating a reliability degree of the traveling compartment line pair candidate by using lane widths W1, W2 of the traveling compartment line pair candidate consisting of a pair of lane compartment candidates and a learned lane width Ws to select the traveling compartment line pair candidate satisfying first conditions including the reliability degree satisfying a criterion; a learning unit 114 for learning the lane width of the selected traveling compartment line pair; and a scene determination unit 113 for determining whether or not a scene appearing in an image is a first scene that is a connection end point A1 of the traveling lane and a branch lane LN2. The learning unit decreases a degree of reflection of the lane width of the traveling compartment line to learning results relative to a case where the scene is not the first scene when the scene is determined to be the first scene.SELECTED DRAWING: Figure 1

Description

The present disclosure relates to a recognition device and a recognition method for recognizing a vehicle lane marking pair.

There is known a technique for recognizing a lane marking of a own vehicle by using an image captured by an in-vehicle camera. Patent Document 1 describes a device that calculates the certainty that the lane marking candidate extracted from the captured image is the traveling lane marking of the own vehicle and selects the traveling lane marking from the lane marking candidates based on the certainty. There is. The certainty is calculated using the learning value obtained by learning the lane width of the selected lane marking candidate.

Japanese Unexamined Patent Publication No. 2016-162323

In the technique of Patent Document 1, the learning value representing the lane width obtained by learning the lane width is expanded when the transition from the straight portion to the curved portion of the lane is determined, and the transition from the curved portion to the straight portion is performed. By narrowing the lane when it is determined, the lane width of the straight portion is narrower than that of the curved portion, so that erroneous recognition of the traveling lane marking is suppressed. However, in Patent Document 1, the recognition of the traveling lane marking when the lane branches in the traveling direction of the vehicle is not considered. The inventors may discontinue the marking lines painted on the road surface near the connection end point between the traveling lane and the branch lane of the vehicle, and the recognition accuracy of the marking lines may decrease. I found that.

The present disclosure can be realized in the following forms.

According to the first aspect of the present disclosure, there is provided a recognition device (105, 105a) that recognizes a traveling lane marking pair that partitions a traveling lane (LN1) of a vehicle (10, 10a). This recognition device includes a candidate extraction unit (111) that extracts a plurality of lane marking candidates (LC1 to LC5, LC11 to LC16) using an image taken by a camera (130) mounted on the vehicle, and the vehicle. A pair of the lane marking candidates that are in front of the vehicle and lined up in the vehicle width direction are extracted as lane marking pair candidates that are candidates for the traveling lane marking pair, and the lane width of the traveling lane marking pair candidate and the past Using the learning lane width (Ws) learned from the lane width of the lane marking pair recognized in the above, the reliability that the lane marking pair candidate is the lane marking pair is calculated. The selection execution unit (112) that selects the traveling lane line pair candidate that satisfies the first condition including the reliability satisfying a predetermined criterion as the traveling lane line pair is selected by the selection execution unit. The learning unit (114) that calculates the learning lane width by learning the lane width of the traveling lane pair and the scene that appears in the image taken by the camera are branched from the traveling lane (LN1) of the vehicle. The learning unit includes a scene determination unit (113) for determining whether or not the first scene is the connection end point (A1) with the lane (LN2), and the learning unit is the first scene. , The degree of reflection of the lane width of the traveling lane pair in the learning result is reduced as compared with the case where it is not the first scene.

According to this form, even if the lane markings may widen or become discontinuous at the end point of the connection between the traveling lane and the branch lane, the lane width is prevented from being erroneously learned. can do. Therefore, since the traveling lane marking pair is selected using an appropriate learning lane width, the recognition accuracy of the traveling lane marking pair can be improved.

According to the second aspect of the present disclosure, there is provided a recognition device (105, 105a) that recognizes a traveling lane marking pair that partitions a traveling lane (LN1) of a vehicle (10, 10a). This recognition device includes a candidate extraction unit (111) that extracts a plurality of lane marking candidates (LC1 to LC5, LC11 to LC16) using an image taken by a camera (130) mounted on the vehicle, and the vehicle. A pair of the lane marking candidates that are in front of the vehicle and lined up in the vehicle width direction are extracted as the traveling lane marking pair candidates that are candidates for the traveling lane marking pair, and the lane width of the traveling lane marking pair candidate and the past Using the learning lane width (Ws) learned from the lane width of the lane marking pair recognized in the above, the reliability that the lane marking pair candidate is the lane marking pair is calculated. The selection execution unit (112) that selects the traveling lane line pair candidate that satisfies the first condition including the reliability satisfying a predetermined criterion as the traveling lane line pair is selected by the selection execution unit. The learning unit (114) that calculates the learning lane width by learning the lane width of the traveling lane pair, and the scene that appears in the image taken by the camera are the traveling lane and the branch lane (LN2) of the vehicle. A scene determination unit (113) for determining whether or not the first scene is the connection end point (A1) with the vehicle, and the scene determination unit is in front of the vehicle and in the vehicle width direction. When the lane marking candidate satisfying a predetermined third condition is extracted on one side of the vehicle, it is determined that the scene appearing in the image is the first scene, and the third condition is the vehicle. Two first pairs (LC3, LC4), which are candidates for the lane markings, which are separated from each other toward the front of the vehicle, are extracted, and the first pair (LC3, LC4) is located in front of the first pair and on the other side in the vehicle width direction. The selection execution unit includes that the separated lane candidate (LC5), which is the separated lane candidate separated from the first pair, is further extracted, and the selection execution unit indicates that the scene appearing in the image is the first scene. When the separation lane line candidate is further extracted, the traveling section including the separation lane line candidate and the lane marking candidate (LC1) extracted on the other side in the vehicle width direction of the vehicle. The criteria for line pair candidates (LC1, LC5) are relaxed more than the criteria for other lane marking pair candidates.

According to this form, even if a lane marking pair with a wide lane width is recognized at the connection end point between the lane and the branch lane, a lane marking pair candidate having an appropriate lane width is selected as the driving lane marking pair. It is possible to increase the possibility of Therefore, it is possible to increase the possibility that the traveling lane marking pair having an appropriate lane width is recognized at an early stage, and thus it is possible to improve the recognition accuracy of the traveling lane marking pair in the recognition device.

The present disclosure can also be realized in various forms other than the recognition device. For example, it can be realized in the form of a method of recognizing a traveling lane marking pair, a computer program for realizing such a method, a storage medium for storing such a computer program, or the like.

Schematic diagram of a vehicle equipped with a recognition device. The process chart which shows the recognition process of a traveling lane marking pair. Image diagram of the extracted lane marking candidates. The figure for demonstrating the first scene. The process diagram of the lane width learning in the 1st Embodiment. The figure for demonstrating the 1st scene determination in the case where the 1st shape does not appear. An image diagram showing a first pair appearing at a connection end point and a candidate for a separation lane marking. The process diagram of the lane width learning in the third embodiment. The figure for demonstrating the road marking pair selected at the connection end point. The figure for demonstrating the traveling lane marking pair selection process in 5th Embodiment. The process chart which shows the traveling lane marking pair selection process in 5th Embodiment. The schematic block diagram of the vehicle provided with the recognition device in 6th Embodiment. Image diagram to inform that the lane marking is recognized. Image diagram to notify that the lane marking is not recognized.

A. First Embodiment As shown in FIG. 1, the vehicle 10 as one embodiment of the present disclosure includes an automatic driving control system 100. In the present embodiment, the automatic operation control system 100 includes a recognition device 105, a camera 130, a detection device 132, an operation control unit 210, a driving force control ECU 220 (Electronic Control Unit) driving force control ECU 220, and braking force control. It includes an ECU 230 and a steering control ECU 240. The recognition device 105, the operation control unit 210, the driving force control ECU 220, the braking force control ECU 230, and the steering control ECU 240 are connected via the vehicle-mounted network 260. The vehicle 10 is also called "own vehicle". In the present embodiment, the automatic driving control system 100 executes the automatic driving of the vehicle 10. The vehicle 10 may be manually driven by the driver.

The camera 130 mounted on the vehicle 10 repeatedly captures images of at least the periphery of the vehicle 10 in the traveling direction at preset time intervals. In the present embodiment, the camera 130 is installed toward the front, side, and rear of the vehicle 10 to acquire captured images of the front, side, and rear. The side is the width direction of the vehicle 10. The rear includes not only the rear front of the vehicle 10 but also the right rear and the left rear. A monocular camera may be used as the camera 130. Further, a stereo camera or a multi-camera composed of two or more cameras may be used. The image taken by the camera 130 includes the road surface in the traveling direction of the vehicle 10.

The detection device 132 includes a sensor group that monitors the periphery of the vehicle 10, a sensor group that monitors the running state of the vehicle 10, and an information communication device that uses V2X (Vehicle to Everything). Examples of the sensor group for monitoring the periphery of the vehicle 10 include an object sensor using a reflected wave such as a millimeter wave radar, LiDAR (Light Detection And Langing or Laser Imaging Detection And Ranking), and an ultrasonic sensor. Examples of the sensor group for monitoring the traveling state of the vehicle 10 include a vehicle speed sensor, an acceleration sensor, a GNSS (Global Navigation Satellite System) sensor, and a yaw rate sensor. The vehicle speed sensor detects the speed of the vehicle 10. The acceleration sensor detects the acceleration of the vehicle 10. The yaw rate sensor detects the rotational angular velocity of the vehicle 10. The GNSS sensor is composed of, for example, a GPS (Global Positioning System) sensor, and detects the current position of the vehicle 10 based on radio waves received from navigation satellites constituting the GPS. The information communication device using V2X executes wireless communication with an intelligent transport system (Intelligent Transport System), inter-vehicle communication with other vehicles, and road-to-vehicle communication with a roadside radio installed in road equipment. By doing so, information on the situation of the vehicle 10 and the surrounding situation can be exchanged with other vehicles and the like. Each sensor included in the detection device 132 is provided with an ECU (Electronic Control Unit) that processes the detection result of each sensor. The detection device 132 can detect the above-mentioned various information. The detection result of the detection device 132 is used for driving control of the vehicle 10.

The recognition device 105 recognizes a traveling lane marking pair, which is a pair of lane markings that partition the traveling lane of the vehicle 10. The recognition device 105 is a device that can estimate the shape of the road on which the vehicle 10 travels based on the recognition result and output it to the driving control unit 210 or the like. The recognition device 105 includes a candidate extraction unit 111, a selection execution unit 112, a scene determination unit 113, and a learning unit 114. The recognition device 105 is configured as one or more computers including a CPU 110, a memory 120, and an interface (not shown). The CPU 110 realizes the functions of the above-mentioned parts by expanding and executing the program P1 stored in the memory 120. However, some or all of the functions of these parts may be realized by a hardware circuit.

The candidate extraction unit 111 extracts a plurality of compartment paints using the image taken by the camera 130. The division paint is a figure painted on the road surface, for example, in white or yellow, for forming a division line. The candidate extraction unit 111 connects the division paints to form a line segment, and extracts the line segment as a division line candidate. The lane marking candidate constitutes a traveling lane marking pair candidate described later.

The selection execution unit 112 extracts a plurality of traveling lane marking pair candidates from the plurality of lane marking candidates extracted by the candidate extraction unit 111. The traveling lane marking pair candidate is a pair of lane marking candidates arranged in the vehicle width direction in front of the vehicle 10 in the traveling direction. The traveling lane marking pair candidate includes a lane marking candidate on one side and a lane marking candidate on the other side in the vehicle width direction of the vehicle 10. Hereinafter, the traveling lane marking pair candidate is also simply referred to as a “pair candidate”.

The selection execution unit 112 calculates the distance between the pair candidates as the lane width of the pair candidates. The selection execution unit 112 calculates the reliability that the pair candidate is a traveling lane marking pair by using the lane width of the pair candidate and the learning lane width. The learning lane width is a learning value of the lane width of the traveling lane pair that has been recognized in the past by the recognition device 105. The relationship between the lane width of the pair candidate and the reliability is predetermined so that the closer the lane width of the pair candidate is to the learning lane width, the higher the reliability. The reliability may be calculated by using the contrast, length, etc. of the pair candidate in addition to the lane width of the pair candidate.

The selection execution unit 112 selects a pair candidate that satisfies the first condition including the reliability satisfying a predetermined criterion from the plurality of pair candidates as the traveling lane marking pair. In the present embodiment, the first condition is satisfied when a predetermined standard is satisfied. The predetermined criteria include the pair candidate having the highest reliability among the plurality of pair candidates. Since the currently recognized lane marking pair (also called the currently recognized lane) is selected and recognized using the learning lane width, the lane width of the pair candidate continuously connected to the currently recognized lane is the most as the learning lane width. It is close and is considered to have the highest reliability. Therefore, in the place where the lane markings are continuous, the pair candidate currently continuous with the recognition line can be selected as the traveling lane marking pair by the selection execution unit 112. In the present embodiment, the predetermined criteria include that the state in which the reliability of the pair candidate is the highest continues for a predetermined first period or more. It should be noted that the predetermined criteria may include that the state in which the reliability of the pair candidate is the highest continues for the first distance or more corresponding to the predetermined first period. For example, it is conceivable that pair candidates closer to the learning lane width may be temporarily extracted due to dirt on the road surface or the like. However, if the state with the highest reliability is not continued for a predetermined first period or more, even a pair candidate closer to the learning lane width is not selected as a traveling lane marking pair. In the present embodiment, since the traveling lane marking pair is selected using the predetermined criteria, it is possible to prevent the recognition device 105 from recognizing the traveling lane marking pair that is not currently connected to the recognition line. Further, it is possible to prevent the lane width of the traveling lane marking pair recognized by the recognition device 105 from being changed in a relatively short period of time.

The pair candidate selected by the selection execution unit 112 is recognized as a traveling lane marking pair by the CPU 110. The CPU 110 estimates the running road shape of the vehicle 10 by applying a road shape model stored in the memory 120, a known filter such as a Kalman filter, or the like to the selected pair candidate, and the driving control unit 210. Output to etc.

The scene determination unit 113 determines whether or not the scene appearing in the image captured by the camera 130 is the first scene. The first scene will be described later.

The learning unit 114 learns the lane width of the pair candidate selected by the selection execution unit 112, that is, the traveling lane marking pair to be recognized, calculates and updates the learning lane width. The learning lane width at the initial stage of learning may be the lane width of a general traveling lane stored in the memory 120 in advance. When the scene appearing in the image taken by the camera 130 is the first scene, the learning unit 114 reflects the degree of reflection of the lane width of the traveling lane pair in the learning result as compared with the case where it is not the first scene. , Decrease.

The recognition process of the traveling lane marking pair executed by the recognition device 105 will be described with reference to FIG. The recognition process is, for example, a process that is started when the start switch of the vehicle 10 is turned on and is repeatedly executed. When the recognition process is started, the recognition device 105 repeatedly acquires the image taken by the camera 130.

In step S10, the candidate extraction unit 111 extracts a plurality of compartment paints by performing predetermined image processing on the image captured by the camera 130. In the present embodiment, the candidate extraction unit 111 extracts an up-edge point in the image in which the brightness value rises by a threshold value or more and a down-edge point in the image in which the brightness value falls by a threshold value or more by performing known edge processing on the image. .. The candidate extraction unit 111 extracts a plurality of section paints using the extracted down edge points and up edge points.

In step S20, the candidate extraction unit 111 connects the extracted section paints and extracts the section line candidates. As shown in FIG. 3, in step S20, for example, the lane marking candidates LC11, LC12, LC14, LC15 specified from the road markings 40 and 41 on the road surface, the lane marking candidate LC16 specified from the roadside object 30, and the lane marking candidates LC16. The marking line candidate LC13 specified from the dirt Ct on the road surface is extracted. Hereinafter, the recognition process will be described with the right side of the vehicle 10 in the vehicle width direction as "one side" and the left side of the vehicle 10 in the vehicle width direction as "the other side".

In step S30, the scene determination unit 113 determines whether or not the scene appearing in the image captured by the camera 130 is the first scene. FIG. 4 is an image diagram for explaining the first scene. The lane LN1 shown in FIG. 4 is a lane partitioned by a compartment paint 40 on the other side of the vehicle 10 and compartment paints 40, 41, 42 on one side. The lane LN2 is partitioned by the compartment paints 40 and 43 on one side of the vehicle 10 and the compartment paint 44 on one side of the compartment paints 40 and 43. The lane LN2 is a branch lane branched from the lane LN1 on the rear side of the vehicle 10 in the traveling direction. The starting position of the branch is not shown. The distance along the traveling direction of the compartment paint 41 is, for example, 50 m or more.

As shown in FIG. 4, the scene determination unit 113 determines that the scene is the first scene when the connection end point A1 between the traveling lane LN1 of the vehicle 10 and the branch lane LN2 appears in the image. The connection end point A1 is a point that appears in front of the vehicle 10 and on one side of the vehicle 10. As shown in FIG. 4, at the connection end point A1, the lane width may be widened or the compartment paints 41 and 42 may be discontinuous due to the compartment paint 41 spreading to one side.

The scene determination unit 113 appears in an image when a section paint satisfying a second condition including forming a predetermined first shape is extracted on one side of the vehicle 10 in front of the vehicle 10. It is determined that the scene is the first scene. The second condition is a condition that can be satisfied when the point A2 included in the connection end point A1 appears in the image taken by the camera 130. The first shape is a shape in which the compartment paints are separated from each other toward the front of the vehicle 10 and connected to each other on the side closer to the vehicle 10. The first shape is a shape corresponding to the so-called "V" shape of the alphabet. The scene determination unit 113 uses the shape of the partition paint at the point A2 at the connection end point A1 stored in the memory 120 in advance, a function representing the shape, and the extracted partition paint to generate the extracted partition paint. It is determined whether or not the first shape is formed. The second condition may include that the extracted compartment paint has a predetermined contrast or a predetermined length. The compartment paints 41 and 43 shown in FIG. 4 are separated from each other toward the front of the vehicle 10 and are connected to each other on the side closer to the vehicle 10. The compartment paints 41 and 43 form a so-called "V" shape. When such section paints 41 and 43 are extracted, the scene determination unit 113 determines that the scene appearing in the image is the first scene.

The scene determination unit 113 may determine that the division paint satisfies the second condition by using the division line candidates obtained by connecting the division paints. For example, the scene determination unit 113 extracts two lane marking candidates satisfying a second condition including forming a predetermined first shape on one side of the vehicle 10 in front of the vehicle 10. Judge whether or not. The scene determination unit 113 determines that the scene appearing in the image is the first scene when two lane marking candidates satisfying the second condition are extracted. In this case, the first shape is a shape in which the two lane marking candidates are separated from each other in the traveling direction of the vehicle 10 and are connected to each other on the side closer to the vehicle 10. "The two lane marking candidates are connected on the side closer to the vehicle" includes that the two lane marking candidates are completely connected and that they exist within a predetermined distance. The second condition may further include that the two marking line candidates have a predetermined contrast and a predetermined length. FIG. 4 shows the lane marking candidates LC1, LC2, LC3, and LC4, which are a part of the plurality of lane marking candidates extracted by the candidate extraction unit 111. The lane marking candidates LC3 and LC4 are in front of the vehicle 10 in the traveling direction, exist on one side in the vehicle width direction, and are separated from each other in the traveling direction of the vehicle 10. The lane marking candidates LC3 and LC4 form a so-called "V" shape. In such a case, the scene determination unit 113 can determine that the scene appearing in the image is the first scene.

In step S40, the selection execution unit 112 executes a travel marking line pair selection process that calculates the reliability and selects the traveling marking line pair. First, the selection execution unit 112 extracts a plurality of pair candidates from the plurality of lane marking candidates extracted by the candidate extraction unit 111. The selection execution unit 112 calculates the reliability for each pair candidate by using the lane width of the pair candidate and the learning lane width. The lane width W1 shown in FIG. 4 is the lane width of the pair candidate LC1 and LC3 including the lane marking candidate LC3 continuous with the lane marking candidate LC2 on one side of the vehicle 10 and the lane marking candidate LC1 on the other side of the vehicle 10. Is. The lane width W2 is the lane width of the pair candidate LC1 and LC4 including the lane marking candidate LC4 on one side of the lane marking candidate LC3 and the lane marking candidate LC1 on the other side of the vehicle 10. FIG. 4 further shows the learning lane width Ws. The lane width W1 of the pair candidates LC1 and LC3 is closer to the learning lane width Ws than the lane width W2 of the pair candidates LC1 and LC4. Therefore, in the example shown in FIG. 4, the reliability of the pair candidates LC1 and LC3 is the highest.

Next, the selection execution unit 112 selects a pair candidate that satisfies the first condition including the reliability satisfying a predetermined criterion from the plurality of pair candidates as the traveling lane marking pair. In the example shown in FIG. 4, the pair candidates having the highest reliability are the pair candidates LC1 and LC3. In the example shown in FIG. 4, the pair candidates LC1 and LC3, which are continuous line segments to the traveling lane marking pairs LC1 and LC2 and have the highest reliability, are selected and determined as the traveling lane marking pairs.

In step S60, the learning unit 114 learns the lane width of the pair candidate selected by the selection execution unit 112. When the learning unit 114 is determined to be the first scene in step S30, the degree of reflection of the lane width of the selected pair candidate in the learning result is higher than that when it is determined not to be the first scene. To reduce.

As shown in FIG. 5, in the lane width learning process of the present embodiment, the learning unit 114 acquires whether or not the scene determination unit 113 has determined that the scene is the first scene in step S61. When it is determined that the scene is the first scene, the learning unit 114 reduces the weighting of the lane width learning in step S62 as compared with the case where it is not the first scene. The weighting of the lane width learning in the case where it is not the first scene is predetermined and stored in the memory 120.

In step S63, the learning unit 114 learns the lane width of the pair candidate selected in step S40 (FIG. 2). If the affirmative determination is made in step S61, the learning unit 114 learns the lane width by using the weighting reduced in step S62. When the negative determination in step S61 is determined, the learning unit 114 learns the lane width by using a predetermined weighting. The time constant may be lowered, or learning using a low-pass filter may be performed.

Returning to FIG. 2, in step S70, the CPU 110 recognizes the pair candidate selected by the selection execution unit 112 as a travel marking line pair. The CPU 110 estimates the road shape while the vehicle 10 is traveling based on the recognition result, the road shape model stored in the memory, a known filter, and the previously recognized driving lane marking pair, and controls the operation. Output to unit 210 or the like. As described above, the recognition process of the traveling lane marking pair by the recognition device 105 is executed. Note that each step of the recognition process can be replaced as appropriate. For example, the order of step S30 and step S40 may be exchanged, and the order of step S60 and step S70 may be exchanged.

According to this embodiment, when the scene appearing in the image taken by the camera 130 is the first scene, the learning unit 114 learns the lane width of the traveling lane marking pair as compared with the case where it is not the first scene. Decrease the degree of reflection in. Therefore, at the connection end point A1 between the traveling lane LN1 and the branch lane LN2, even if the lane markings may widen or become discontinuous in the vehicle width direction, the lane width is prevented from being erroneously learned. be able to. Therefore, since the traveling lane marking pair can be recognized by using an appropriate learning lane width, the recognition accuracy of the traveling lane marking pair can be improved.

According to this embodiment, when the scene appearing in the image taken by the camera 130 is the first scene, the learning unit 114 weights the learning of the lane width of the traveling lane marking pair selected by the selection execution unit 112. By lowering the lane width as compared with the case where it is not the first scene, erroneous learning of the lane width can be suppressed.

According to this form, the scene determination unit 113 can determine whether or not it is the first scene by using the characteristic first shape appearing at the connection end point A1.

B. In the following description of the second embodiment, the same reference numerals as those of the first embodiment indicate the same configuration, and the preceding description will be referred to. The first scene determination process (see FIG. 2, step S30) in the present embodiment will be described with reference to FIGS. 6 and 7. As shown in FIG. 6, in the scene determination unit 113, the point A3 included in the connection end point A1 appears in the image even when the V-shaped section paint does not appear in the image due to an obstacle Ob or the like. In this case, it is determined that it is the first scene. Point A3 is a point where two compartment paints 41 and 43 that are separated from each other in the traveling direction of the vehicle 10 exist. The point A3 is a point where the compartment paints 41 and 42, which are further divided in the traveling direction and have different positions in the vehicle width direction, exist. FIG. 7 is an example of an image G in which the point A3 appears, which is taken by the camera 130.

In the present embodiment, the scene determination unit 113 has a predetermined third condition in front of the vehicle 10 and on one side in the vehicle width direction even when the division paint constituting the V-shape is not extracted. When the lane marking candidates satisfying the above conditions are extracted, it is determined that the scene is the first scene. The third condition can be satisfied when the point A3 appears in the image taken by the camera 130. The third condition is that the first pair, which is two lane marking candidates separated from each other in the traveling direction of the vehicle 10, is extracted, and is one side in the vehicle width direction of the vehicle 10, and the first pair. On the other side of the above, the lane marking candidates separated from the first pair are further extracted. Hereinafter, the lane marking candidate separated from the first pair is also referred to as a "separated lane marking candidate". The shortest distance between the first pair and the distance marking line candidate along the vehicle width direction is a predetermined distance, and may be, for example, a distance in any range of 0.5 m to 10 m. The third condition may further include that the first pair and the separation marking line candidate have a predetermined contrast or a predetermined length.

In the example shown in FIG. 6, the lane marking candidates LC3 and LC4 are in front of the vehicle 10 in the traveling direction, exist on one side of the vehicle 10, and are separated from each other in the traveling direction of the vehicle 10. The lane marking candidates LC3 and LC4 correspond to the first pair. Further, the separation lane marking candidate LC5 is extracted on one side of the vehicle 10 in the vehicle width direction and on the other side of the first pair LC3 and LC4. In such a case, the scene determination unit 113 determines that the scene appearing in the image is the first scene. Since the recognition device 105 and other configurations of the recognition process in the present embodiment are the same as those in the first embodiment, the description thereof will be omitted.

According to this form, the scene determination unit 113 can determine that it is the first scene even when the so-called "V" shape does not appear due to an obstacle Ob or the like, so that the lane width is erroneously learned. Can be further suppressed.

C. Third Embodiment In the recognition process in the third embodiment, the step of learning the lane width (FIG. 2, step S60) is different from that of the first embodiment. In the present embodiment, as shown in FIG. 8, when step S61 is affirmed, the learning unit 114 does not learn the lane width of the selected traveling lane marking pair in step S62a. If step S61 is denied, the learning unit 114 learns the lane width of the selected traveling lane marking pair in step S63a. Since the configuration of the recognition device 105 and other steps of the recognition process are the same as those in the above-described embodiment, the description thereof will be omitted.

According to this embodiment, in the case of the first scene, the learning unit 114 does not learn the lane width of the traveling lane marking pair selected by the selection execution unit 112, so that erroneous learning of the lane width can be further suppressed. Can be done. Therefore, the recognition accuracy of the traveling lane marking pair can be further improved.

D. Fourth Embodiment In the above embodiment, the selection execution unit 112 determines that the reliability of the plurality of pair candidates satisfies a predetermined criterion in the traveling lane marking pair selection process (see FIG. 2, step S40). A pair candidate satisfying the first condition including the first condition was selected as a traveling lane marking pair. Then, in the above embodiment, the first condition is satisfied when the predetermined standard is satisfied. On the other hand, in the third embodiment, the first condition includes that the lane marking candidate is not the lane marking candidate constituting the first scene. That is, in the present embodiment, the selection execution unit 112 does not select the pair candidate including the lane marking candidate constituting the first scene as the traveling lane marking pair. In the present embodiment, the selection execution unit 112 includes a line segment in which the lane marking candidate on the other side of the vehicle 10 is offset to one side by a distance corresponding to the learning lane width Ws, and the lane marking candidate on the other side of the vehicle 10. A pair candidate consisting of is selected as a road marking pair.

In the example shown in FIG. 9, the lane marking candidate LC3 is a lane marking candidate constituting the first scene. Therefore, the selection execution unit 112 does not select the pair candidates LC1 and LC3 as the traveling lane marking pair. When the scene determination unit 113 determines that the scene is the first scene, the selection execution unit 112 converts the lane marking candidate LC1 extracted on the other side of the vehicle 10 and the lane marking candidate LC1 into the learning lane width Ws. The line segment LCx obtained by moving the corresponding distance to one side of the vehicle 10 and the line segment LCx are selected as the traveling lane marking pair. The CPU 110 recognizes the lane marking candidate LC1 and the line segment LCx as a traveling lane marking pair. Since the configuration of the recognition device 105 and other steps of the recognition process are the same as those in the first embodiment, the description thereof will be omitted.

According to this embodiment, it is possible to prevent the lane width of the traveling lane pair recognized by the recognition device 105 from expanding at the connection end point A1 between the traveling lane LN1 and the branch lane LN2.

E. Fifth Embodiment The recognition process of the fifth embodiment will be described with reference to FIGS. 10 and 11. In the fifth embodiment, as shown in FIG. 10, when the separation lane marking candidate LC5 is extracted, the selection execution unit 112 has the separation lane marking candidate LC5 and the section extracted on the other side of the vehicle 10. The selection logic is changed so that the line candidate LC1 and the pair candidates LC1 and LC5 including the line candidate LC1 are selected at an early stage. The selection execution unit 112 is predetermined for selecting the traveling lane marking pair for the pair candidate LC1 and LC5 including the separation lane marking candidate LC5 and the lane marking candidate LC1 extracted on the other side of the vehicle 10. The criteria are relaxed more than other pair candidates.

In the traveling lane marking pair selection process (see FIG. 2, step S40), the selection execution unit 112 calculates the reliability of the extracted traveling lane marking pair candidates in step S41b as shown in FIG.

In step S42b, the selection execution unit 112 determines whether the scene appearing in the image is the first scene by the scene determination unit 113, and whether or not the separation division line candidate LC5 is extracted by the candidate extraction unit 111. To judge. When the affirmative determination is made in step S42b, the selection execution unit 112 relaxes the criteria used for selection in step 43b for the traveling lane marking pair candidates including the separation lane marking candidate LC5. In the present embodiment, the selection execution unit 112 selects the pair candidates LC1 and LC5 including the separation lane marking candidate LC5 as the traveling lane marking pair when the state having the highest reliability continues for the second period. The second period is a shorter period than the predetermined first period for the other pair candidates to be selected as the road marking pair.

In step S44b, the selection execution unit 112 selects a pair candidate whose reliability satisfies a predetermined criterion as a travel marking line pair. When the affirmative determination is made in step S42b, the selection execution unit 112 determines in advance that the pair candidate LC1 and LC5 have the highest reliability for the pair candidate LC1 and LC5 including the separation marking line candidate LC5. If the above is continued, it is selected as a traveling lane marking pair. When the negative determination in step S42b is made, the selection execution unit 112 selects the pair candidate whose highest reliability state continues for a predetermined first period or longer, as in the first embodiment, on the travel marking line. Select as a pair. The configuration of the recognition device 105 and other recognition processes in the present embodiment are the same as those in the first embodiment, and thus the description thereof will be omitted.

According to this embodiment, even if the traveling lane marking pairs LC1 and LC3 having wide lane widths are recognized at the connection end point A1 shown in FIG. 10, the traveling lane marking pair candidates LC1 and LC5 having appropriate lane widths are traveling compartments. It can increase the possibility of being selected as a line pair. Therefore, it is possible to increase the possibility that the traveling lane marking pairs LC1 and LC5 having an appropriate lane width are recognized at an early stage, so that the recognition accuracy of the traveling lane marking pair in the recognition device 105 can be improved.

According to this embodiment, when the traveling lane marking pairs LC1 and LC3 are recognized at the connection end point A1 shown in FIG. 10 and the separation lane marking candidate LC5 is extracted by the candidate extraction unit 111, the pair candidate LC1 , LC5 is selected as a road marking pair when the state having the highest reliability continues for a second period shorter than a predetermined first period. Therefore, the traveling lane marking pairs LC1 and LC5 having an appropriate lane width can be recognized at an early stage. Therefore, it is possible to improve the recognition accuracy of the traveling lane marking pair in the recognition device.

In the sixth embodiment, the selection execution unit 112 relaxes the criteria used for selection by making the pair candidates LC1 and LC5 including the separation marking line candidate LC5 more reliable than the other pair candidates. May be good. For example, the selection execution unit 112 may multiply the reliability of the traveling lane marking pair candidates LC1 and LC5 including the separation lane marking candidate LC5 by a coefficient larger than 1.

In the sixth embodiment, the learning unit 114 does not have to reduce the degree of reflection of the lane width of the selected traveling lane marking pair in the learning result even if the scene appearing in the image is the first scene. According to this embodiment, the possibility that the traveling lane marking pair candidates LC1 and LC5 having an appropriate lane width are selected as the traveling lane marking pair is increased, so that the recognition accuracy of the traveling lane marking pair in the recognition device 105 can be improved. it can.

F. Sixth Embodiment The vehicle 10a in the sixth embodiment shown in FIG. 12 includes an automatic driving control system 100a. The recognition device 105a included in the automatic operation control system 100a includes a CPU 110a and a memory 120a. The CPU 110a functions as a candidate extraction unit 111, a selection execution unit 112, a scene determination unit 113, a learning unit 114, and a notification control unit 115 by expanding and executing the program P1a stored in the memory 120a. The recognition device 105a is connected to the notification device 250 via the vehicle-mounted network 260.

The notification device 250 is a device for notifying the user of the vehicle 10a of information. In the present embodiment, the notification device 250 is a device for exchanging information between the user and the vehicle 10, and includes a liquid crystal panel, a lever, a button, and the like. In the present embodiment, the notification device 250 is a so-called HMI (Human Machine Interface). The notification control unit 115 displays information on the liquid crystal panel, and accepts user operations via the liquid crystal panel that accepts buttons, levers, and touch operations.

The notification control unit 115 acquires the determination result of the scene determination unit 113 and the selection result of the selection execution unit 112, that is, the recognition result of the recognition device 105. When the notification control unit 115 does not determine that the scene is the first scene by the scene determination unit 113, for example, as shown in FIG. 13, the notification device 250 recognizes the traveling lane marking pairs K1 and K2. Notify that you are there. From this display, the driver of the vehicle 10a can know that the lane markings are normally recognized on both sides of the vehicle 10a.

When the notification control unit 115 determines that it is the first scene, the notification control unit 115 notifies the notification device 250 that the lane marking K2 on one side of the vehicle 10 is not recognized. In the present embodiment, as shown in FIG. 14, the notification control unit 115 causes the notification device 250 to display only the lane marking K1 on the other side of the vehicle 10. Further, as shown in FIG. 14, the notification control unit 115 may display the message m1 on which the lane marking K2 on one side of the vehicle 10 is not recognized on the notification device 250. The notification device 250 may be a speaker, and the notification control unit 115 may notify by voice that the lane marking K2 on one side of the vehicle 10 is not recognized. Since other configurations and recognition processing of the recognition device 105a in this embodiment are the same as those in the above-described embodiment, the description thereof will be omitted.

According to this form, the driver of the vehicle 10a can be notified that the lane marking on one side of the vehicle 10a is not recognized. Therefore, when the traveling control of the vehicle 10a is performed using the recognized lane markings, the driver's attention to the traveling can be called out.

G. Other Embodiments In the above embodiment, one side of the vehicle 10 is on the right side with respect to the traveling direction of the vehicle 10, the other side of the vehicle 10 is on the left side with respect to the traveling direction of the vehicle 10, and the connection end point A1 is on the right side of the vehicle 10. An example in which is present has been described. On the other hand, the right side of the vehicle 10 with respect to the traveling direction may be the other side, and the left side of the vehicle 10 with respect to the traveling direction may be the other side. The connection end point A1 may exist on the left side of the vehicle 10.

In the above embodiment, a part of the configuration realized by the hardware may be replaced with software, or a part of the configuration realized by the software may be replaced with hardware. For example, at least one functional unit of the recognition devices 105 and 105a may be realized by an integrated circuit, a discrete circuit, or a module combining these circuits. In addition, when a part or all of the functions of the present disclosure are realized by software, the software (computer program) can be provided in a form stored in a computer-readable recording medium. "Computer readable recording medium" is not limited to portable recording media such as flexible disks and CD-ROMs, but is fixed to internal storage devices in computers such as various RAMs and ROMs, and computers such as hard disks. It also includes external storage devices that have been installed. That is, the term "computer-readable recording medium" has a broad meaning including any recording medium on which data packets can be fixed rather than temporarily.

The present disclosure is not limited to the above-described embodiment, and can be realized by various configurations within a range not deviating from the gist thereof. For example, the technical features in the embodiments corresponding to the technical features in the embodiments described in the column of the outline of the invention may be used to solve some or all of the above-mentioned problems, or a part of the above-mentioned effects. Or, in order to achieve all of them, it is possible to replace or combine them as appropriate. Further, if the technical feature is not described as essential in the present specification, it can be appropriately deleted.

10, 10a vehicle, 105, 105a recognition device, 111 candidate extraction unit, 112 selection execution unit, 113 scene judgment unit, 114 learning unit, 130 camera, A1 connection end point, LC1 to LC16 lane marking candidate, LN1, LN2 lane, W1, W2 lane width, Ws learning lane width

Claims (12)

A recognition device (105, 105a) that recognizes a traveling lane marking pair that divides a traveling lane (LN1) of a vehicle (10, 10a).
A candidate extraction unit (111) that extracts a plurality of lane marking candidates (LC1 to LC5, LC11 to LC16) using an image taken by a camera (130) mounted on the vehicle, and a candidate extraction unit (111).
A pair of the lane marking candidates that are in front of the vehicle and lined up in the vehicle width direction are extracted as the traveling lane marking pair candidates that are candidates for the traveling lane marking pair, and the lane width of the traveling lane marking pair candidate ( Using W1, W2) and the learning lane width (Ws) learned from the lane width of the traveling lane marking pair recognized in the past, the traveling lane marking pair candidate is the traveling lane marking pair. The selection execution unit (112), which calculates the reliability of the road markings and selects the traveling lane marking pair candidate satisfying the first condition including the reliability satisfying a predetermined criterion as the traveling lane marking pair,
A learning unit (114) that calculates the learning lane width by learning the lane width of the traveling lane pair selected by the selection execution unit.
A scene determination unit for determining whether or not the scene appearing in the image taken by the camera is the first scene which is the connection end point (A1) between the traveling lane (LN1) and the branch lane (LN2) of the vehicle. (113) and
In the case of the first scene, the learning unit reduces the degree of reflection of the lane width of the traveling lane pair in the learning result as compared with the case of not the first scene.
Recognition device. The recognition device according to claim 1.
When the scene appearing in the image is the first scene, the learning unit weights the learning of the lane width of the traveling lane marking pair selected by the selection execution unit as compared with the case where the scene is not the first scene. A recognition device that lowers. The recognition device according to claim 1.
The learning unit
When the scene appearing in the image is not the first scene, the lane width of the traveling lane marking pair selected by the selection execution unit is learned.
When the scene appearing in the image is the first scene, the recognition device does not learn the lane width of the traveling lane marking pair selected by the selection execution unit. The recognition device according to any one of claims 1 to 3.
When the lane marking candidate satisfying the second condition including forming a predetermined first shape is extracted from the scene determination unit in front of the vehicle and on one side in the vehicle width direction. , It is determined that the scene appearing in the image is the first scene,
The first shape is a recognition device in which the two lane marking candidates are separated from each other toward the front of the vehicle and are connected to each other on the side closer to the vehicle. The recognition device according to any one of claims 1 to 3.
In the scene determination unit, when the lane marking candidate satisfying a predetermined third condition is extracted on one side in the vehicle width direction in front of the vehicle, the scene appearing in the image is described. Judging that it is the first scene,
In the third condition, two first pairs (LC3, LC4), which are two lane marking candidates that are separated from each other toward the front of the vehicle, are extracted, and the vehicle is ahead of the first pair and is the vehicle. A recognition device comprising further extracting a separation lane marking candidate (LC5), which is a lane marking candidate separated from the first pair, on the other side in the width direction. The recognition device according to any one of claims 1 to 5.
The first condition includes that the lane marking candidate included in the traveling lane marking pair candidate is not the lane marking candidate constituting the first scene.
When the scene appearing in the image is the first scene, the selection execution unit includes the lane marking candidate (LC1) extracted on the other side of the vehicle in the vehicle width direction and the vehicle of the vehicle. The traveling lane line pair consisting of a line segment (LCx) obtained by moving the lane marking candidate extracted on the other side in the width direction to one side in the vehicle width direction of the vehicle by a distance corresponding to the learning lane width. A recognition device that selects candidates (LC1, LCx) as the traveling lane marking pair. The recognition device according to claim 5.
When the scene appearing in the image is the first scene and the separation lane marking candidates are further extracted, the selection execution unit has the separation lane marking candidates and the vehicle width direction of the vehicle. The reference for the traveling lane marking pair candidate (LC1, LC5) including the lane marking candidate (LC1) extracted on the other side of the above is used as the reference for the other traveling lane marking pair candidate (LC1, LC3). A recognition device that relaxes more than the standard. The recognition device according to claim 7.
The criteria include that the reliability of the traveling lane marking pair candidate is higher than that of the other traveling lane marking pair candidates for a predetermined first period.
The selection execution unit shortens the first period for the traveling lane marking pair candidate including the separation lane marking candidate and the lane marking candidate extracted on the other side in the vehicle width direction of the vehicle. A recognition device that relaxes the above criteria. The recognition device according to any one of claims 1 to 8.
Notification control for causing the notification device (250) provided in the vehicle to notify that the lane marking on one side of the vehicle in the vehicle width direction is not recognized when the scene appearing in the image is the first scene. A recognition device further comprising a unit (115). A recognition device (105, 105a) that recognizes a traveling lane marking pair that divides a traveling lane (LN1) of a vehicle (10, 10a).
A candidate extraction unit (111) that extracts a plurality of lane marking candidates (LC1 to LC5, LC11 to LC16) using an image taken by a camera (130) mounted on the vehicle, and a candidate extraction unit (111).
A pair of the lane marking candidates that are in front of the vehicle and lined up in the vehicle width direction are extracted as the traveling lane marking pair candidates that are candidates for the traveling lane marking pair, and the lane width of the traveling lane marking pair candidate ( Using W1, W2) and the learning lane width (Ws) learned from the lane width of the traveling lane marking pair recognized in the past, the traveling lane marking pair candidate is the traveling lane marking pair. The selection execution unit (112), which calculates the reliability of the road markings and selects the traveling lane marking pair candidate satisfying the first condition including the reliability satisfying a predetermined criterion as the traveling lane marking pair,
A learning unit (114) that calculates the learning lane width by learning the lane width of the traveling lane pair selected by the selection execution unit.
A scene determination unit (113) for determining whether or not the scene appearing in the image taken by the camera is the first scene which is the connection end point (A1) between the traveling lane and the branch lane (LN2) of the vehicle. And with
The scene determination unit
When the lane marking candidate satisfying a predetermined third condition is extracted on one side in the vehicle width direction in front of the vehicle, the scene appearing in the image is the first scene. Judge,
In the third condition, two first pairs (LC3, LC4), which are two lane marking candidates that are separated from each other toward the front of the vehicle, are extracted, and the vehicle is ahead of the first pair and is the vehicle. On the other side in the width direction, the separation lane marking candidate (LC5), which is the lane marking candidate separated from the first pair, is further extracted.
When the scene appearing in the image is the first scene and the separation lane marking candidates are further extracted, the selection execution unit has the separation lane marking candidates and the vehicle width direction of the vehicle. The criteria for the traveling lane marking pair candidates (LC1, LC5) including the lane marking candidate (LC1) extracted on the other side of the above are relaxed more than the criteria for the other traveling lane marking pair candidates. ,
Recognition device. It is a method of recognizing a traveling lane marking pair that divides a traveling lane (LN1) of a vehicle (10, 10a).
A step of extracting a plurality of lane marking candidates (LC1 to LC5, LC11 to LC16) using an image taken by a camera (130) mounted on the vehicle and (S20).
A pair of the lane marking candidates that are in front of the vehicle and lined up in the vehicle width direction are extracted as the traveling lane marking pair candidates that are candidates for the traveling lane marking pair, and the lane width of the traveling lane marking pair candidate ( Using W1, W2) and the learning lane width (Ws) learned from the lane width of the traveling lane marking pair recognized in the past, the traveling lane marking pair candidate is the traveling lane marking pair. The step of calculating the reliability of the road markings and selecting the traveling lane marking pair candidate satisfying the first condition including the reliability satisfying a predetermined criterion as the traveling lane marking pair (S40).
A step of calculating the learning lane width by learning the lane width of the selected traveling lane pair (S60).
A step of determining whether or not the scene appearing in the image taken by the camera is the first scene which is the connection end point (A1) between the traveling lane (LN1) and the branch lane (LN2) of the vehicle. S30), equipped with
In the step of calculating the learning lane width, in the case of the first scene, the degree of reflection of the lane width of the traveling lane in the learning result is reduced as compared with the case of not the first scene. It is a method of recognizing a traveling lane marking pair that divides a traveling lane (LN1) of a vehicle (10, 10a).
A step of extracting a plurality of lane marking candidates (LC1 to LC5, LC11 to LC16) using an image taken by a camera (130) mounted on a vehicle and (S20).
A pair of the lane marking candidates that are in front of the vehicle and lined up in the vehicle width direction are extracted as lane marking pair candidates that are candidates for the traveling lane marking pair, and are combined with the lane width of the traveling lane marking pair candidate. , The learning lane width (Ws) learned from the lane width of the traveling lane marking pair recognized in the past is used to calculate the reliability that the traveling lane marking pair candidate is the traveling lane marking pair. Then, in the step of selecting the traveling lane marking pair that satisfies the first condition including the reliability satisfying a predetermined criterion as the traveling lane marking pair (S40).
A step of calculating the learning lane width by learning the lane width of the selected traveling lane pair (S60).
A step of determining whether or not the scene appearing in the image taken by the camera is the first scene which is the connection end point (A1) between the traveling lane and the branch lane (LN2) of the vehicle (S30). With
In the determination step,
When the lane marking candidate satisfying a predetermined third condition is extracted on one side in the vehicle width direction in front of the vehicle, the scene appearing in the image is the first scene. Judge,
In the third condition, two first pairs (LC3, LC4), which are two lane marking candidates that are separated from each other toward the front of the vehicle, are extracted, and the vehicle is ahead of the first pair and is the vehicle. On the other side in the width direction, the separation lane marking candidate (LC5), which is the lane marking candidate separated from the first pair, is further extracted.
In the selection step, when the scene appearing in the image is the first scene and the separation lane marking candidate is further extracted, the separation lane marking candidate and the vehicle width direction of the vehicle A method in which the criteria for the traveling lane marking pair candidates (LC1, LC5) including the lane marking candidate extracted on the other side of the above are relaxed more than the criteria for the other traveling lane marking pair candidates.

Images