JP2019194751A - Lane information processing device, lane identification system, lane information processing method and lane information processing program - Google Patents

Abstract

To more accurately discriminate the lane on which a vehicle is located.SOLUTION: A lane information processing device comprises: an acquisition part which acquires measured information based on a measurement result of a vehicle on a road; and an output part which outputs an identification result based on the measured information acquired by the acquisition part by using an identification model which enables output of identification result on what number of lane from the right the car is on as well as from the left among multiple lanes in the same direction on the road.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a lane information processing apparatus, a lane identification system, a lane information processing method, and a lane information processing program.

  Conventionally, a technique for discriminating a lane in which a vehicle is traveling has been developed based on a line separating the lanes. For example, Patent Document 1 (Japanese Patent Laid-Open No. 2008-276642) discloses the following technique. That is, the traveling lane recognition device includes a detecting means for detecting the current location of the vehicle, a map data storing means for storing the number of lanes on the road, a current location of the vehicle detected by the detecting means, and the road stored by the map data storing means. Lane number detecting means for detecting the number of lanes of the road on which the vehicle is traveling based on the number of lanes of the vehicle, boundary line recognizing means mounted on the vehicle and recognizing the boundary line of the traveling lane, Travel lane recognition means for recognizing the travel lane of the vehicle based on the number of lanes detected by the lane number detection means and the boundary line recognized by the boundary line recognition means.

  Patent Document 2 (Japanese Patent Laid-Open No. 2005-4442) discloses the following technique. That is, the travel lane discrimination device is a travel lane discrimination device that can be connected to at least one imaging device that generates a road surface image representing a road surface of a road on which the vehicle travels, and is an image that acquires a road surface image from the imaging device. An acquisition unit, a partial image creation unit that creates a partial image by cutting out a region that can be assumed in advance as having no obstacle on the road surface from the road surface image acquired by the image acquisition unit, and the partial image creation unit A lane line extraction unit that extracts partial lane lines included in the created partial image, and is drawn on both sides of the driving lane of the vehicle based on the partial lane lines extracted by the lane line extraction unit. A lane line discriminating unit that discriminates the characteristics of the lane line that is present, and a travel lane discriminating unit that discriminates the type of the lane of the vehicle based on the lane line characteristics discriminated by the lane line discriminating unit.

JP 2008-276642 A JP 2005-4442 A

Ramprasath R. Selvaraju, 5 others, “Grad-CAM: Visual Expansions from Deep Networks via Gradient-Based Localization”, arXiv: 1610.02391v3, May 2017

  Beyond the techniques described in Patent Document 1 and Patent Document 2, a technique for more accurately discriminating the lane in which the vehicle is located is desired.

  This invention was made in order to solve the above-mentioned subject, The objective is the lane information processing apparatus, lane identification system, lane information processing method, and the lane information processing apparatus which can discriminate | determine more precisely the lane where a vehicle is located It is to provide a lane information processing program.

  (1) In order to solve the above-described problem, a lane information processing apparatus according to an aspect of the present invention is the same as an acquisition unit that acquires measurement information based on measurement results of the vehicle on the road where the vehicle is located. Acquired by the acquisition unit using an identification model that can output as an identification result what number lane from the leftmost lane and what number from the rightmost lane among a plurality of lanes in the direction An output unit that outputs the identification result based on the measured information.

  (7) In order to solve the above-described problem, a lane identification system according to an aspect of the present invention includes an on-vehicle device mounted on a vehicle and a lane information processing device, and the on-vehicle device is located on the vehicle. The measurement information based on the measurement result of the vehicle on the road is acquired, the acquired measurement information is transmitted to the lane information processing device, and the lane information processing device receives the measurement information transmitted from the in-vehicle device. Using the identification model that can output the number of lanes from the leftmost lane and the number of lanes from the rightmost lane among the multiple lanes in the same direction on the road, The identification result based on the received measurement information is acquired, and the lane of the road on which the vehicle is located is determined based on the acquired identification result.

  (8) In order to solve the above-described problem, a lane information processing method according to an aspect of the present invention is a lane information processing method in a lane information processing apparatus, and is based on a measurement result in the vehicle on a road on which the vehicle is located. It is possible to output the identification result of the step of obtaining measurement information and the number of lanes from the leftmost lane and the number of lanes from the rightmost lane among multiple lanes in the same direction on the road Outputting the identification result based on the acquired measurement information using a simple identification model.

  (9) In order to solve the above-described problem, a lane information processing program according to an aspect of the present invention is a lane information processing program used in a lane information processing apparatus, and the computer is connected to the vehicle on the road where the vehicle is located. The acquisition unit that acquires measurement information based on the measurement results in and the number of lanes from the leftmost lane and the number of lanes from the rightmost lane among multiple lanes in the same direction on the road This is a program for functioning as an output unit that outputs the identification result based on the measurement information acquired by the acquisition unit using an identification model that can be output as an identification result.

  The present invention can be realized not only as a lane information processing apparatus including such a characteristic processing unit, but also as a semiconductor integrated circuit that realizes part or all of the lane information processing apparatus.

  The present invention can be realized not only as a lane identification system including such a characteristic processing unit, but also as a method using such characteristic processing as a step, or for causing a computer to execute such a step. Or as a program. Further, it can be realized as a semiconductor integrated circuit that realizes part or all of the lane identification system.

  According to the present invention, it is possible to more accurately determine the lane in which the vehicle is located.

FIG. 1 is a diagram showing a configuration of a lane identification system according to an embodiment of the present invention. FIG. 2 is a diagram showing a configuration of the lane information processing apparatus in the lane identification system according to the embodiment of the present invention. FIG. 3 is a diagram showing an example of an image taken by the camera in the lane identification system according to the embodiment of the present invention. FIG. 4 is a diagram illustrating an example of an image photographed by the camera in the lane identification system according to the embodiment of the present invention. FIG. 5 is a diagram showing an example of an image taken by the camera in the lane identification system according to the embodiment of the present invention. FIG. 6 is a diagram illustrating an example of an image photographed by the camera in the lane identification system according to the embodiment of the present invention. FIG. 7 is a diagram showing an example of an image photographed by the camera in the lane identification system according to the embodiment of the present invention. FIG. 8 is a diagram showing an example of a label used in the lane identification system according to the embodiment of the present invention. FIG. 9 is a flowchart defining an operation procedure when the lane information processing apparatus in the lane identification system according to the embodiment of the present invention creates an identification model. FIG. 10 is a flowchart that defines an operation procedure when the lane information processing device in the lane identification system according to the embodiment of the present invention determines the lane of the road on which the vehicle is located.

  First, the contents of the embodiment of the present invention will be listed and described.

  (1) A lane information processing apparatus according to an embodiment of the present invention includes an acquisition unit that acquires measurement information based on measurement results of the road on which the vehicle is located, and a plurality of lanes in the same direction aligned on the road Based on the measurement information acquired by the acquisition unit using an identification model capable of outputting as an identification result what number lane is from the leftmost lane and what number is the lane from the rightmost lane An output unit for outputting the identification result.

  With this configuration, when creating an identification model by machine learning, the number of lanes can be determined by using the number of lanes from the leftmost lane and the number of lanes from the rightmost lane as identification results. Machine learning can be converged without creating a plurality of identification models according to the differences. And the structure using such an identification model can discriminate | determine the lane where a vehicle is located also in the situation where it is difficult to recognize the line which separates a lane on a road. Therefore, it is possible to more accurately determine the lane in which the vehicle is located.

  (2) Preferably, the measurement information includes the measurement result regarding at least one of a leftmost lane and a rightmost lane among a plurality of lanes arranged in the same direction on the road.

  With such a configuration, it is possible to perform labeling focusing on features that appear on the lane side at the end of the road in the measurement result by using the identification model focusing on both ends of the road to determine the lane, Road measurement results can be classified well.

  (3) Preferably, the said lane information processing apparatus is further provided with the discrimination | determination part which discriminate | determines the lane of the road where the said vehicle is located based on the said identification result output by the said output part.

  With such a configuration, it is possible to obtain a more accurate discrimination result of the lane in which the vehicle is located using the discrimination result based on the discrimination model as described above.

  (4) Preferably, the identification model is an identification model created by machine learning according to deep learning.

  With such a configuration, it is possible to more accurately determine the lane in which the vehicle is located using the deep learning model.

  (5) Preferably, the said measurement result is a detection result by the image by a camera, or LiDAR (Light Detection and Ranging).

  With such a configuration, it is possible to determine the lane in which the vehicle is located with a simple configuration using an image from the camera or a detection result by LiDAR.

  (6) Preferably, the road is a road including three or more lanes on one side.

  With such a configuration, it is possible to more accurately determine the lane in which the vehicle is located, particularly on a road having three or more lanes.

  (7) A lane identification system according to an embodiment of the present invention includes an in-vehicle device mounted on a vehicle and a lane information processing device, and the in-vehicle device is a measurement result in the vehicle on a road where the vehicle is located. The measurement information based on is acquired, and the acquired measurement information is transmitted to the lane information processing device. The lane information processing device receives the measurement information transmitted from the in-vehicle device and Based on the received measurement information, using an identification model that can output as an identification result what number lane is from the leftmost lane and what number is the lane from the rightmost lane among a plurality of lanes The identification result is acquired, and the lane of the road on which the vehicle is located is determined based on the acquired identification result.

  With this configuration, when creating an identification model by machine learning, the number of lanes can be determined by using the number of lanes from the leftmost lane and the number of lanes from the rightmost lane as identification results. Machine learning can be converged without creating a plurality of identification models according to the differences. And the structure using such an identification model can discriminate | determine the lane where a vehicle is located also in the situation where it is difficult to recognize the line which separates a lane on a road. Therefore, it is possible to more accurately determine the lane in which the vehicle is located.

  (8) A lane information processing method according to an embodiment of the present invention is a lane information processing method in a lane information processing apparatus, and a step of acquiring measurement information based on a measurement result in the vehicle on a road where the vehicle is located; Using the identification model that can output the number of lanes from the leftmost lane and the number of lanes from the rightmost lane among the multiple lanes in the same direction on the road, Outputting the identification result based on the acquired measurement information.

  With this configuration, when creating an identification model by machine learning, the number of lanes can be determined by using the number of lanes from the leftmost lane and the number of lanes from the rightmost lane as identification results. Machine learning can be converged without creating a plurality of identification models according to the differences. And the structure using such an identification model can discriminate | determine the lane where a vehicle is located also in the situation where it is difficult to recognize the line which separates a lane on a road. Therefore, it is possible to more accurately determine the lane in which the vehicle is located.

  (9) A lane information processing program according to an embodiment of the present invention is a lane information processing program used in a lane information processing apparatus, and uses a measurement information based on a measurement result of the vehicle on a road where the vehicle is located. And the number of lanes from the leftmost lane and the number of lanes from the rightmost lane among the multiple lanes in the same direction on the road can be output as identification results It is a program for functioning as an output unit that outputs the identification result based on the measurement information acquired by the acquisition unit using an identification model.

  With this configuration, when creating an identification model by machine learning, the number of lanes can be determined by using the number of lanes from the leftmost lane and the number of lanes from the rightmost lane as identification results. Machine learning can be converged without creating a plurality of identification models according to the differences. And the structure using such an identification model can discriminate | determine the lane where a vehicle is located also in the situation where it is difficult to recognize the line which separates a lane on a road. Therefore, it is possible to more accurately determine the lane in which the vehicle is located.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals and description thereof will not be repeated. Moreover, you may combine arbitrarily at least one part of embodiment described below.

[Configuration and basic operation]
FIG. 1 is a diagram showing a configuration of a lane identification system according to an embodiment of the present invention.

  With reference to FIG. 1, the lane identification system 301 includes an in-vehicle device 11 and a lane information processing device 101.

  The in-vehicle device 11, the camera 12, and the LiDAR 13 are mounted on the vehicle 1.

  The in-vehicle device 11 transmits measurement information based on the measurement result of the road on which the vehicle-mounted device 11 is located to the lane information processing device 101.

  For example, the measurement information includes a measurement result regarding at least one of the leftmost lane and the rightmost lane among a plurality of lanes in the same direction arranged on the road. The measurement result is, for example, an image by the camera 12 or a detection result by the LiDAR 13. In this specification, the “leftmost lane” means a leftmost lane among a plurality of lanes arranged in the same direction on the road, and the “rightmost lane” means a plurality of lanes arranged in the same direction on the road. Of these, it means the rightmost lane.

  More specifically, the camera 12 is an imaging device that periodically images the surroundings of the vehicle 1 while the vehicle 1 is traveling and the vehicle 1 is stopped, for example. , Including a CPU and processing circuit equipped with corresponding software.

  The camera 12 is an image obtained by photographing the front of the vehicle 1 including a left lane and a right lane among a plurality of lanes arranged in the same direction on the road, and walls, buildings, plants, and the like outside the lane. (Hereinafter also referred to as a captured image) is output to the in-vehicle device 11.

  The LiDAR 13 includes, for example, a laser device for irradiating laser light, a light receiving element for receiving scattered light of laser light from an object, a CPU (Central Processing Unit) equipped with corresponding software, a processing circuit, and the like.

  One or a plurality of LiDARs 13 are provided in the own vehicle 1. For example, when the vehicle 1 is traveling and the vehicle 1 is stopped, the LiDAR 13 detects objects located on the left side, the right side, the front side, and the rear side with respect to the vehicle 1. Is possible. The LiDAR 13 periodically outputs the detection result to the in-vehicle device 11.

  The detection results include objects in the vicinity of the vehicle 1 such as at least one of the leftmost lane and the rightmost lane among a plurality of lanes arranged in the same direction on the road, and walls, buildings, and plants outside the lane. included.

  The in-vehicle device 11 is a communication device used in telematics, for example, and performs communication by LTE (Long Term Evolution) or wireless LAN.

  The in-vehicle device 11 transmits the measurement information including the captured image received from the camera 12 and the detection result received from the LiDAR 13 and added with the ID (Identification) of the vehicle 1 to the lane information processing device 101.

[Task]
Patent Document 1 (Japanese Patent Laid-Open No. 2008-276642) discloses a method of identifying a traveling lane based on a white line pattern on a road in an image taken by a camera or the like.

  However, according to this method, the traveling lane cannot be specified unless the road has a known white line pattern. Also, with this method, when the white line is buried due to snow or the like and it is difficult to recognize the white line pattern, or when the shape of the white line changes due to secular change or the like, the white line pattern is mistakenly identified and the wrong driving lane is specified. there is a possibility. For example, in the case of a four-lane road on one side, if the pattern of the white line in the center two lanes is the same, it cannot be determined which lane the vehicle is traveling.

  Patent Document 2 (Japanese Patent Application Laid-Open No. 2005-4442) discloses a method of identifying a traveling lane using machine learning.

  However, in this method, since it is necessary to create an identification model according to the number of lanes to travel, the number of identification models increases and it is difficult to converge machine learning.

  On the other hand, in the lane identification system according to the embodiment of the present invention, the above problem is solved by the following configuration and operation.

[Configuration of lane information processing device]
FIG. 2 is a diagram showing a configuration of the lane information processing apparatus in the lane identification system according to the embodiment of the present invention.

  With reference to FIG. 2, the lane information processing apparatus 101 includes a reception unit 21, an acquisition unit 22, an output unit 23, a determination unit 24, a storage unit 25, and a generation unit 26.

  The receiving unit 21 receives measurement information transmitted from the in-vehicle device 11 mounted on one or a plurality of vehicles 1 and stores it in the storage unit 25.

  3-7 is a figure which shows an example of the image image | photographed with the camera in the lane identification system which concerns on embodiment of this invention.

  The image shown in FIG. 3 is a captured image S21 in which the camera 12 captures the front of the vehicle 1 in a state where the vehicle 1 is traveling in the leftmost lane on a two-lane road. The storage unit 25 stores a plurality of such photographed images S21.

  The image shown in FIG. 4 is a captured image S22 in which the camera 12 captures the front of the vehicle 1 in a state where the vehicle 1 is traveling in the rightmost lane on a two-lane road. A plurality of such photographed images S22 are stored in the storage unit 25.

  The image shown in FIG. 5 is a captured image S31 in which the camera 12 captures the front of the vehicle 1 in a state where the vehicle 1 is traveling in the leftmost lane on a road with three lanes on one side. The storage unit 25 stores a plurality of such photographed images S31.

  The image shown in FIG. 6 is a captured image S32 in which the camera 12 captures the front of the vehicle 1 in a state in which the vehicle 1 is traveling in the center lane on a three-lane road on one side. A plurality of such photographed images S32 are stored in the storage unit 25.

  The image shown in FIG. 7 is a captured image S33 in which the camera 12 captures the front of the vehicle 1 in a state in which the vehicle 1 is traveling in the rightmost lane on a three-lane road on one side. A plurality of such photographed images S33 are stored in the storage unit 25.

  Further, in the storage unit 25, for example, in a state where the vehicle 1 is traveling on a one-lane road, the camera 12 captures a plurality of captured images of the front of the vehicle 1, and the vehicle 1 has four or more lanes on one side. A plurality of captured images in which the camera 12 captures the front of the vehicle 1 in a state where the vehicle is traveling on each lane on the road are stored.

  Further, the storage unit 25 stores, for example, detection results by a plurality of LiDARs 13. The detection result by the LiDAR 13 is, for example, three-dimensional data in which depth information is added to the images illustrated in FIGS. 3 to 7 and can be handled in the same manner as the captured image.

  The generation unit 26 creates, for example, an identification model that is a model generated by machine learning according to deep learning. The identification model is an algorithm automatically created by machine learning, for example. The deep learning model is an example of a neural network model.

  Here, Non-Patent Document 1 (Ramprasaath R. Selvaraju, 5 others, “Grad-CAM: Visual Expansions from Deep Networks via Gradient-Based Localization”, 17-91, arXiv: 161, 1961, arXiv: 160, 1953. By using CAM (Gradient-Weighted Class Activation Mapping), it is possible to visualize where in the deep learning model the focused image is focused.

  The inventors of the present application have found that when a photographed image obtained by photographing the front of the vehicle as shown in FIGS. 3 to 7 is input to the deep learning model, the deep learning model focuses on both ends of the road.

  Then, the inventors of the present invention use a label indicating the number of the lane from the leftmost lane and the number of the lane from the rightmost lane among the plurality of lanes arranged in the same direction on the road. The idea is that the lane in which the vehicle 1 is located can be easily classified.

  Referring to FIGS. 3 and 5, in the vicinity of the left end of the captured image S21 and the captured image S31 captured in the state where the vehicle 1 is traveling in the left lane, outside the lane in which the vehicle 1 is traveling. The vertical ratio of the existing walls is about one third.

  Even if the number of lanes on the road is different, the distance from the lane in which the vehicle 1 is traveling to the wall does not change, so the vertical direction of the wall that appears near the left end of each captured image on a road with a different number of lanes The ratio is almost the same.

  Similarly, referring to FIG. 4 and FIG. 7, in the vicinity of the right end of the captured image S22 and the captured image S33 captured in a state where the vehicle 1 is traveling in the rightmost lane, The ratio of the vertical direction of the wall existing outside the lane is about one third, and the ratio is substantially the same in the captured image S22 and the captured image S33.

  In this way, even if the number of lanes on the road is different, paying attention to the fact that the ratio in the vertical direction is substantially the same, the captured images are classified using the labels indicating the lane where the vehicle 1 is located.

  Here, for example, when only the label indicating the number of the lane from the leftmost lane is used, the rightmost lane on the road including the lane of two lanes on one side and the center of the road including the lane of three lanes on one side are used. Indistinguishable from lane.

  On the other hand, by using the label as described above, the lane in which the vehicle 1 is located can be easily determined even on a road with three or more lanes on one side.

  FIG. 8 is a diagram showing an example of a label used in the lane identification system according to the embodiment of the present invention.

  Referring to FIG. 8, in lane identification system 301, a label indicating the lane in which vehicle 1 is located is attached to the captured image as follows.

  That is, when the vehicle 1 is traveling on a one-lane road, a label L1 indicating that the vehicle 1 is located in the first lane from the leftmost lane is attached to the photographed image. Alternatively, a label R1 indicating that the vehicle is located in the first lane from the rightmost lane is attached to the captured image.

  Further, when the vehicle 1 is traveling in the first lane at the left end on a two-lane road on one side, a label L1 indicating that the vehicle 1 is located in the first lane from the left lane is attached to the photographed image.

  In addition, when the vehicle 1 is traveling in the second lane at the right end on a two-lane road on one side, a label R1 indicating that the vehicle 1 is located in the first lane from the right lane is attached to the photographed image.

  Further, when the vehicle 1 is traveling on the leftmost first lane on a three-lane road on one side, a label L1 indicating that the vehicle 1 is located in the first lane from the leftmost lane is attached to the photographed image.

  In addition, when the vehicle 1 is traveling in the center second lane on a three-lane road on one side, a label L2 indicating that the vehicle 1 is located in the second lane from the leftmost lane is attached to the photographed image. Alternatively, a label R2 indicating that the vehicle is located in the second lane from the rightmost lane is attached to the captured image.

  Further, when the vehicle 1 is traveling in the third lane on the right end on the three-lane road on one side, a label R1 indicating that the vehicle 1 is located in the first lane from the right lane is attached to the photographed image.

  Further, when the vehicle 1 is traveling on the leftmost first lane on a four-lane road on one side, a label L1 indicating that the vehicle 1 is located in the first lane from the leftmost lane is attached to the photographed image.

  Further, when the vehicle 1 is traveling in the second lane second from the left lane on the four-lane road on one side, a label L2 indicating that the vehicle 1 is located in the second lane from the left lane is attached to the photographed image. The

  Further, when the vehicle 1 is traveling on the second lane from the right lane on the four-lane road on one side, a label R2 indicating that the vehicle 1 is located in the second lane from the right lane is attached to the photographed image. The

  Further, when the vehicle 1 is traveling in the fourth lane at the right end on a four-lane road on one side, a label R1 indicating that the vehicle 1 is located in the first lane from the right lane is attached to the photographed image.

[Flow of operation]
Each device in the lane identification system 301 includes a computer, and an arithmetic processing unit such as a CPU in the computer reads and executes a program including a part or all of each step of the following flowchart from a memory (not shown). Each of the programs of the plurality of apparatuses can be installed from the outside. The programs of the plurality of apparatuses are distributed while being stored in a recording medium.

  FIG. 9 is a flowchart defining an operation procedure when the lane information processing apparatus in the lane identification system according to the embodiment of the present invention creates an identification model.

  Referring to FIG. 9, first, generation unit 26 creates image data PL1 in accordance with a user operation.

  More specifically, the user, for example, in order to create an identification model, a captured image in a state where the vehicle 1 is traveling in the first lane from the left end among a plurality of images stored in the storage unit 25, that is, a captured image. A plurality of, for example, 500 images S21 and photographed images S31 (see FIGS. 3 and 5) are selected, and an operation for inputting a selection result is performed on the lane information processing apparatus 101.

  The selected 500 captured images include a captured image in a state where the vehicle 1 is traveling on a road with one lane on one side, and a vehicle 1 is traveling on the leftmost lane on a road with four or more lanes on one side. At least one of the captured images in the state may be included.

  The generation unit 26 creates image data PL1 including 500 selected photographed images (step S101).

  Next, the production | generation part 26 produces the image data PR1 according to a user's operation.

  More specifically, the user, for example, in order to create an identification model, among the plurality of captured images stored in the storage unit 25, the captured image in a state where the vehicle 1 is traveling in the first lane from the right end, that is, A plurality of, for example, 500 photographed images S22 and photographed images S33 (see FIGS. 4 and 7) are selected, and an operation for inputting a selection result is performed on the lane information processing apparatus 101.

  In the selected 500 captured images, a captured image in a state where the vehicle 1 is traveling on a road with one lane on one side, and a vehicle 1 is traveling in the rightmost lane on a road with four or more lanes on one side. At least one of the captured images in the state may be included.

  The generation unit 26 creates image data PR1 including 500 selected photographed images (step S102).

  Next, the production | generation part 26 produces the image data PL2 according to a user's operation.

  More specifically, the user, for example, in order to create an identification model, among the plurality of captured images stored in the storage unit 25, the captured image in a state where the vehicle 1 is traveling in the second lane from the left end, that is, A plurality of, for example, 500 photographed images S32 (see FIG. 6) are selected, and an operation for inputting a selection result is performed on the lane information processing apparatus 101.

  Note that the selected 500 captured images may include captured images in a state where the vehicle 1 is traveling in the second lane from the leftmost lane on a road with four or more lanes on one side.

  The generation unit 26 creates image data PL2 including 500 selected photographed images (step S103).

  Next, the production | generation part 26 produces the image data PR2 according to a user's operation.

  More specifically, the user, for example, in order to create an identification model, a captured image in a state where the vehicle 1 is traveling in the second lane from the right end among a plurality of images stored in the storage unit 25, that is, a captured image. A plurality of, for example, 500 images S32 (see FIG. 6) are selected, and an operation for inputting the selection result is performed on the lane information processing apparatus 101.

  The selected 500 images may include photographed images in a state where the vehicle 1 is traveling in the second lane from the rightmost lane on a road with four or more lanes on one side.

  The generation unit 26 creates image data PR2 including 500 selected photographed images (step S104).

  Next, the generation unit 26 creates a learning data set.

  More specifically, the generation unit 26 acquires one captured image from the image data PL1 including 500 images, and applies the label L1 indicating that the vehicle 1 is located in the first lane from the left end. Attached to the shot image.

  The generation unit 26 creates learning data by processing the other captured images in the image data PL1 in the same manner. In this way, 500 pieces of learning data are created for the captured image in a state where the vehicle 1 is positioned in the first lane from the left end.

  Further, the generation unit 26 acquires one captured image from the image data PR1 including 500 captured images, and labels the captured image R1 indicating that the vehicle 1 is located in the first lane from the right end. Attached.

  The generation unit 26 creates learning data by processing the other captured images in the image data PR1 in the same manner. In this way, 500 pieces of learning data are created for the captured image in a state where the vehicle 1 is positioned in the first lane from the right end.

  Further, the generation unit 26 acquires one captured image from the image data PL2 including 500 captured images, and displays a label L2 indicating that the vehicle 1 is located in the second lane from the left end. Attached.

  The generation unit 26 creates learning data by processing the other captured images in the image data PL2 in the same manner. In this way, 500 pieces of learning data are created for the captured image in a state where the vehicle 1 is positioned in the second lane from the left end.

  Further, the generation unit 26 acquires one captured image from the image data PR2 including 500 captured images, and labels the captured image R2 indicating that the vehicle 1 is located in the second lane from the right end. Attached.

  The generation unit 26 generates learning data by processing the other captured images in the image data PR2 in the same manner. In this way, 500 pieces of learning data are created for the captured image in a state where the vehicle 1 is positioned in the second lane from the right end.

  The generation unit 26 creates a learning data set from the generated total 2000 pieces of learning data (step S105).

  Specifically, the generation unit 26 captures each captured image included in the generated 2000 learning data from the leftmost lane among a plurality of lanes in the same direction in which the lane where the vehicle 1 is located is aligned on the road. A learning data set classified for each label indicating the number of lanes and the number of lanes from the rightmost lane is created.

  Next, the production | generation part 26 produces an identification model using the produced learning data set.

  Specifically, the generation unit 26 inputs a learning data set to a multilayered neural network in accordance with, for example, a deep learning technique.

  The generation unit 26 determines, for the multilayered neural network, the number of the lane in which the vehicle 1 is located from the leftmost lane among the plurality of lanes arranged in the same direction on the road, and the rightmost lane. An identification model is created by machine learning so that it can be identified what lane the vehicle is from (step S106).

  As a result of such machine learning, the identification model is obtained from, for example, a plurality of lanes in the same direction in which the lane where the vehicle 1 is located is aligned on the road based on features such as walls appearing at the left and right ends of the road in the image data. Among these, it is possible to output as an identification result which number is the lane from the leftmost lane and what number is the lane from the rightmost lane.

  The content of the identification model is determined when a predetermined condition is satisfied, for example. In this example, the content of the identification model is determined when machine learning using all the captured images included in the learning data set is completed.

  Hereinafter, the identification model for which machine learning has been completed is also referred to as a learned identification model. When image data is input to the learned identification model, the lane in which the vehicle 1 is located is the number of the lane from the leftmost lane among the plurality of lanes arranged in the same direction on the road, and the rightmost lane. The identification result indicating what number of lane is attached to the image data and output.

  The generation unit 26 stores the generated learned identification model in the storage unit 25 (step S107).

  Note that the order of steps S101 to S104 is not limited to the above, and a part or all of the order may be changed.

  FIG. 10 is a flowchart that defines an operation procedure when the lane information processing device in the lane identification system according to the embodiment of the present invention determines the lane of the road on which the vehicle is located.

  With reference to FIG. 10, first, the acquisition unit 22 acquires measurement information based on the measurement result of the vehicle 1 on the road where the vehicle 1 is located (step S201).

  More specifically, the acquisition unit 22 acquires measurement information stored in the storage unit 25, and outputs a captured image (hereinafter also referred to as a determination image) included in the acquired measurement information to the output unit 23.

  Next, the output unit 23 outputs an identification result based on the measurement information acquired by the acquisition unit 22 using the learned identification model (step S202).

  More specifically, the output unit 23 acquires a learned identification model stored in the storage unit 25. And the output part 23 inputs the image for determination from the acquisition part 22 into the said learned identification model, and the lane where the corresponding vehicle 1 is located is leftmost among a plurality of lanes in the same direction lined up on the road. An identification result that alternatively indicates the number of the lane from the lane and the number of the lane from the rightmost lane is attached to the determination image and output to the determination unit 24.

  Specifically, the output unit 23 inputs the determination image to the learned identification model, and thereby the identification result KL1 indicating that the vehicle 1 is positioned in the first lane from the left lane, and the vehicle 1 is the right end. Identification result KR1 indicating that the vehicle is located in the first lane from the lane, identification result KL2 indicating that the vehicle 1 is located in the second lane from the leftmost lane, and vehicle 1 is located in the second lane from the rightmost lane One of the identification results KR2 indicating that this is to be performed is attached to the determination image and output to the determination unit 24.

  For example, the determination unit 24 determines the lane of the road where the corresponding vehicle 1 is located based on the identification result attached to the determination image received from the output unit 23 (step S203).

  More specifically, when receiving the determination image with the identification result KL1 from the output unit 23, the determination unit 24 determines that the vehicle 1 is located in the first lane from the leftmost lane.

  Further, when the determination unit 24 receives the determination image with the identification result KR1 from the output unit 23, the determination unit 24 determines that the vehicle 1 is located in the first lane from the rightmost lane.

  Further, when receiving the determination image with the identification result KL2 from the output unit 23, the determination unit 24 determines that the vehicle 1 is located in the second lane from the leftmost lane.

  Further, when receiving the determination image with the identification result KR2 from the output unit 23, the determination unit 24 determines that the vehicle 1 is located in the second lane from the rightmost lane.

  The lane identification system 301 is not limited to the configuration in which the identification model is generated using the captured image, and the identification model using the detection result by the LiDAR 13 may be generated. In this case, the learning data set includes detection results with labels L1, R1, L2, and R2. The output unit 23 inputs the detection result by the LiDAR 13 received from the acquisition unit 22 to the learned identification model, and outputs the detection result to which any identification result is attached to the determination unit 24. The determination unit 24 determines the lane of the road on which the vehicle 1 is located based on the identification result attached to the detection result received from the output unit 23.

  Further, in the lane identification system according to the embodiment of the present invention, the measurement information includes a measurement result regarding at least one of the leftmost lane and the rightmost lane among the plurality of lanes arranged in the same direction on the road. However, the present invention is not limited to this. The measurement information may include a measurement result including some characteristic that can identify the lane without including the measurement result regarding the left lane and the right lane.

In the lane identification system according to the embodiment of the present invention, the lane information processing apparatus 101 is
Although it was set as the structure provided with the discrimination | determination part 24, it is not limited to this. The lane information processing apparatus 101 may be configured not to include the determination unit 24. In this case, the output unit 23 outputs the identification result to the outside of the lane information processing apparatus 101.

  In the lane identification system according to the embodiment of the present invention, the output unit 23 is configured to use machine learning according to deep learning, but the present invention is not limited to this. The output unit 23 may be configured to use a neural network other than deep learning, for example. Alternatively, the output unit 23 may be a support vector machine, C4.5, CART (Classification and Regression Tree), random forest, bagging, decision tree, naive bayes, adoboost, and k-nearest neighbor algorithm (kNN). A configuration using supervised learning may be used.

  In the lane identification system according to the embodiment of the present invention, the measurement result in the vehicle 1 is an image by a camera or a detection result by LiDAR. However, the present invention is not limited to this. The measurement result in the vehicle 1 may be a measurement result other than the image by the camera and the detection result by LiDAR.

  In the lane identification system according to the embodiment of the present invention, the measurement information includes the image received from the camera 12 and the detection result received from the LiDAR 13, but the present invention is not limited to this. The measurement information may include either an image received from the camera 12 or a detection result received from the LiDAR 13.

  By the way, beyond the technique of patent document 1 and patent document 2, the technique of discriminating the lane where a vehicle is located more correctly is desired.

  In the lane information processing apparatus according to the embodiment of the present invention, the acquisition unit 22 acquires measurement information based on the measurement result of the vehicle 1 on the road where the vehicle 1 is located. The output unit 23 is an identification model that can output, as an identification result, the number of the lane from the leftmost lane and the number of the lane from the rightmost lane among a plurality of lanes arranged in the same direction on the road. The identification result based on the measurement information acquired by the acquisition unit 22 is output.

  With this configuration, when creating an identification model by machine learning, the number of lanes can be determined by using the number of lanes from the leftmost lane and the number of lanes from the rightmost lane as identification results. Machine learning can be converged without creating a plurality of identification models according to the differences. And the structure using such an identification model can discriminate | determine the lane where the vehicle 1 is located also in the situation where it is difficult to recognize the line which separates a lane on a road.

  Therefore, the lane information processing apparatus according to the embodiment of the present invention can more accurately determine the lane in which the vehicle 1 is located.

  In the lane information processing apparatus according to the embodiment of the present invention, the measurement information includes a measurement result regarding at least one of the left-end lane and the right-end lane among a plurality of lanes arranged in the same direction on the road.

  With such a configuration, it is possible to perform labeling focusing on features that appear on the lane side at the end of the road in the measurement result by using the identification model focusing on both ends of the road to determine the lane, Road measurement results can be classified well.

  In the lane information processing apparatus according to the embodiment of the present invention, the determination unit 24 determines the lane of the road on which the vehicle 1 is located based on the identification result output by the output unit 23.

  With such a configuration, it is possible to obtain a more accurate discrimination result of the lane in which the vehicle 1 is located using the discrimination result based on the discrimination model as described above.

  In the lane information processing apparatus according to the embodiment of the present invention, the identification model is an identification model created by machine learning according to deep learning.

  With such a configuration, it is possible to more accurately determine the lane in which the vehicle 1 is located using the deep learning model.

  In the lane information processing apparatus according to the embodiment of the present invention, the measurement result is an image obtained by the camera 12 or a detection result obtained by the LiDAR 13.

  With such a configuration, it is possible to determine the lane in which the vehicle 1 is located with a simple configuration using the image by the camera 12 or the detection result by the LiDAR 13.

  In the lane information processing apparatus according to the embodiment of the present invention, the road is a road including three or more lanes on one side.

  With such a configuration, it is possible to more accurately determine the lane in which the vehicle 1 is located, particularly on a road having three or more lanes.

  In the lane identification system according to the embodiment of the present invention, the in-vehicle device 11 acquires measurement information based on the measurement result of the vehicle 1 on the road where the vehicle 1 is located, and the acquired measurement information is used as the lane information processing device 101. Send to. The lane information processing apparatus 101 receives the measurement information transmitted from the in-vehicle apparatus 11, and among the lanes in the same direction aligned on the road, the lane information processing apparatus 101 indicates the number of the lane from the leftmost lane and the number from the rightmost lane. The identification model based on the received measurement information is acquired using an identification model that can output whether the vehicle is the th lane as the identification result, and the lane of the road on which the vehicle 1 is located is determined based on the acquired identification result. .

  With this configuration, when creating an identification model by machine learning, the number of lanes can be determined by using the number of lanes from the leftmost lane and the number of lanes from the rightmost lane as identification results. Machine learning can be converged without creating a plurality of identification models according to the differences. And the structure using such an identification model can discriminate | determine the lane where the vehicle 1 is located also in the situation where it is difficult to recognize the line which separates a lane on a road.

  Therefore, in the lane identification system according to the embodiment of the present invention, it is possible to more accurately determine the lane in which the vehicle 1 is located.

  In the lane information processing method according to the embodiment of the present invention, first, measurement information based on the measurement result of the vehicle 1 on the road where the vehicle 1 is located is acquired. Next, an identification model is used that can output the number of lanes from the leftmost lane and the number of lanes from the rightmost lane among the multiple lanes in the same direction on the road. The identification result based on the acquired measurement information is output.

  With this configuration, when creating an identification model by machine learning, the number of lanes can be determined by using the number of lanes from the leftmost lane and the number of lanes from the rightmost lane as identification results. Machine learning can be converged without creating a plurality of identification models according to the differences. And the structure using such an identification model can discriminate | determine the lane where the vehicle 1 is located also in the situation where it is difficult to recognize the line which separates a lane on a road.

  Therefore, in the lane information processing method according to the embodiment of the present invention, it is possible to more accurately determine the lane in which the vehicle 1 is located.

  The above embodiment should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

The above description includes the following features.
[Appendix 1]
An acquisition unit for acquiring measurement information based on a measurement result in the vehicle on the road where the vehicle is located;
Among the plurality of lanes in the same direction arranged on the road, using an identification model that can output the number of lanes from the leftmost lane and the number of lanes from the rightmost lane as an identification result, An output unit that outputs the identification result based on the measurement information acquired by the acquisition unit;
The measurement information includes the measurement results related to the leftmost lane and the rightmost lane among a plurality of lanes in the same direction on the road,
The measurement result includes a lane information processing apparatus including a result of measuring the outside of the lane.

[Appendix 2]
An in-vehicle device mounted on the vehicle;
A lane information processing device,
The in-vehicle device acquires measurement information based on a measurement result in the vehicle on a road where the vehicle is located, and transmits the acquired measurement information to the lane information processing device,
The lane information processing device receives the measurement information transmitted from the in-vehicle device, and from among a plurality of lanes arranged in the same direction on the road, the number of lanes from the leftmost lane and the rightmost lane Using an identification model that can be output as an identification result indicating what number lane, based on the identification result based on the received measurement information, determine the lane of the road where the vehicle is located,
The measurement information includes the measurement results related to the leftmost lane and the rightmost lane among a plurality of lanes in the same direction on the road,
The said measurement result is a lane identification system containing the result of having measured the outer side of the lane.

1 vehicle 11 in-vehicle device 12 camera 13 LiDAR
DESCRIPTION OF SYMBOLS 21 Reception part 22 Acquisition part 23 Output part 24 Discrimination part 25 Storage part 26 Generation part 101 Lane information processing apparatus 301 Lane identification system

Claims (9)

An acquisition unit for acquiring measurement information based on a measurement result in the vehicle on the road where the vehicle is located;
Among the plurality of lanes in the same direction arranged on the road, using an identification model that can output the number of lanes from the leftmost lane and the number of lanes from the rightmost lane as an identification result, A lane information processing apparatus comprising: an output unit that outputs the identification result based on the measurement information acquired by the acquisition unit.   2. The lane information processing apparatus according to claim 1, wherein the measurement information includes the measurement result regarding at least one of a leftmost lane and a rightmost lane among a plurality of lanes arranged in the same direction on a road. The lane information processing device further includes:
The lane information processing apparatus according to claim 1, further comprising: a determination unit configured to determine a lane of a road on which the vehicle is located based on the identification result output by the output unit.   The lane information processing device according to any one of claims 1 to 3, wherein the identification model is an identification model created by machine learning according to deep learning.   The lane information processing apparatus according to any one of claims 1 to 4, wherein the measurement result is an image obtained by a camera or a detection result obtained by LiDAR (Light Detection and Ranging).   The lane information processing apparatus according to any one of claims 1 to 5, wherein the road is a road including three or more lanes on one side. An in-vehicle device mounted on the vehicle;
A lane information processing device,
The in-vehicle device acquires measurement information based on a measurement result in the vehicle on a road where the vehicle is located, and transmits the acquired measurement information to the lane information processing device,
The lane information processing device receives the measurement information transmitted from the in-vehicle device, and from among a plurality of lanes arranged in the same direction on the road, the number of lanes from the leftmost lane and the rightmost lane Using an identification model that can output as an identification result what number lane it is, obtain the identification result based on the received measurement information, and based on the obtained identification result, the road on which the vehicle is located A lane identification system that identifies lanes. A lane information processing method in a lane information processing apparatus,
Obtaining measurement information based on measurement results in the vehicle on the road where the vehicle is located;
Acquired by using an identification model that can output the number of lanes from the leftmost lane and the numbered lane from the rightmost lane among multiple lanes in the same direction on the road Outputting the identification result based on the measured information. A lane information processing program used in a lane information processing apparatus,
Computer
An acquisition unit for acquiring measurement information based on a measurement result in the vehicle on the road where the vehicle is located;
Among the plurality of lanes in the same direction arranged on the road, using an identification model that can output the number of lanes from the leftmost lane and the number of lanes from the rightmost lane as an identification result, An output unit that outputs the identification result based on the measurement information acquired by the acquisition unit;
Lane information processing program to function as

Images