WO2021088504A1 - 路口检测、神经网络训练及智能行驶方法、装置和设备 - Google Patents

路口检测、神经网络训练及智能行驶方法、装置和设备 Download PDF

Info

Publication number
WO2021088504A1
WO2021088504A1 PCT/CN2020/114095 CN2020114095W WO2021088504A1 WO 2021088504 A1 WO2021088504 A1 WO 2021088504A1 CN 2020114095 W CN2020114095 W CN 2020114095W WO 2021088504 A1 WO2021088504 A1 WO 2021088504A1
Authority
WO
WIPO (PCT)
Prior art keywords
road
intersection
image
sample image
detection
Prior art date
Application number
PCT/CN2020/114095
Other languages
English (en)
French (fr)
Inventor
程光亮
石建萍
Original Assignee
北京市商汤科技开发有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京市商汤科技开发有限公司 filed Critical 北京市商汤科技开发有限公司
Priority to KR1020217016327A priority Critical patent/KR20210082518A/ko
Priority to JP2021532862A priority patent/JP2022512165A/ja
Publication of WO2021088504A1 publication Critical patent/WO2021088504A1/zh

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/02—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to ambient conditions
    • B60W40/06—Road conditions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
    • B60W30/10—Path keeping
    • B60W30/12—Lane keeping
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
    • B60W30/18—Propelling the vehicle
    • B60W30/18009—Propelling the vehicle related to particular drive situations
    • B60W30/18154—Approaching an intersection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W2050/0001—Details of the control system
    • B60W2050/0002—Automatic control, details of type of controller or control system architecture
    • B60W2050/0004—In digital systems, e.g. discrete-time systems involving sampling
    • B60W2050/0005—Processor details or data handling, e.g. memory registers or chip architecture
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60Y—INDEXING SCHEME RELATING TO ASPECTS CROSS-CUTTING VEHICLE TECHNOLOGY
    • B60Y2300/00—Purposes or special features of road vehicle drive control systems
    • B60Y2300/10—Path keeping
    • B60Y2300/12—Lane keeping
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60Y—INDEXING SCHEME RELATING TO ASPECTS CROSS-CUTTING VEHICLE TECHNOLOGY
    • B60Y2300/00—Purposes or special features of road vehicle drive control systems
    • B60Y2300/18—Propelling the vehicle
    • B60Y2300/18008—Propelling the vehicle related to particular drive situations
    • B60Y2300/18158—Approaching intersection

Definitions

  • This application relates to computer vision processing technology, and relates to but is not limited to an intersection detection, neural network training and intelligent driving method, device, electronic equipment, computer storage medium and computer program.
  • the embodiments of the present application expect to provide a technical solution for intersection detection.
  • the embodiment of the present application provides a method for detecting an intersection, and the method includes:
  • the detection frame of the intersection on the road shown in the road image is determined; the detection frame of the intersection indicates the area of the intersection in the road image, and the detection frame of the intersection is under The frame is on the pavement of the road;
  • the method further includes:
  • the feature map of the road image it is determined that the road shown in the road image does not have an intersection.
  • the determining the distance between the device that collects the road image and the intersection according to the lower border of the detection frame of the intersection includes:
  • the distance between the device that collects the road image and the intersection is obtained .
  • the method is executed by a neural network, the neural network is trained using sample images and the annotation results of the sample images, and the annotation results of the sample images include the intersections on the roads shown in the positive sample images
  • the labeled frame represents the position of the intersection in the positive sample image, and the lower frame of the labeled frame is on the road surface of the road shown in the positive sample image.
  • the embodiment of the present application also provides a neural network training method, including:
  • the labeling result of the sample image is the labeling frame of the intersection on the road shown in the positive sample image, and the labeling frame represents the position of the intersection in the positive sample image , And the lower border of the labeling frame of the intersection on the road shown in the positive sample image is on the road surface of the road shown in the positive sample image.
  • the positive sample image includes a stop line of a road intersection, and the lower border of the label frame of the intersection on the road shown in the positive sample image is aligned with the stop line.
  • the height difference of the labeled frames in the multiple positive sample images containing the same intersection is within a preset range.
  • the labeling result of the sample image includes that there is no labeling frame in the negative sample image .
  • the embodiment of the present application also provides an intelligent driving method, including:
  • intersection detection on the road image according to any one of the foregoing intersection detection methods
  • the driving control of the device is performed according to the distance between the intelligent driving device that collects the road image and the intersection.
  • the embodiment of the present application also provides an intersection detection device, which includes a first extraction module, a detection module, and a first determination module; wherein,
  • the first extraction module is configured to perform feature extraction on a road image to obtain a feature map of the road image
  • the detection module is configured to determine the detection frame of the intersection on the road shown in the road image according to the feature map of the road image; the detection frame of the intersection represents the area of the intersection in the road image, and the intersection The lower border of the detection frame is on the road surface of the road;
  • the first determining module is configured to determine the distance between the device that collects the road image and the intersection according to the lower border of the detection frame of the intersection.
  • the detection module is further configured to determine, according to the feature map of the road image, that the road shown in the road image does not have an intersection.
  • the first determining module is configured to determine the position of the lower frame of the detection frame of the intersection in the road image and the distance between the plane of the road image and the road surface of the road.
  • the position of the lower border of the detection frame of the intersection on the road is determined according to the coordinate conversion relationship of the intersection; the position of the lower border of the detection frame of the intersection on the road is related to the location of the device that collects the road image. According to the position on the road, the distance between the device that collects the road image and the intersection is obtained.
  • the device is implemented based on a neural network, and the neural network is trained using sample images and labeling results of sample images.
  • the labeling results of sample images include the roads shown in the positive sample images.
  • the labeling frame of the intersection of, the labeling frame represents the position of the intersection in the positive sample image, and the lower border of the labeling frame is on the road surface of the road shown in the positive sample image.
  • An embodiment of the present application also provides a neural network training device, the device includes: a second extraction module, a second determination module, and an adjustment module, wherein:
  • the second extraction module is configured to perform feature extraction on a sample image to obtain a feature map of the sample image
  • the second determining module is configured to determine the detection result of the sample image according to the feature map of the sample image
  • An adjustment module configured to adjust the network parameter value of the neural network according to the annotation result of the sample image and the detection result
  • the labeling result of the sample image is the labeling frame of the intersection on the road shown in the positive sample image, and the labeling frame represents the position of the intersection in the positive sample image , And the lower border of the labeling frame of the intersection on the road shown in the positive sample image is on the road surface of the road shown in the positive sample image.
  • the positive sample image includes a stop line of a road intersection, and the lower border of the label frame of the intersection on the road shown in the positive sample image is aligned with the stop line.
  • the height difference of the labeled frames in the multiple positive sample images containing the same intersection is within a preset range.
  • the labeling result of the sample image includes that there is no labeling frame in the negative sample image .
  • the embodiment of the present application also provides an intelligent driving device, the device includes: an acquisition module and a processing module, wherein:
  • An acquisition module configured to acquire road images
  • the processing module is configured to perform intersection detection on the road image according to any one of the foregoing intersection detection methods; and perform driving control on the device according to the distance between the intelligent driving device that collects the road image and the intersection.
  • the embodiment of the present application also provides an electronic device, including a processor and a memory configured to store a computer program that can run on the processor; wherein,
  • the processor is configured to run the computer program to execute any one of the above-mentioned intersection detection methods or any one of the above-mentioned neural network training methods or any one of the above-mentioned intelligent driving test methods.
  • the embodiment of the present application also provides a computer storage medium on which a computer program is stored, and when the computer program is executed by a processor, any one of the above-mentioned intersection detection methods, any one of the above-mentioned neural network training methods, or any one of the above-mentioned methods is implemented Intelligent driving test method.
  • the embodiment of the present application also provides a computer program, including computer-readable code, when the computer-readable code is run in an electronic device, the processor in the electronic device executes any one of the above-mentioned intersection detections. Method or any one of the above neural network training methods or any one of the above intelligent driving test methods.
  • the intersection detection method includes: extracting features from a road image to obtain a feature map of the road image;
  • the feature map of the road image determines the detection frame of the intersection on the road shown in the road image;
  • the detection frame of the intersection represents the area of the intersection in the road image;
  • the lower border of the detection frame is on the road surface, so the distance between the device that collects the road image and the intersection can be determined according to the lower border of the detection frame of the intersection; this way, even when clear traffic lights or traffic lights cannot be obtained Ground stop line image, or when there is no traffic light or ground stop line at the intersection, the embodiment of the present application can also implement intersection detection according to the feature map of the road image, so as to determine the distance between the device that collects the road image and the intersection .
  • FIG. 1 is a flow chart of the intersection detection method according to an embodiment of the application
  • Fig. 2 is a flowchart of a neural network training method according to an embodiment of the application
  • FIG. 3 is an example diagram of intersection detection using a trained neural network in the embodiment of the present application.
  • Fig. 4 is a flowchart of a smart driving method according to an embodiment of the application.
  • FIG. 5 is a schematic diagram of the composition structure of an intersection detection device according to an embodiment of the application.
  • FIG. 6 is a schematic diagram of the composition structure of a neural network training device according to an embodiment of the application.
  • FIG. 7 is a schematic diagram of the composition structure of a smart driving device according to an embodiment of the application.
  • FIG. 8 is a schematic structural diagram of an electronic device according to an embodiment of the application.
  • the terms "including”, “including” or any other variants thereof are intended to cover non-exclusive inclusion, so that a method or device including a series of elements not only includes what is clearly stated Elements, and also include other elements not explicitly listed, or elements inherent to the implementation of the method or device. Without more restrictions, the element defined by the sentence “including a" does not exclude the existence of other related elements in the method or device that includes the element (such as steps or steps in the method).
  • the unit in the device for example, the unit may be a part of a circuit, a part of a processor, a part of a program or software, etc.).
  • intersection detection, neural network training method, and smart driving method provided in the embodiments of this application include a series of steps, but the intersection detection, neural network training method, and smart driving method provided in the embodiments of this application are not limited to the recorded steps.
  • the intersection detection device, neural network training device, and smart driving device provided in the embodiments of the application include a series of modules, but the devices provided in the embodiments of the application are not limited to include the explicitly recorded modules, and may also include Related information, or modules that need to be set for processing based on information.
  • the embodiments of the present application can be applied to a computer system composed of a terminal and a server, and can be operated with many other general-purpose or special-purpose computing system environments or configurations.
  • the terminal can be a thin client, a thick client, a handheld or laptop device, a microprocessor-based system, a set-top box, a programmable consumer electronic product, a network personal computer, a small computer system, etc.
  • the server can be a server computer System small computer system, large computer system and distributed cloud computing technology environment including any of the above systems, etc.
  • Electronic devices such as terminals and servers can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system.
  • program modules may include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types.
  • the computer system/server can be implemented in a distributed cloud computing environment. In the distributed cloud computing environment, tasks are executed by remote processing equipment linked through a communication network.
  • program modules may be located on a storage medium of a local or remote computing system including a storage device.
  • intersection traffic lights or ground stop lines captured by the vehicle's camera
  • clear traffic lights or ground stop line images cannot be obtained, resulting in the above-mentioned intersection detection
  • the solution cannot accurately detect intersections; in addition, some intersections do not have traffic lights or ground stop lines, which will cause the aforementioned intersection detection solutions to fail to achieve intersection detection.
  • a method for detecting intersections is proposed, and the embodiments of the present application can be applied to scenarios such as automatic driving and assisted driving.
  • Fig. 1 is a flow chart of the intersection detection method according to an embodiment of this application. As shown in Fig. 1, the process may include:
  • Step 101 Perform feature extraction on the road image to obtain a feature map of the road image.
  • the road image is an image that requires intersection detection.
  • the format of the road image may be Joint Photographic Experts Group (JPEG), Bitmap (BMP), Portable Network Graphics (PNG) or other formats; it should be noted that, Here, only the format of the road image is described as an example, and the embodiment of the present application does not limit the format of the sample image.
  • road images can be acquired from the local storage area or the network, or image acquisition equipment can be used to acquire road images.
  • the image acquisition equipment can include a camera installed on the vehicle, etc.; in practical applications, the vehicle can be Set up one or more cameras to collect road images in front of the vehicle.
  • the feature map of the road image may be used to characterize at least one of the following features of the road image: color feature, texture feature, shape feature, and spatial relationship feature.
  • the feature map of the road image may be used to characterize at least one of the following features of the road image: color feature, texture feature, shape feature, and spatial relationship feature.
  • SIFT Scale-invariant feature transform
  • HOG Histogram of Oriented Gradient
  • a pre-trained neural network for extracting image feature maps can also be used to extract features from road images.
  • Step 102 Determine the detection frame of the intersection on the road shown in the road image according to the feature map of the road image; the detection frame of the intersection represents the area of the intersection in the road image, and the lower border of the detection frame of the intersection is On the pavement of the road.
  • the judgment result includes the following two situations: the road shown in the road image has an intersection, and the road image has an intersection.
  • the road shown does not have an intersection; when there is an intersection on the road shown in the road image, the detection frame of the intersection on the road shown in the road image can be determined according to the feature map of the road image, and the detection frame of the intersection is output; in the road image When there is no intersection on the road shown, no output is performed.
  • the pre-trained neural network for extracting the intersection detection frame can be used to determine the detection frame of the intersection on the road shown in the road image.
  • the shape of the detection frame of the intersection is not limited.
  • the shape of the detection frame of the intersection may be a rectangle, a trapezoid, etc.; in a specific example, the road shown in the road image has an intersection, and the road
  • the neural network used to extract the intersection detection frame can output the detection frame of a rectangular intersection; in another specific example, the road shown in the road image is not There are intersections.
  • the neural network for extracting the intersection detection frame does not output any data.
  • Step 103 Determine the distance between the device that collects road images and the intersection according to the lower border of the detection frame of the intersection.
  • the position of the intersection can be determined according to the lower border of the detection frame of the intersection, and further, combined with the known location of the device that collects road images, it can be Determine the distance between the device that collects road images and the intersection.
  • the processor can be an Application Specific Integrated Circuit (ASIC) or a Digital Signal Processor (DSP). , Digital Signal Processing Device (Digital Signal Processing Device, DSPD), Programmable Logic Device (Programmable Logic Device, PLD), Field Programmable Gate Array (Field-Programmable Gate Array, FPGA), Central Processing Unit (Central Processing Unit, CPU ), at least one of a controller, a microcontroller, and a microprocessor.
  • ASIC Application Specific Integrated Circuit
  • DSP Digital Signal Processor
  • DSPD Digital Signal Processing Device
  • PLD Programmable Logic Device
  • FPGA Field Programmable Gate Array
  • CPU Central Processing Unit
  • CPU Central Processing Unit
  • the embodiment of the present application first, feature extraction is performed on the road image to obtain the feature map of the road image; then, according to the feature map of the road image, the road image on the road shown in the road image is determined The detection frame of the intersection; and, because the lower border of the detection frame of the intersection in the embodiment of the present application is on the road surface, it can be determined based on the lower border of the detection frame of the intersection to determine whether the device that collects the road image and the intersection are In this way, even if a clear traffic light or ground stop line image cannot be obtained, or there is no traffic light or ground stop line at an intersection, the embodiment of the present application can also implement intersection detection based on the feature map of the road image, thereby determining The distance between the device that collects the road image and the intersection.
  • intersection detection method of the embodiment of the present application has strong universality.
  • the intersection can be accurately detected on the image in front of the vehicle, which can be realized when the intersection is far away
  • Intersection detection helps to provide sufficient reaction time for driving decisions and ensures driving safety. For example, it can provide sufficient reaction time for braking.
  • the position of the lower border of the detection frame of the intersection in the road image and the plane of the road image can be used as an example.
  • the coordinate conversion relationship between the road surface and the road surface determines the position of the lower border of the detection frame of the intersection on the road; according to the position of the lower border of the detection frame of the intersection on the road and the position of the device that collects road images on the road, Get the distance between the device that collects road images and the intersection.
  • the distance between the intersection and the vehicle cannot be determined; and in the embodiment of the present application, when the device for collecting road images is located in the vehicle, the distance between the device for collecting road images and the intersection can be set. The distance is taken as the distance between the vehicle and the intersection.
  • the embodiment of the application can consider that the lower border of the intersection detection frame will fit the road surface. According to the position of the lower border of the intersection detection frame in the road image, it can be accurately Estimating the distance between the vehicle and the intersection is conducive to providing sufficient reaction time for driving decisions and ensuring driving safety.
  • the position coordinates of the lower border of the detection frame of the intersection can be converted to the world coordinate system to obtain that the lower border of the detection frame of the intersection is in the world.
  • the position of the coordinate system that is, the position of the lower border of the detection frame of the intersection on the road.
  • the plane of the road image and the road surface are two different planes.
  • a Homography matrix can be used to express the coordinate conversion relationship between the plane of the road image and the road surface.
  • the homography matrix the position coordinates of the lower border of the detection frame of the intersection are converted to the world coordinate system; the homography matrix can be calculated from the road image and some corresponding points in the world coordinate system, based on this homography matrix , Can accurately get the position of each point in the road image in the world coordinate system.
  • the foregoing intersection detection method may be executed by a neural network.
  • the neural network is trained using sample images and labeling results of the sample images.
  • the labeling results of the sample images include the labeling frame of the intersection on the road shown in the positive sample image.
  • the label frame of the intersection on the road shown in the positive sample image represents the position of the intersection in the positive sample image, and the lower border of the label frame of the intersection on the road shown in the positive sample image is at the edge of the road shown in the positive sample image. On the road.
  • the format of the sample image can be Joint Photographic Experts Group (JPEG), Bitmap (BMP), Portable Network Graphics (PNG) or other formats; it should be noted that here only The format of the road image is illustrated by an example, and the embodiment of the present application does not limit the format of the sample image.
  • JPEG Joint Photographic Experts Group
  • BMP Bitmap
  • PNG Portable Network Graphics
  • sample images can be obtained from a local storage area or the network, or image collection equipment can be used to collect sample images.
  • the training of the neural network based on the positive sample image is beneficial to enable the trained neural network to detect the intersection in the road image.
  • FIG. 2 is a flowchart of a neural network training method according to an embodiment of the application. As shown in FIG. 2, the process may include:
  • Step 201 Perform feature extraction on a sample image to obtain a feature map of the sample image.
  • the feature map of the sample image can be used to characterize at least one of the following features of the sample image: color feature, texture feature, shape feature, and spatial relationship feature; for the implementation of this step, for example,
  • the sample image is input into the neural network, and the neural network is used to extract the features of the sample image to obtain the feature map of the sample image.
  • the type of neural network is not limited.
  • the neural network may be a single-shot multibox detector (SSD), you only look once (You Only Look Once), Faster Regional-Convolutional Neural Networks (Faster RCNN) or other neural networks based on deep learning.
  • the network structure of the neural network is not limited.
  • the network structure of the neural network may be a 50-layer residual network structure, a VGG16 network structure, or a MobileNet network structure.
  • Step 202 Determine the detection result of the sample image according to the feature map of the sample image.
  • the detection result includes the following two situations: the road shown in the sample image has an intersection, and the sample image has an intersection. There is no intersection in the road shown.
  • the labeling result of the sample image is the labeling frame of the intersection on the road shown in the positive sample image, and the labeling frame represents the position of the intersection in the positive sample image, And the lower border of the above-mentioned labeling frame is on the road surface shown in the positive sample image; obviously, when the sample image is a positive sample image, the detection result of the sample image can be determined according to the feature map of the sample image, that is, the detection result of the sample image can be determined. Check box.
  • Step 203 Adjust the network parameter value of the neural network according to the labeling result of the sample image and the detection result.
  • the network parameter value of the neural network can be adjusted according to the difference between the annotation result of the sample image and the above detection result.
  • the loss of the neural network can be calculated. The loss of the neural network is used to characterize the difference between the labeling results of the sample image and the above detection results; then, the loss of the initial neural network can be reduced according to the loss of the initial neural network as The goal is to adjust the network parameter values of the neural network.
  • Step 204 Determine whether the detection result of the neural network on the sample image after the network parameter value adjustment meets the set accuracy requirement, if not, return to step 201; if it meets, then execute step 205.
  • the set accuracy requirement may be that the difference between the detection result of the sample image and the annotation result of the sample image is within a preset range.
  • Step 205 Use the neural network after the network parameter values are adjusted as the neural network that has been trained.
  • steps 201 to 205 can be implemented by a processor in an electronic device.
  • the aforementioned processor can be at least ASIC, DSP, DSPD, PLD, FPGA, CPU, controller, microcontroller, and microprocessor.
  • ASIC ASIC
  • DSP digital signal processor
  • DSPD DSPD
  • PLD PLD
  • FPGA field-programmable gate array
  • CPU controller
  • microcontroller microprocessor
  • the detection result of the sample image can be determined according to the feature map of the sample image; therefore, the trained neural network can be able to obtain a clear Traffic lights or ground stop line images, or when there are no traffic lights or ground stop lines at the intersection, intersection detection can also be realized based on the feature map of the road image; and, since the positive sample image includes the intersection, the neural network is performed based on the positive sample image
  • the training is beneficial to enable the trained neural network to detect intersections in the road image.
  • the lower border of the label frame of the intersection on the road shown in the positive sample image is on the road surface of the road shown in the positive sample image; in this way, even when there is no obvious sign at the intersection, it can be determined
  • the lower border of the marking frame of the intersection is conducive to marking; further, since the marking frame of the intersection is on the pavement of the road, the marking frame of the intersection is consistent with the actual situation, and then it is on the road shown in the sample image. Based on the marking frame of the intersection, the trained neural network can accurately obtain the detection frame of the intersection.
  • the label box of the intersection on the road shown in the positive sample image The border is aligned with the above stop line; because the lower border of the marking box marking the exit is aligned with the stop line, the marking box of the intersection is consistent with the actual situation, and then based on the marking box of the intersection on the road shown in the sample image , So that the trained neural network can accurately obtain the detection frame of the intersection.
  • the intersection of the positive sample image is marked with a rectangular marking frame. If the intersection is far away, it needs to be based on experience and observation of the intersection area.
  • the lower border of the labeling frame is marked on the road surface, and the height of the rectangular labeling frame is set to a fixed value, for example, the height of the rectangular labeling frame is 80 pixels.
  • the difference between the heights of the labeled frames in multiple positive sample images containing the same intersection is within a preset range; the preset range can be preset according to the actual situation, for example, containing multiple positive samples of the same intersection
  • the heights of the label boxes in the image are the same, all of which are 80 pixels.
  • multiple positive sample images containing the same intersection may be images taken continuously.
  • the marking frame of the intersection needs to be marked.
  • the sample image when the sample image is a negative sample image, there is no intersection on the road in the negative sample image, and the labeling result of the sample image indicates that there is no labeling frame in the negative sample image.
  • the error detection rate of the trained neural network for the image that does not include the intersection area can be reduced, that is, the error detection rate can be detected more accurately. Contains the image of the intersection area.
  • the ratio of the positive sample image to the negative sample image is greater than the set ratio threshold; in this way, by inputting enough positive sample images to the neural network,
  • the network training of the neural network can enable the trained neural network to more accurately detect the intersection area containing the image of the intersection.
  • the road image can be input to the trained neural network, and the trained neural network can be used for intersection detection, and further, the intersection on the road shown in the road image can be determined Or, it is determined that there is no intersection on the road shown in the road image.
  • Fig. 3 is an example diagram of the embodiment of the application using a trained neural network for intersection detection.
  • the image to be detected represents a road image taken by a single camera of a vehicle
  • the detection network represents a trained neural network. It can be seen that the intersection detection result includes a detection frame representing the intersection, and the lower border of the detection frame at the intersection is fitted to the road surface.
  • smart driving equipment includes, but is not limited to, self-driving vehicles, equipped with advanced Advanced Driving Assistant System (ADAS) vehicles, ADAS-equipped robots, etc.
  • ADAS Advanced Driving Assistant System
  • Fig. 4 is a flowchart of a smart driving method according to an embodiment of the application. As shown in Fig. 4, the process may include:
  • Step 401 Obtain a road image.
  • Step 402 Perform intersection detection on the road image according to any of the foregoing intersection detection methods.
  • the intersection detection on the road image the detection result obtained can be a detection frame to determine the intersection on the road shown in the road image, or it can be determined that the road shown in the road image does not have an intersection ;
  • the distance between the device that collects road images and the intersection can also be determined.
  • Step 403 Perform driving control on the smart driving device according to the distance between the smart driving device that collects the road image and the intersection.
  • smart driving equipment can be directly controlled to drive (automatic driving and robots), or instructions can be sent to the driver, and the driver can control the vehicle (for example, a vehicle equipped with ADAS) to drive.
  • vehicle for example, a vehicle equipped with ADAS
  • the distance between the intelligent driving device that collects road images and the intersection can be obtained, which is conducive to providing assistance to vehicle driving according to the distance between the intelligent driving device that collects road images and the intersection.
  • the safety of vehicle driving is conducive to providing assistance to vehicle driving according to the distance between the intelligent driving device that collects road images and the intersection.
  • an embodiment of the present application proposes an intersection detection device.
  • FIG. 5 is a schematic diagram of the composition structure of an intersection detection device according to an embodiment of the application. As shown in FIG. 5, the device includes: a first extraction module 501, a detection module 502, and a first determination module 503, wherein:
  • the first extraction module 501 is configured to perform feature extraction on a road image to obtain a feature map of the road image
  • the detection module 502 is configured to determine the detection frame of the intersection on the road shown in the road image according to the feature map of the road image; the detection frame of the intersection represents the area of the intersection in the road image, and The lower border of the detection frame of the intersection is on the road surface of the road;
  • the first determining module 503 is configured to determine the distance between the device that collects the road image and the intersection according to the lower border of the detection frame of the intersection.
  • the detection module 502 is further configured to determine, according to the feature map of the road image, that the road shown in the road image does not have an intersection.
  • the first determining module 503 is configured to determine the position of the lower frame of the detection frame of the intersection in the road image and the difference between the plane of the road image and the road surface of the road.
  • the position of the lower border of the detection frame of the intersection on the road is determined according to the coordinate conversion relationship between the intersections; the position of the lower border of the detection frame of the intersection on the road is in relation to the device that collects the road image. From the position on the road, the distance between the device that collects the road image and the intersection is obtained.
  • the device is implemented based on a neural network, and the neural network is trained using sample images and labeling results of sample images.
  • the labeling results of sample images include the roads shown in the positive sample images.
  • the labeling frame of the intersection of, the labeling frame represents the position of the intersection in the positive sample image, and the lower border of the labeling frame is on the road surface of the road shown in the positive sample image.
  • the first extraction module 501, the detection module 502, and the first determination module 503 can all be implemented by a processor in an electronic device.
  • the aforementioned processors can be ASIC, DSP, DSPD, PLD, FPGA, CPU, and controller. , At least one of microcontroller and microprocessor.
  • FIG. 6 is a schematic diagram of the composition structure of a neural network training device according to an embodiment of the application. As shown in FIG. 6, the device may include a second extraction module 601, a second determination module 602, and an adjustment module 603, where:
  • the second extraction module 601 is configured to perform feature extraction on a sample image to obtain a feature map of the sample image
  • the second determining module 602 is configured to determine the detection result of the sample image according to the feature map of the sample image;
  • the adjustment module 603 is configured to adjust the network parameter value of the neural network according to the annotation result of the sample image and the detection result;
  • the labeling result of the sample image is the labeling frame of the intersection on the road shown in the positive sample image, and the labeling frame represents the position of the intersection in the positive sample image , And the lower border of the labeling frame of the intersection on the road shown in the positive sample image is on the road surface of the road shown in the positive sample image.
  • the positive sample image includes a stop line of a road intersection, and the lower border of the label frame of the intersection on the road shown in the positive sample image is aligned with the stop line.
  • the height difference of the labeled frames in the multiple positive sample images containing the same intersection is within a preset range.
  • the labeling result of the sample image includes that there is no labeling frame in the negative sample image .
  • the second extraction module 601, the second determination module 602, and the adjustment module 603 can all be implemented by a processor in an electronic device.
  • the aforementioned processors can be ASIC, DSP, DSPD, PLD, FPGA, CPU, and controller. , At least one of microcontroller and microprocessor.
  • FIG. 7 is a schematic diagram of the composition structure of a smart driving device according to an embodiment of the application. As shown in FIG. 7, the device includes: an acquisition module 701 and a processing module 702, wherein,
  • the obtaining module 701 is configured to obtain road images
  • the processing module 702 is configured to perform intersection detection on the road image according to any one of the foregoing intersection detection methods; and perform driving control on the device according to the distance between the intelligent driving device that collects the road image and the intersection.
  • both the acquisition module 701 and the processing module 702 can be implemented by a processor in a smart driving device.
  • the aforementioned processors can be ASIC, DSP, DSPD, PLD, FPGA, CPU, controller, microcontroller, and microprocessor. At least one of them.
  • the functional modules in this embodiment may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit can be realized in the form of hardware or software function module.
  • the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer readable storage medium.
  • the technical solution of this embodiment is essentially or It is said that the part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product.
  • the computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can A personal computer, a server, or a network device, etc.) or a processor (processor) executes all or part of the steps of the method described in this embodiment.
  • the aforementioned storage media include: U disk, mobile hard disk, read only memory (Read Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes.
  • the computer program instructions corresponding to any intersection detection method, neural network training method, or smart driving method in this embodiment can be stored on storage media such as optical disks, hard disks, and USB flash drives.
  • storage media such as optical disks, hard disks, and USB flash drives.
  • FIG. 8 shows an electronic device 80 provided by an embodiment of the present application, which may include: a memory 81 and a processor 82; wherein,
  • the memory 81 is configured to store computer programs and data
  • the processor 82 is configured to execute a computer program stored in the memory to implement any intersection detection method, neural network training method, or smart driving method in the foregoing embodiments.
  • the aforementioned memory 81 may be a volatile memory (volatile memory), such as RAM; or a non-volatile memory (non-volatile memory), such as ROM, flash memory, or hard disk (Hard Disk). Drive, HDD) or Solid-State Drive (SSD); or a combination of the foregoing types of memories, and provide instructions and data to the processor 82.
  • volatile memory volatile memory
  • non-volatile memory non-volatile memory
  • ROM read-only memory
  • flash memory read-only memory
  • HDD hard disk
  • SSD Solid-State Drive
  • the aforementioned processor 82 may be at least one of ASIC, DSP, DSPD, PLD, FPGA, CPU, controller, microcontroller, and microprocessor. It can be understood that, for different devices, the electronic devices used to implement the above-mentioned processor functions may also be other, which is not specifically limited in the embodiment of the present application.
  • the embodiment of the present application also provides a computer program, including computer-readable code, when the computer-readable code is run in an electronic device, the processor in the electronic device executes any one of the above-mentioned intersection detections. Method or any one of the above neural network training methods or any one of the above intelligent driving test methods.
  • the functions or modules contained in the apparatus provided in the embodiments of the present application can be used to execute the methods described in the above method embodiments.
  • the functions or modules contained in the apparatus provided in the embodiments of the present application can be used to execute the methods described in the above method embodiments.
  • the technical solution of the present invention essentially or the part that contributes to the existing technology can be embodied in the form of a software product, the computer software product is stored in a storage medium (such as ROM/RAM, magnetic disk, The optical disc) includes a number of instructions to enable a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the method described in each embodiment of the present invention.
  • a terminal which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.
  • the embodiments of the present application provide a method, device, electronic device, computer storage medium and computer program for intersection detection, neural network training, and intelligent driving.
  • the intersection detection method includes: extracting features of road images to obtain information about the road image. Feature map; according to the feature map of the road image, determine the detection frame of the intersection on the road shown in the road image; the detection frame of the intersection represents the area of the intersection in the road image, and the detection of the intersection.
  • the lower frame of the frame is on the road surface of the road; according to the lower frame of the detection frame of the intersection, the distance between the device that collects the road image and the intersection is determined.
  • the embodiment of the present application can implement intersection detection based on the feature map of the road image, thereby determining to collect the road The distance between the image device and the intersection.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Automation & Control Theory (AREA)
  • Mechanical Engineering (AREA)
  • Transportation (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Software Systems (AREA)
  • Evolutionary Biology (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Image Analysis (AREA)
  • Traffic Control Systems (AREA)
  • Control Of Driving Devices And Active Controlling Of Vehicle (AREA)

Abstract

一种路口检测、神经网络训练及智能行驶方法、装置、电子设备和计算机存储介质，该路口检测方法包括：对道路图像进行特征提取，获得所述道路图像的特征图；根据所述道路图像的特征图，确定所述道路图像所示的道路上的路口的检测框；所述路口的检测框表示路口在所述道路图像中的区域，所述路口的检测框的下边框在所述道路的路面上；根据所述路口的检测框的下边框，确定采集所述道路图像的设备与所述路口之间的距离。

Description

路口检测、神经网络训练及智能行驶方法、装置和设备
相关申请的交叉引用
本申请基于申请号为201911083615.4、申请日为2019年11月7日的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此引入本申请作为参考。
技术领域
本申请涉及计算机视觉处理技术，涉及但不限于一种路口检测、神经网络训练及智能行驶方法、装置、电子设备、计算机存储介质和计算机程序。
背景技术
近年来，随着生活水平的提高以及辅助驾驶技术的提升，越来越多的辅助驾驶相关的需求被提出，同时越来越多的学者和公司将深度学习应用到辅助驾驶方案中。在执行辅助驾驶或者自动驾驶任务时，路口检测以及根据检测到的路口确定车辆与路口之间的距离是非常重要的任务。
发明内容
本申请实施例期望提供路口检测的技术方案。
本申请实施例提供了一种路口检测方法，所述方法包括：
对道路图像进行特征提取，获得所述道路图像的特征图；
根据所述道路图像的特征图，确定所述道路图像所示的道路上的路口的检测框；所述路口的检测框表示路口在所述道路图像中的区域，所述路口的检测框的下边框在所述道路的路面上；
根据所述路口的检测框的下边框，确定采集所述道路图像的设备与所述路口之间的距离。
本申请的一些实施例中，所述方法还包括：
根据所述道路图像的特征图，确定所述道路图像所示的道路不存在路口。
本申请的一些实施例中，所述根据所述路口的检测框的下边框，确定采集所述道路图像的设备与所述路口之间的距离，包括：
根据所述路口的检测框的下边框在所述道路图像中的位置以及所述道路图像的平面和所述道路的路面之间的坐标转换关系，确定所述路口的检测框的下边框在所述道路上的位置；
根据所述路口的检测框的下边框在所述道路上的位置与采集所述道路图像的设备在所述道路上的位置，得出采集所述道路图像的设备与所述路口之间的距离。
本申请的一些实施例中，所述方法由神经网络执行，所述神经网络采用样本图像以及样本图像的标注结果训练得到，所述样本图像的标注结果包括正样本图像所示的道路上的路口的标注框，所述标注框表征路口在所述正样本图像中的位置，且所述标注框的下边框在所述正样本图像所示的道路的路面上。
本申请实施例还提供了一种神经网络训练方法，包括：
对样本图像进行特征提取，获得所述样本图像的特征图；
根据所述样本图像的特征图，确定所述样本图像的检测结果；
根据所述样本图像的标注结果和所述检测结果，调整所述神经网络的网络参数值；
当所述样本图像为正样本图像时，所述样本图像的标注结果为所述正样本图像所示的道路上的路口的标注框，所述标注框表征路口在所述正样本图像中的位置，且所述正样本图像所示的道路上的路口的标注框的下边框在所述正样本图像所示的道路的路面 上。
本申请的一些实施例中，所述正样本图像中包括道路的路口的停止线，所述正样本图像所示的道路上的路口的标注框的下边框与所述停止线对齐。
本申请的一些实施例中，包含同一路口的多个正样本图像中的标注框的高度之差在预设范围内。
本申请的一些实施例中，当所述样本图像为负样本图像时，所述负样本图像中的道路上不存在路口，所述样本图像的标注结果包括所述负样本图像中不存在标注框。
本申请实施例还提供了一种智能行驶方法，包括：
获取道路图像；
根据上述任意一种路口检测方法，对所述道路图像进行路口检测；
根据采集所述道路图像的智能行驶设备与所述路口之间的距离对所述设备进行行驶控制。
本申请实施例还提供了一种路口检测装置，所述装置包括第一提取模块、检测模块和第一确定模块；其中，
第一提取模块，配置为对道路图像进行特征提取，获得所述道路图像的特征图；
检测模块，配置为根据所述道路图像的特征图，确定所述道路图像所示的道路上的路口的检测框；所述路口的检测框表示路口在所述道路图像中的区域，所述路口的检测框的下边框在所述道路的路面上；
第一确定模块，配置为根据所述路口的检测框的下边框，确定采集所述道路图像的设备与所述路口之间的距离。
本申请的一些实施例中，所述检测模块，还配置为根据所述道路图像的特征图，确定所述道路图像所示的道路不存在路口。
本申请的一些实施例中，所述第一确定模块，配置为根据所述路口的检测框的下边框在所述道路图像中的位置以及所述道路图像的平面和所述道路的路面之间的坐标转换关系，确定所述路口的检测框的下边框在所述道路上的位置；根据所述路口的检测框的下边框在所述道路上的位置与采集所述道路图像的设备在所述道路上的位置，得出采集所述道路图像的设备与所述路口之间的距离。
本申请的一些实施例中，所述装置是基于神经网络实现的，所述神经网络采用样本图像以及样本图像的标注结果训练得到，所述样本图像的标注结果包括正样本图像所示的道路上的路口的标注框，所述标注框表征路口在所述正样本图像中的位置，且所述标注框的下边框在所述正样本图像所示的道路的路面上。
本申请实施例还提供了一种神经网络训练装置，所述装置包括：第二提取模块、第二确定模块和调整模块，其中，
第二提取模块，配置为对样本图像进行特征提取，获得所述样本图像的特征图；
第二确定模块，配置为根据所述样本图像的特征图，确定所述样本图像的检测结果；
调整模块，配置为根据所述样本图像的标注结果和所述检测结果，调整所述神经网络的网络参数值；
当所述样本图像为正样本图像时，所述样本图像的标注结果为所述正样本图像所示的道路上的路口的标注框，所述标注框表征路口在所述正样本图像中的位置，且所述正样本图像所示的道路上的路口的标注框的下边框在所述正样本图像所示的道路的路面上。
本申请的一些实施例中，所述正样本图像中包括道路的路口的停止线，所述正样本图像所示的道路上的路口的标注框的下边框与所述停止线对齐。
本申请的一些实施例中，包含同一路口的多个正样本图像中的标注框的高度之差在预设范围内。
本申请的一些实施例中，当所述样本图像为负样本图像时，所述负样本图像中的道路上不存在路口，所述样本图像的标注结果包括所述负样本图像中不存在标注框。
本申请实施例还提供了一种智能行驶装置，所述装置包括：获取模块和处理模块，其中，
获取模块，配置为获取道路图像；
处理模块，配置为根据上述任意一种路口检测方法，对所述道路图像进行路口检测；根据采集所述道路图像的智能行驶设备与所述路口之间的距离对所述设备进行行驶控制。
本申请实施例还提供了一种电子设备，包括处理器和配置为存储能够在处理器上运行的计算机程序的存储器；其中，
所述处理器配置为运行所述计算机程序以执行上述任意一种路口检测方法或上述任意一种神经网络训练方法或上述任意一种智能行驶测方法。
本申请实施例还提供了一种计算机存储介质，其上存储有计算机程序，该计算机程序被处理器执行时实现上述任意一种路口检测方法或上述任意一种神经网络训练方法或上述任意一种智能行驶测方法。
本申请实施例还提供了一种计算机程序，包括计算机可读代码，当所述计算机可读代码在电子设备中运行时，所述电子设备中的处理器执行用于实现上述任意一种路口检测方法或上述任意一种神经网络训练方法或上述任意一种智能行驶测方法。
本申请实施例提出的路口检测、神经网络训练及智能行驶方法、装置、电子设备和计算机存储介质中，路口检测方法包括：对道路图像进行特征提取，获得所述道路图像的特征图；根据所述道路图像的特征图，确定所述道路图像所示的道路上的路口的检测框；所述路口的检测框表示路口在所述道路图像中的区域；并且，由于本申请实施例中路口的检测框的下边框在道路的路面上，因此可以根据路口的检测框的下边框，确定采集所述道路图像的设备与所述路口之间的距离；如此，即使在无法获取到清晰的红绿灯或地面停止线图像，或路口没有红绿灯或地面停止线的情况下，本申请实施例也可以根据道路图像的特征图实现路口检测，从而确定采集所述道路图像的设备与所述路口之间的距离。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，而非限制本申请。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，这些附图示出了符合本申请的实施例，并与说明书一起用于说明本申请的技术方案。
图1为本申请实施例的路口检测方法的流程图；
图2为本申请实施例的神经网络训练方法的流程图；
图3的本申请实施例利用训练完成的神经网络进行路口检测的示例图；
图4为本申请实施例的智能行驶方法的流程图；
图5为本申请实施例的路口检测装置的组成结构示意图；
图6为本申请实施例的神经网络训练装置的组成结构示意图；
图7为本申请实施例的智能行驶装置的组成结构示意图；
图8为本申请实施例的电子设备的结构示意图。
具体实施方式
以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处所提供的实施例仅仅用以解释本申请，并不用于限定本申请。另外，以下所提供的实施例是用于 实施本申请的部分实施例，而非提供实施本申请的全部实施例，在不冲突的情况下，本申请实施例记载的技术方案可以任意组合的方式实施。
需要说明的是，在本申请实施例中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的方法或者装置不仅包括所明确记载的要素，而且还包括没有明确列出的其他要素，或者是还包括为实施方法或者装置所固有的要素。在没有更多限制的情况下，由语句“包括一个......”限定的要素，并不排除在包括该要素的方法或者装置中还存在另外的相关要素(例如方法中的步骤或者装置中的单元，例如的单元可以是部分电路、部分处理器、部分程序或软件等等)。
例如，本申请实施例提供的路口检测、神经网络训练方法和智能行驶方法包含了一系列的步骤，但是本申请实施例提供的路口检测、神经网络训练方法和智能行驶方法不限于所记载的步骤，同样地，本申请实施例提供的路口检测装置、神经网络训练装置和智能行驶装置包括了一系列模块，但是本申请实施例提供的装置不限于包括所明确记载的模块，还可以包括为获取相关信息、或基于信息进行处理时所需要设置的模块。
本文中术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中术语“至少一种”表示多种中的任意一种或多种中的至少两种的任意组合，例如，包括A、B、C中的至少一种，可以表示包括从A、B和C构成的集合中选择的任意一个或多个元素。
本申请实施例可以应用于终端和服务器组成的计算机系统中，并可以与众多其它通用或专用计算系统环境或配置一起操作。这里，终端可以是瘦客户机、厚客户机、手持或膝上设备、基于微处理器的系统、机顶盒、可编程消费电子产品、网络个人电脑、小型计算机系统，等等，服务器可以是服务器计算机系统小型计算机系统﹑大型计算机系统和包括上述任何系统的分布式云计算技术环境，等等。
终端、服务器等电子设备可以在由计算机系统执行的计算机系统可执行指令(诸如程序模块)的一般语境下描述。通常，程序模块可以包括例程、程序、目标程序、组件、逻辑、数据结构等等，它们执行特定的任务或者实现特定的抽象数据类型。计算机系统/服务器可以在分布式云计算环境中实施，分布式云计算环境中，任务是由通过通信网络链接的远程处理设备执行的。在分布式云计算环境中，程序模块可以位于包括存储设备的本地或远程计算系统存储介质上。
在辅助驾驶或者自动驾驶任务中，需要通过摄像头和雷达来感知周围信息，同时需要给出准确的决策信息，如加速、避让、减速等；路口区域的结构往往比较复杂，在车辆距离路口较远的时候，如何对路口进行准确的预测以及距离估计则显得尤为重要。一般来说，对路口区域进行准确检测的目的为：可以有效地为自动驾驶决策提供充足的反应时间，且可以预留出充足的时间用于车辆减速。在相关技术中，通常是利用车辆的摄像头拍摄的路口红绿灯或者地面停止线等信息进行判断；在车辆距离路口较远的情况下，无法获取到清晰的红绿灯或地面停止线图像，导致上述路口检测方案无法进行准确地路口检测；另外，一些路口并没有红绿灯或地面停止线，会导致上述路口检测方案无法实现路口检测。
针对上述记载的问题，在本申请的一些实施例中，提出了一种路口检测方法，本申请实施例可以应用于自动驾驶、辅助驾驶等场景。
图1为本申请实施例的路口检测方法的流程图，如图1所示，该流程可以包括：
步骤101：对道路图像进行特征提取，获得道路图像的特征图。
这里，道路图像为需要进行路口检测的图像。示例性地，道路图像的格式可以是联合图像专家小组(Joint Photographic Experts GROUP，JPEG)、位图(Bitmap，BMP)、便携式网络图形(Portable Network Graphics，PNG)或其他格式；需要说明的是，这里 仅仅是对道路图像的格式进行了举例说明，本申请实施例并不对样本图像的格式进行限定。
在实际应用中，可以从本地存储区域或网络获取道路图像，也可以利用图像采集设备采集道路图像，这里，图像采集设备可以包括在车辆上安装的摄像头等；在实际应用中，在车辆上可以设置一个或多个摄像头，用于采集车辆前方的道路图像。
本申请实施例中，道路图像的特征图可以用于表征道路图像的以下至少一种特征：颜色特征、纹理特征、形状特征、空间关系特征。对于本步骤的实现方式，在一个示例中，可以利用尺度不变特征变换(Scale-invariant feature transform，SIFT)方法或方向梯度直方图(Histogram of Oriented Gradient，HOG)特征提取方法提取道路图像的特征图；在另一个示例中，也可以利用预先训练的提取图像特征图的神经网络，对道路图像进行特征提取。
步骤102：根据道路图像的特征图，确定道路图像所示的道路上的路口的检测框；路口的检测框表示路口在所述道路图像中的区域，所述路口的检测框的下边框在所述道路的路面上。
本申请实施例中，可以根据道路图像的特征图判断道路图像所示的道路是否存在路口，得到判断结果，显然，判断结果包括以下两种情况：道路图像所示的道路存在路口、道路图像所示的道路不存在路口；在道路图像所示的道路存在路口时，可以根据道路图像的特征图，确定道路图像所示的道路上的路口的检测框，并输出路口的检测框；在道路图像所示的道路不存在路口时，不进行任何输出。
在实际应用中，在道路图像所示的道路存在路口时，可以利用预先训练的提取路口检测框的神经网络，确定道路图像所示的道路上的路口的检测框。
本申请实施例中，并不对路口的检测框的形状进行限定，例如，路口的检测框的形状可以是矩形、梯形等；在一个具体的示例中，道路图像所示的道路存在路口，将道路图像的特征图输入至用于提取路口检测框的神经网络后，用于提取路口检测框的神经网络可以输出矩形的路口的检测框；在另一个具体的示例中，道路图像所示的道路不存在路口，将道路图像的特征图输入至用于提取路口检测框的神经网络后，用于提取路口检测框的神经网络不输出任何数据。
步骤103：根据路口的检测框的下边框，确定采集道路图像的设备与路口之间的距离。
可以理解的，由于路口的检测框的下边框在道路的路面上，因而，可以根据路口的检测框的下边框确定路口的位置，进而，结合已知的采集道路图像的设备的位置，便可以确定出采集道路图像的设备与路口之间的距离。
在实际应用中，步骤101至步骤103均可以利用电子设备中的处理器实现，上述处理器可以为特定用途集成电路(Application Specific Integrated Circuit，ASIC)、数字信号处理器(Digital Signal Processor，DSP)、数字信号处理装置(Digital Signal Processing Device，DSPD)、可编程逻辑装置(Programmable Logic Device，PLD)、现场可编程门阵列(Field－Programmable Gate Array，FPGA)、中央处理器(Central Processing Unit，CPU)、控制器、微控制器、微处理器中的至少一种。
可以看出，本申请实施例中，首先，对道路图像进行特征提取，获得所述道路图像的特征图；然后，根据所述道路图像的特征图，确定所述道路图像所示的道路上的路口的检测框；并且，由于本申请实施例中路口的检测框的下边框在道路的路面上，因此可以根据路口的检测框的下边框，确定采集所述道路图像的设备与所述路口之间的距离；如此，即使在无法获取到清晰的红绿灯或地面停止线图像，或路口没有红绿灯或地面停止线的情况下，本申请实施例也可以根据道路图像的特征图实现路口检测，从而确定采集所述道路图像的设备与所述路口之间的距离。
另外，本申请实施例的路口检测方法的普适性较强，在车辆上安装至少一个摄像头 的情况下，可以对车辆前方的图像进行准确地路口检测，可以在距离路口较远的情况下实现路口检测，有利于为驾驶决策提供充足的反应时间，保证了驾驶安全性，例如可以为刹车提供充足的反应时间。
本申请的一些实施例中，针对一些不包含路口的道路图像，可以直接根据道路图像的特征图，确认道路图像所示的道路不存在路口，有利于对驾驶决策提供帮助，保证了驾驶安全性。
对于根据路口的检测框的下边框，确定采集道路图像的设备与路口之间的距离的实现方式，示例性地，可以根据路口的检测框的下边框在道路图像中的位置以及道路图像的平面和道路的路面之间的坐标转换关系，确定路口的检测框的下边框在道路上的位置；根据路口的检测框的下边框在道路上的位置与采集道路图像的设备在道路上的位置，得出采集道路图像的设备与路口之间的距离。
在相关的路口检测方案中，无法确定路口与车辆之间的距离；而在本申请实施例中，在采集道路图像的设备位于车辆的情况下，可以将采集道路图像的设备与路口之间的距离作为车辆与路口之间的距离，也就是说，本申请实施例可以认为路口检测框的下边框会与路面贴合，根据路口的检测框的下边框在道路图像中的位置，可以准确地预估出车辆与路口之间的距离，有利于为驾驶决策提供充足的反应时间，保证了驾驶安全性。
在一实施方式中，可以根据道路图像的平面和道路的路面之间的坐标转换关系，将路口的检测框的下边框的位置坐标转换至世界坐标系，得到路口的检测框的下边框在世界坐标系的位置，即，得到路口的检测框的下边框在道路上的位置。
在实际应用中，道路图像的平面和道路的路面为两个不同的平面，如此，可以利用单应性(Homography)矩阵表示道路图像的平面和道路的路面之间的坐标转换关系，进而，可以根据单应性矩阵，将路口的检测框的下边框的位置坐标转换至世界坐标系；单应性矩阵可以通过道路图像与世界坐标系下的一些对应点计算得出，基于此单应性矩阵，可以准确地得出道路图像中的每一个点在世界坐标系下的位置。
作为一种实施方式，上述路口检测方法可以由神经网络执行，上述神经网络采用样本图像以及样本图像的标注结果训练得到，样本图像的标注结果包括正样本图像所示的道路上的路口的标注框，正样本图像所示的道路上的路口的标注框表征路口在正样本图像中的位置，且正样本图像所示的道路上的路口的标注框的下边框在正样本图像所示的道路的路面上。
这里，样本图像的格式可以是联合图像专家小组(Joint Photographic Experts GROUP，JPEG)、位图(Bitmap，BMP)、便携式网络图形(Portable Network Graphics，PNG)或其他格式；需要说明的是，这里仅仅是对道路图像的格式进行了举例说明，本申请实施例并不对样本图像的格式进行限定。
在实际应用中，可以从本地存储区域或网络获取样本图像，也可以利用图像采集设备采集样本图像。
可以理解地，由于正样本图像包括路口，因而，通过基于正样本图像进行神经网络的训练，有利于使训练完成的神经网络能够检测出道路图像中的路口。
下面结合附图示例性地说明上述神经网络的训练过程。
图2为本申请实施例的神经网络训练方法的流程图，如图2所示，该流程可以包括：
步骤201：对样本图像进行特征提取，获得所述样本图像的特征图。
本申请实施例中，样本图像的特征图可以用于表征样本图像的以下至少一种特征：颜色特征、纹理特征、形状特征、空间关系特征；对于本步骤的实现方式，示例性地，可以将样本图像输入至神经网络中，利用神经网络对样本图像进行特征提取，获得样本图像的特征图。
本申请实施例中，并不对神经网络的种类进行限定，示例性地，神经网络可以是单 步多框检测器(Single Shot MultiBox Detector，SSD)、你只看一次(You Only Look Once，)、快速区域卷积神经网络(Faster Region-Convolutional Neural Networks，Faster RCNN)或其他基于深度学习的神经网络。本申请实施例中，也不对神经网络的网络结构进行限定，例如，神经网络的网络结构可以是50层的残差网络结构、VGG16网络结构或MobileNet网络结构等。
步骤202：根据样本图像的特征图，确定样本图像的检测结果。
本申请实施例中，可以根据样本图像的特征图判断样本图像所示的道路是否存在路口，得到检测结果，显然，检测结果包括以下两种情况：样本图像所示的道路存在路口、样本图像所示的道路不存在路口。
本申请实施例中，当样本图像为正样本图像时，样本图像的标注结果为正样本图像所示的道路上的路口的标注框，上述标注框表征路口在所述正样本图像中的位置，且上述标注框的下边框在正样本图像所示的道路的路面上；显然，当样本图像为正样本图像时，可以根据样本图像的特征图，确定样本图像的检测结果，即，确定路口的检测框。
步骤203：根据样本图像的标注结果和所述检测结果，调整神经网络的网络参数值。
对于本步骤的实现方式，示例性地，可以根据样本图像的标注结果和上述检测结果的差异，调整神经网络的网络参数值。在实际实施时，可以计算神经网络的损失，神经网络的损失用于表征样本图像的标注结果和上述检测结果的差异；然后，可以根据初始神经网络的损失，以减小初始神经网络的损失为目标，调整神经网络的网络参数值。
步骤204：判断网络参数值调整后的神经网络对样本图像的检测结果是否满足设定的精度需求，如果不满足，则返回执行步骤201；如果满足，则执行步骤205。
这里，设定的精度需求可以是样本图像的检测结果与样本图像的标注结果的差异在预设范围内。
步骤205：将网络参数值调整后的神经网络作为训练完成的神经网络。
在实际应用中，步骤201至步骤205可以利用电子设备中的处理器实现，上述处理器可以为ASIC、DSP、DSPD、PLD、FPGA、CPU、控制器、微控制器、微处理器中的至少一种。
可以看出，本申请实施例中，在神经网络的训练过程中，由于可以根据样本图像的特征图，确定样本图像的检测结果；因而，可以使得训练完成的神经网络能够在无法获取到清晰的红绿灯或地面停止线图像，或路口没有红绿灯或地面停止线的情况下，也可以根据道路图像的特征图实现路口检测；并且，由于正样本图像包括路口，因而，通过基于正样本图像进行神经网络的训练，有利于使训练完成的神经网络能够检测出道路图像中的路口。
在实际应用中，在针对正样本图像标注路口的标注框时，由于很多路口没有明显的标志物，因此在进行数据标注时，也存在很大的困难；针对该问题，在本申请实施例中，可以采用多种方式解决，下面通过几个示例进行说明。
在第一个示例中，正样本图像所示的道路上的路口的标注框的下边框在正样本图像所示的道路的路面上；这样，即使在路口没有明显的标志物时，可以确定出路口的标注框的下边框，有利于进行标注；进一步地，由于标注出路口的标注框在道路的路面上，因而，路口的标注框与实际情况相符，进而在样本图像所示的道路上的路口的标注框的基础上，可以使训练完成的神经网络能够准确地得出路口的检测框。
在第一个示例中的基础上，作为一种可选的实施方式，在正样本图像中包括道路的路口的停止线的情况下，正样本图像所示的道路上的路口的标注框的下边框与上述停止线对齐；由于标注出路口的标注框的下边框与停止线对齐，因而，路口的标注框与实际情况相符，进而在样本图像所示的道路上的路口的标注框的基础上，可以使训练完成的神经网络能够准确地得出路口的检测框。
在第一个示例中的基础上，作为一种可选的实施方式，对正样本图像的路口利用矩形标注框标注，若路口距离较远时，需要根据经验和对路口区域的观察，将矩形标注框的下边框标注到路面上，同时将矩形标注框的高度设为固定值，例如，矩形标注框的高度为80个像素。
在第二个示例中，包含同一路口的多个正样本图像中的标注框的高度之差在预设范围内；预设范围可以根据实际情况预先设置，例如，包含同一路口的多个正样本图像中的标注框的高度一致，均为80个像素。
可以看出，由于包含同一路口的多个正样本图像中路口的标注框的高度之差在预设范围内，可以保证多个正样本图像的路口的标注框的一致性，在多个正样本图像的路口的标注框的基础上，有利于加快神经网络的训练过程。
在实际应用中，包含同一路口的多个正样本图像可以是连续拍摄到的图像。
在第三个示例中，在正样本图像中，当能够识别出前方路口时，则需要标注出路口的标注框。
在第四个示例中，在正样本图像中，当存在严重遮挡或者肉眼无法分辨是否是路口区域的情况时，不进行路口的标注。
本申请的一些实施例中，当样本图像为负样本图像时，负样本图像中的道路上不存在路口，样本图像的标注结果表示负样本图像中不存在标注框。
可以看出，通过将负样本图像输入至神经网络，进行神经网络的网络训练，可以使训练完成的神经网络针对不包含路口区域的图像的错误检测率降低，即，可以较为准确地检测出不包含路口区域的图像。
在一实施方式中，上述样本图像包括正样本图像和负样本图像时，正样本图像与负样本图像的比例大于设定比例阈值；如此，通过将足够多的正样本图像输入至神经网络，进行神经网络的网络训练，可以使训练完成的神经网络能够较为准确地检测出包含路口的图像的路口区域。
本申请实施例中，在得到训练完成的神经网络后，便可以将道路图像输入至训练完成的神经网络，利用训练完成的神经网络进行路口检测，进而，确定道路图像所示的道路上的路口的检测框，或，确定道路图像所示的道路不存在路口。
图3的本申请实施例利用训练完成的神经网络进行路口检测的示例图，如图3所示，待检测图像表示利用车辆的单摄像头拍摄的道路图像，检测网络表示训练完成的神经网络，可以看出，路口检测结果包含一个表示路口的检测框，路口的检测框的下边框与路面贴合。
在前述实施例提出的路口检测方法的基础上，本申请实施例还提出了一种智能行驶方法，可以应用于智能行驶设备中，这里，智能行驶设备包括但不限于自动驾驶车辆、装有高级驾驶辅助系统(Advanced Driving Assistant System，ADAS)的车辆、装有ADAS的机器人等。
图4为本申请实施例的智能行驶方法的流程图，如图4所示，该流程可以包括：
步骤401：获取道路图像。
本步骤的实现方式已经在前述记载的内容中作出说明，这里不再赘述。
步骤402：根据上述任意一种路口检测方法，对道路图像进行路口检测。
结合前述记载的内容，可以看出，对道路图像进行路口检测，得到的检测结果可以是确定道路图像所示的道路上的路口的检测框，或者，是确定道路图像所示的道路不存在路口；在确定路口的检测框的基础上，还可以确定采集道路图像的设备与路口之间的距离。
步骤403：根据采集道路图像的智能行驶设备与路口之间的距离对智能行驶设备进行行驶控制。
在实际应用中，可以直接控制智能行驶设备行驶(自动驾驶以及机器人)，也可以向驾驶员发送指令，由驾驶员来控制车辆(例如装有ADAS的车辆)行驶。
可以看出，基于路口检测方法，可以得出采集道路图像的智能行驶设备与路口之间的距离，有利于根据采集道路图像的智能行驶设备与路口之间的距离，对车辆驾驶提供帮助，提高车辆驾驶的安全性。
本领域技术人员可以理解，在具体实施方式的上述方法中，各步骤的撰写顺序并不意味着严格的执行顺序而对实施过程构成任何限定，各步骤的具体执行顺序应当以其功能和可能的内在逻辑确定
在前述实施例提出的路口检测方法的基础上，本申请实施例提出了一种路口检测装置。
图5为本申请实施例的路口检测装置的组成结构示意图，如图5所示，所述装置包括：第一提取模块501、检测模块502和第一确定模块503，其中，
第一提取模块501，配置为对道路图像进行特征提取，获得所述道路图像的特征图；
检测模块502，配置为根据所述道路图像的特征图，确定所述道路图像所示的道路上的路口的检测框；所述路口的检测框表示路口在所述道路图像中的区域，所述路口的检测框的下边框在所述道路的路面上；
第一确定模块503，配置为根据所述路口的检测框的下边框，确定采集所述道路图像的设备与所述路口之间的距离。
本申请的一些实施例中，检测模块502，还配置为根据所述道路图像的特征图，确定所述道路图像所示的道路不存在路口。
本申请的一些实施例中，所述第一确定模块503，配置为根据所述路口的检测框的下边框在所述道路图像中的位置以及所述道路图像的平面和所述道路的路面之间的坐标转换关系，确定所述路口的检测框的下边框在所述道路上的位置；根据所述路口的检测框的下边框在所述道路上的位置与采集所述道路图像的设备在所述道路上的位置，得出采集所述道路图像的设备与所述路口之间的距离。
本申请的一些实施例中，所述装置是基于神经网络实现的，所述神经网络采用样本图像以及样本图像的标注结果训练得到，所述样本图像的标注结果包括正样本图像所示的道路上的路口的标注框，所述标注框表征路口在所述正样本图像中的位置，且所述标注框的下边框在所述正样本图像所示的道路的路面上。
在实际应用中，第一提取模块501、检测模块502和第一确定模块503均可以利用电子设备中的处理器实现，上述处理器可以为ASIC、DSP、DSPD、PLD、FPGA、CPU、控制器、微控制器、微处理器中的至少一种。
图6为本申请实施例的神经网络训练装置的组成结构示意图，如图6所示，该装置可以包括第二提取模块601、第二确定模块602和调整模块603，其中，
第二提取模块601，配置为对样本图像进行特征提取，获得所述样本图像的特征图；
第二确定模块602，配置为根据所述样本图像的特征图，确定所述样本图像的检测结果；
调整模块603，配置为根据所述样本图像的标注结果和所述检测结果，调整所述神经网络的网络参数值；
当所述样本图像为正样本图像时，所述样本图像的标注结果为所述正样本图像所示的道路上的路口的标注框，所述标注框表征路口在所述正样本图像中的位置，且所述正样本图像所示的道路上的路口的标注框的下边框在所述正样本图像所示的道路的路面上。
本申请的一些实施例中，所述正样本图像中包括道路的路口的停止线，所述正样本图像所示的道路上的路口的标注框的下边框与所述停止线对齐。
本申请的一些实施例中，包含同一路口的多个正样本图像中的标注框的高度之差在预设范围内。
本申请的一些实施例中，当所述样本图像为负样本图像时，所述负样本图像中的道路上不存在路口，所述样本图像的标注结果包括所述负样本图像中不存在标注框。
在实际应用中，第二提取模块601、第二确定模块602和调整模块603均可以利用电子设备中的处理器实现，上述处理器可以为ASIC、DSP、DSPD、PLD、FPGA、CPU、控制器、微控制器、微处理器中的至少一种。
图7为本申请实施例的智能行驶装置的组成结构示意图，如图7所示，所述装置包括：获取模块701和处理模块702，其中，
获取模块701，配置为获取道路图像；
处理模块702，配置为根据上述任意一种路口检测方法，对所述道路图像进行路口检测；根据采集所述道路图像的智能行驶设备与所述路口之间的距离对所述设备进行行驶控制。
实际应用中，获取模块701和处理模块702均可以利用智能行驶设备中的处理器实现，上述处理器可以为ASIC、DSP、DSPD、PLD、FPGA、CPU、控制器、微控制器、微处理器中的至少一种。
另外，在本实施例中的各功能模块可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。
所述集成的单元如果以软件功能模块的形式实现并非作为独立的产品进行销售或使用时，可以存储在一个计算机可读取存储介质中，基于这样的理解，本实施例的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)或processor(处理器)执行本实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
具体来讲，本实施例中的任意一种路口检测方法、神经网络训练方法或智能行驶方法对应的计算机程序指令可以被存储在光盘，硬盘，U盘等存储介质上，当存储介质中的与任意一种路口检测方法、神经网络训练方法或智能行驶方法对应的计算机程序指令被一电子设备读取或被执行时，实现前述实施例的任意一种路口检测方法、神经网络训练方法或智能行驶方法。
基于前述实施例相同的技术构思，参见图8，其示出了本申请实施例提供的一种电子设备80，可以包括：存储器81和处理器82；其中，
所述存储器81，配置为存储计算机程序和数据；
所述处理器82，配置为执行所述存储器中存储的计算机程序，以实现前述实施例的任意一种路口检测方法、神经网络训练方法或智能行驶方法。
在实际应用中，上述存储器81可以是易失性存储器(volatile memory)，例如RAM；或者非易失性存储器(non-volatile memory)，例如ROM，快闪存储器(flash memory)，硬盘(Hard Disk Drive，HDD)或固态硬盘(Solid-State Drive，SSD)；或者上述种类的存储器的组合，并向处理器82提供指令和数据。
上述处理器82可以为ASIC、DSP、DSPD、PLD、FPGA、CPU、控制器、微控制器、微处理器中的至少一种。可以理解地，对于不同的设备，用于实现上述处理器功能的电子器件还可以为其它，本申请实施例不作具体限定。
本申请实施例还提供了一种计算机程序，包括计算机可读代码，当所述计算机可读 代码在电子设备中运行时，所述电子设备中的处理器执行用于实现上述任意一种路口检测方法或上述任意一种神经网络训练方法或上述任意一种智能行驶测方法。
在一些实施例中，本申请实施例提供的装置具有的功能或包含的模块可以用于执行上文方法实施例描述的方法，其具体实现可以参照上文方法实施例的描述，为了简洁，这里不再赘述。
上文对各个实施例的描述倾向于强调各个实施例之间的不同之处，其相同或相似之处可以互相参考，为了简洁，本文不再赘述
本申请所提供的各方法实施例中所揭露的方法，在不冲突的情况下可以任意组合，得到新的方法实施例。
本申请所提供的各产品实施例中所揭露的特征，在不冲突的情况下可以任意组合，得到新的产品实施例。
本申请所提供的各方法或设备实施例中所揭露的特征，在不冲突的情况下可以任意组合，得到新的方法实施例或设备实施例。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质(如ROM/RAM、磁碟、光盘)中，包括若干指令用以使得一台终端(可以是手机，计算机，服务器，空调器，或者网络设备等)执行本发明各个实施例所述的方法。
上面结合附图对本发明的实施例进行了描述，但是本发明并不局限于上述的具体实施方式，上述的具体实施方式仅仅是示意性的，而不是限制性的，本领域的普通技术人员在本发明的启示下，在不脱离本发明宗旨和权利要求所保护的范围情况下，还可做出很多形式，这些均属于本发明的保护之内。
工业实用性
本申请实施例提供了一种路口检测、神经网络训练及智能行驶方法、装置、电子设备、计算机存储介质和计算机程序，该路口检测方法包括：对道路图像进行特征提取，获得所述道路图像的特征图；根据所述道路图像的特征图，确定所述道路图像所示的道路上的路口的检测框；所述路口的检测框表示路口在所述道路图像中的区域，所述路口的检测框的下边框在所述道路的路面上；根据所述路口的检测框的下边框，确定采集所述道路图像的设备与所述路口之间的距离。如此，即使在无法获取到清晰的红绿灯或地面停止线图像，或路口没有红绿灯或地面停止线的情况下，本申请实施例也可以根据道路图像的特征图实现路口检测，从而确定采集所述道路图像的设备与所述路口之间的距离。

Claims (21)

  1. 一种路口检测方法，所述方法包括：
    对道路图像进行特征提取，获得所述道路图像的特征图；
    根据所述道路图像的特征图，确定所述道路图像所示的道路上的路口的检测框；所述路口的检测框表示路口在所述道路图像中的区域，所述路口的检测框的下边框在所述道路的路面上；
    根据所述路口的检测框的下边框，确定采集所述道路图像的设备与所述路口之间的距离。
  2. 根据权利要求1所述的方法，其中，所述方法还包括：
    根据所述道路图像的特征图，确定所述道路图像所示的道路不存在路口。
  3. 根据权利要求1所述的方法，其中，所述根据所述路口的检测框的下边框，确定采集所述道路图像的设备与所述路口之间的距离，包括：
    根据所述路口的检测框的下边框在所述道路图像中的位置以及所述道路图像的平面和所述道路的路面之间的坐标转换关系，确定所述路口的检测框的下边框在所述道路上的位置；
    根据所述路口的检测框的下边框在所述道路上的位置与采集所述道路图像的设备在所述道路上的位置，得出采集所述道路图像的设备与所述路口之间的距离。
  4. 根据权利要求1至3任一项所述的方法，其中，所述方法由神经网络执行，所述神经网络采用样本图像以及样本图像的标注结果训练得到，所述样本图像的标注结果包括正样本图像所示的道路上的路口的标注框，所述标注框表征路口在所述正样本图像中的位置，且所述标注框的下边框在所述正样本图像所示的道路的路面上。
  5. 一种神经网络训练方法，其中，包括：
    对样本图像进行特征提取，获得所述样本图像的特征图；
    根据所述样本图像的特征图，确定所述样本图像的检测结果；
    根据所述样本图像的标注结果和所述检测结果，调整所述神经网络的网络参数值；
    当所述样本图像为正样本图像时，所述样本图像的标注结果为所述正样本图像所示的道路上的路口的标注框，所述标注框表征路口在所述正样本图像中的位置，且所述正样本图像所示的道路上的路口的标注框的下边框在所述正样本图像所示的道路的路面上。
  6. 根据权利要求5所述的方法，其中，所述正样本图像中包括道路的路口的停止线，所述正样本图像所示的道路上的路口的标注框的下边框与所述停止线对齐。
  7. 根据权利要求5所述的方法，其中，包含同一路口的多个正样本图像中的标注框的高度之差在预设范围内。
  8. 根据权利要求5-7任一所述的方法，其中，当所述样本图像为负样本图像时，所述负样本图像中的道路上不存在路口，所述样本图像的标注结果包括所述负样本图像中不存在标注框。
  9. 一种智能行驶方法，包括：
    获取道路图像；
    根据权利要求1-4任一所述的方法，对所述道路图像进行路口检测；
    根据采集所述道路图像的智能行驶设备与所述路口之间的距离对所述设备进行行驶控制。
  10. 一种路口检测装置，所述装置包括第一提取模块、检测模块和第一确定模块； 其中，
    第一提取模块，配置为对道路图像进行特征提取，获得所述道路图像的特征图；
    检测模块，配置为根据所述道路图像的特征图，确定所述道路图像所示的道路上的路口的检测框；所述路口的检测框表示路口在所述道路图像中的区域，所述路口的检测框的下边框在所述道路的路面上；
    第一确定模块，配置为根据所述路口的检测框的下边框，确定采集所述道路图像的设备与所述路口之间的距离。
  11. 根据权利要求10所述的装置，其中，所述检测模块，还配置为根据所述道路图像的特征图，确定所述道路图像所示的道路不存在路口。
  12. 根据权利要求10所述的装置，其中，所述第一确定模块，配置为根据所述路口的检测框的下边框在所述道路图像中的位置以及所述道路图像的平面和所述道路的路面之间的坐标转换关系，确定所述路口的检测框的下边框在所述道路上的位置；根据所述路口的检测框的下边框在所述道路上的位置与采集所述道路图像的设备在所述道路上的位置，得出采集所述道路图像的设备与所述路口之间的距离。
  13. 根据权利要求10至12任一项所述的装置，其中，所述装置是基于神经网络实现的，所述神经网络采用样本图像以及样本图像的标注结果训练得到，所述样本图像的标注结果包括正样本图像所示的道路上的路口的标注框，所述标注框表征路口在所述正样本图像中的位置，且所述标注框的下边框在所述正样本图像所示的道路的路面上。
  14. 一种神经网络训练装置，所述装置包括：第二提取模块、第二确定模块和调整模块，其中，
    第二提取模块，配置为对样本图像进行特征提取，获得所述样本图像的特征图；
    第二确定模块，配置为根据所述样本图像的特征图，确定所述样本图像的检测结果；
    调整模块，配置为根据所述样本图像的标注结果和所述检测结果，调整所述神经网络的网络参数值；
    当所述样本图像为正样本图像时，所述样本图像的标注结果为所述正样本图像所示的道路上的路口的标注框，所述标注框表征路口在所述正样本图像中的位置，且所述正样本图像所示的道路上的路口的标注框的下边框在所述正样本图像所示的道路的路面上。
  15. 根据权利要求14所述的装置，其中，所述正样本图像中包括道路的路口的停止线，所述正样本图像所示的道路上的路口的标注框的下边框与所述停止线对齐。
  16. 根据权利要求14所述的装置，其中，包含同一路口的多个正样本图像中的标注框的高度之差在预设范围内。
  17. 根据权利要求14至16任一项所述的装置，其中，当所述样本图像为负样本图像时，所述负样本图像中的道路上不存在路口，所述样本图像的标注结果包括所述负样本图像中不存在标注框。
  18. 一种智能行驶装置，所述装置包括：获取模块和处理模块，其中，
    获取模块，配置为获取道路图像；
    处理模块，配置为根据权利要求1-4任一所述的方法，对所述道路图像进行路口检测；根据采集所述道路图像的智能行驶设备与所述路口之间的距离对所述设备进行行驶控制。
  19. 一种电子设备，包括处理器和用于存储能够在处理器上运行的计算机程序的存储器；其中，
    所述处理器配置为运行所述计算机程序以执行权利要求1至4任一项所述的路口检测方法或权利要求5至8任一项所述的神经网络训练方法或权利要求9所述的智能行驶方法。
  20. 一种计算机存储介质，其上存储有计算机程序，该计算机程序被处理器执行时实现权利要求1至4任一项所述的路口检测方法或权利要求5至8任一项所述的神经网络训练方法或权利要求9所述的智能行驶方法。
  21. 一种计算机程序，包括计算机可读代码，当所述计算机可读代码在电子设备中运行时，所述电子设备中的处理器执行用于实现1至4任一项所述的路口检测方法或权利要求5至8任一项所述的神经网络训练方法或权利要求9所述的智能行驶方法。
PCT/CN2020/114095 2019-11-07 2020-09-08 路口检测、神经网络训练及智能行驶方法、装置和设备 WO2021088504A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
KR1020217016327A KR20210082518A (ko) 2019-11-07 2020-09-08 교차로 검출, 신경망 훈련 및 스마트 주행 방법, 장치 및 기기
JP2021532862A JP2022512165A (ja) 2019-11-07 2020-09-08 交差点検出、ニューラルネットワークトレーニング及びインテリジェント走行方法、装置及びデバイス

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911083615.4 2019-11-07
CN201911083615.4A CN112784639A (zh) 2019-11-07 2019-11-07 路口检测、神经网络训练及智能行驶方法、装置和设备

Publications (1)

Publication Number Publication Date
WO2021088504A1 true WO2021088504A1 (zh) 2021-05-14

Family

ID=75747994

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/114095 WO2021088504A1 (zh) 2019-11-07 2020-09-08 路口检测、神经网络训练及智能行驶方法、装置和设备

Country Status (4)

Country Link
JP (1) JP2022512165A (zh)
KR (1) KR20210082518A (zh)
CN (1) CN112784639A (zh)
WO (1) WO2021088504A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113380035A (zh) * 2021-06-16 2021-09-10 山东省交通规划设计院集团有限公司 一种道路交叉口交通量分析方法及系统

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023107002A2 (en) * 2021-12-09 2023-06-15 Grabtaxi Holdings Pte. Ltd. System and method for adaptively predicting a road segment attribute based on a graph indicative of relationship between a road segment and a detection
GB2617866A (en) * 2022-04-21 2023-10-25 Continental Automotive Romania Srl Computer implemented method for training a decision tree model for detecting an intersection, computer implemented method detecting an intersection,

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140267415A1 (en) * 2013-03-12 2014-09-18 Xueming Tang Road marking illuminattion system and method
CN108230817A (zh) * 2017-11-30 2018-06-29 商汤集团有限公司 车辆驾驶模拟方法和装置、电子设备、系统、程序和介质
CN108216229A (zh) * 2017-09-08 2018-06-29 北京市商汤科技开发有限公司 交通工具、道路线检测和驾驶控制方法及装置
CN110059554A (zh) * 2019-03-13 2019-07-26 重庆邮电大学 一种基于交通场景的多支路目标检测方法

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002193025A (ja) * 2000-12-27 2002-07-10 Koito Mfg Co Ltd 車両用前照灯装置
JP4915739B2 (ja) * 2007-05-31 2012-04-11 アイシン・エィ・ダブリュ株式会社 運転支援装置
JP5942822B2 (ja) * 2012-11-30 2016-06-29 富士通株式会社 交差点検出方法および交差点検出システム
KR102628654B1 (ko) * 2016-11-07 2024-01-24 삼성전자주식회사 차선을 표시하는 방법 및 장치
US10008110B1 (en) * 2017-02-16 2018-06-26 Mapbox, Inc. Detecting restrictions on turning paths in digital maps
CN107689157B (zh) * 2017-08-30 2021-04-30 电子科技大学 基于深度学习的交通路口可通行道路规划方法
JP7346401B2 (ja) * 2017-11-10 2023-09-19 エヌビディア コーポレーション 安全で信頼できる自動運転車両のためのシステム及び方法
CN108376235A (zh) * 2018-01-15 2018-08-07 深圳市易成自动驾驶技术有限公司 图像检测方法、装置及计算机可读存储介质
US11107347B2 (en) * 2018-04-27 2021-08-31 Cubic Corporation Adaptively controlling traffic movements for driver safety
CN108596116B (zh) * 2018-04-27 2021-11-05 深圳市商汤科技有限公司 测距方法、智能控制方法及装置、电子设备和存储介质
CN108877267B (zh) * 2018-08-06 2020-11-03 武汉理工大学 一种基于车载单目相机的交叉路口检测方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140267415A1 (en) * 2013-03-12 2014-09-18 Xueming Tang Road marking illuminattion system and method
CN108216229A (zh) * 2017-09-08 2018-06-29 北京市商汤科技开发有限公司 交通工具、道路线检测和驾驶控制方法及装置
CN108230817A (zh) * 2017-11-30 2018-06-29 商汤集团有限公司 车辆驾驶模拟方法和装置、电子设备、系统、程序和介质
CN110059554A (zh) * 2019-03-13 2019-07-26 重庆邮电大学 一种基于交通场景的多支路目标检测方法

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113380035A (zh) * 2021-06-16 2021-09-10 山东省交通规划设计院集团有限公司 一种道路交叉口交通量分析方法及系统
CN113380035B (zh) * 2021-06-16 2022-11-11 山东省交通规划设计院集团有限公司 一种道路交叉口交通量分析方法及系统

Also Published As

Publication number Publication date
KR20210082518A (ko) 2021-07-05
JP2022512165A (ja) 2022-02-02
CN112784639A (zh) 2021-05-11

Similar Documents

Publication Publication Date Title
US10964054B2 (en) Method and device for positioning
CN109271944B (zh) 障碍物检测方法、装置、电子设备、车辆及存储介质
WO2021088504A1 (zh) 路口检测、神经网络训练及智能行驶方法、装置和设备
EP3171292B1 (en) Driving lane data processing method, device, storage medium and apparatus
WO2020103893A1 (zh) 车道线属性检测方法、装置、电子设备及可读存储介质
US10212397B2 (en) Abandoned object detection apparatus and method and system
Ding et al. Fast lane detection based on bird’s eye view and improved random sample consensus algorithm
US20210191397A1 (en) Autonomous vehicle semantic map establishment system and establishment method
CN111967396A (zh) 障碍物检测的处理方法、装置、设备及存储介质
CN111091023A (zh) 一种车辆检测方法、装置及电子设备
CN112487884A (zh) 一种交通违法行为检测方法、装置及计算机可读存储介质
JP2021103160A (ja) 自律走行車意味マップ確立システム及び確立方法
CN112434657A (zh) 飘散运载物检测方法、设备、程序及计算机可读介质
CN115147328A (zh) 三维目标检测方法及装置
WO2021088505A1 (zh) 目标属性检测、神经网络训练及智能行驶方法、装置
CN116228756B (zh) 一种自动驾驶中相机坏点的检测方法及检测系统
CN110111018B (zh) 评估车辆感测能力的方法、装置、电子设备及存储介质
Muniruzzaman et al. Deterministic algorithm for traffic detection in free-flow and congestion using video sensor
Lu et al. Monocular multi-kernel based lane marking detection
CN115565072A (zh) 一种道路垃圾识别和定位方法、装置、电子设备及介质
CN115618602A (zh) 一种车道级场景仿真方法及系统
US11461944B2 (en) Region clipping method and recording medium storing region clipping program
CN113869440A (zh) 图像处理方法、装置、设备、介质及程序产品
CN113435350A (zh) 一种交通标线检测方法、装置、设备和介质
CN111383268A (zh) 车辆间距状态获取方法、装置、计算机设备和存储介质

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 20217016327

Country of ref document: KR

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2021532862

Country of ref document: JP

Kind code of ref document: A

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20883748

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20883748

Country of ref document: EP

Kind code of ref document: A1