CN109977845B - Driving region detection method and vehicle-mounted terminal - Google Patents
Driving region detection method and vehicle-mounted terminal Download PDFInfo
- Publication number
- CN109977845B CN109977845B CN201910218775.9A CN201910218775A CN109977845B CN 109977845 B CN109977845 B CN 109977845B CN 201910218775 A CN201910218775 A CN 201910218775A CN 109977845 B CN109977845 B CN 109977845B
- Authority
- CN
- China
- Prior art keywords
- image
- data
- travelable
- travelable region
- detection model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/04—Inference or reasoning models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
Abstract
The invention provides a drivable area detection method and a vehicle-mounted terminal, wherein the method comprises the following steps: acquiring a first image acquired by a monocular vision sensor; inputting the first image into a travelable region detection model to obtain a travelable region of the first image; and the travelable area detection model is obtained by training according to the second images, the lane lines and the inference model of the obstacles. The embodiment of the invention improves the distance for detecting the travelable area.
Description
Technical Field
The invention relates to the technical field of communication, in particular to a drivable area detection method and a vehicle-mounted terminal.
Background
The technology of Free Space Detection (Free Space Detection) is a key technology applied to a solution of an automatic driving system, and is used for judging a drivable or undrivable state of a certain area so as to assist the driving system in making decisions and judging abnormal driving boundaries. Static/dynamic obstacles (vehicles, pedestrians, and the like) on a road, road edges on two sides of the road, guardrails in the middle of the road, cones/tripods on the road, and the like are all part of the non-drivable area, and the automatic driving system needs to distinguish all the areas influencing driving safety, so the drivable area detection technology is developed accordingly.
In the prior art, a point cloud analysis scheme of a laser radar is generally adopted for detecting a travelable area. The scheme uses the point cloud data generated by the laser radar, and can effectively analyze the drivable state and the non-drivable state of the surrounding area. Due to the limited detection range of the lidar, the detection range of 64-line lidar is usually 60m, so that the lidar is only suitable for low-speed driving scenes. Therefore, the distance that can be detected in the travel area is short in the prior art.
Disclosure of Invention
The embodiment of the invention provides a drivable area detection method and a vehicle-mounted terminal, and aims to solve the problem that the distance for detecting the drivable area is short.
In a first aspect, an embodiment of the present invention provides a travelable area detection method, including:
acquiring a first image acquired by a monocular vision sensor;
inputting the first image into a travelable region detection model to obtain a travelable region of the first image;
and the travelable area detection model is obtained by training according to the second images, the lane lines and the inference model of the obstacles.
In a second aspect, an embodiment of the present invention further provides a vehicle-mounted terminal, which is characterized by including:
the acquisition module is used for acquiring a first image acquired by the monocular vision sensor;
the first input module is used for inputting the first image into a travelable region detection model to obtain a travelable region of the first image;
and the travelable area detection model is obtained by training according to the second images, the lane lines and the inference model of the obstacles.
In a third aspect, an embodiment of the present invention further provides an in-vehicle terminal, including a processor, a memory, and a computer program stored on the memory and operable on the processor, where the computer program, when executed by the processor, implements the steps of the above method for detecting a travelable area.
In a fourth aspect, the embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps of the above-mentioned travelable area detection method.
According to the embodiment of the invention, the travelable area of the first image is obtained through inference by inputting the first image acquired by the monocular vision sensor into the travelable area detection model obtained through the inference model training based on the plurality of second images, the lane line and the obstacle. In this way, since the first image acquired by the monocular vision sensor may include image data at a longer distance, the distance of travelable region detection may be increased. In addition, the second image does not need to be subjected to the marking of the drivable area, so that the data marking cost is effectively reduced.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments of the present invention will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without inventive exercise.
Fig. 1 is a flowchart of a travelable area detection method according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a second image in the travelable region detection method according to the embodiment of the present invention;
fig. 3 is a schematic diagram of performing travelable region detection model training based on a second image in the travelable region detection method according to the embodiment of the present invention;
FIG. 4 is a block diagram of an in-vehicle terminal according to an embodiment of the present invention;
fig. 5 is a second structural diagram of the in-vehicle terminal according to the embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, fig. 1 is a flowchart of a travelable area detection method according to an embodiment of the present invention, as shown in fig. 1, including the following steps:
the drivable area detection method provided by the embodiment of the invention is mainly applied to the vehicle-mounted terminal, and the vehicle-mounted terminal is used for detecting the drivable area of the vehicle.
Specifically, a monocular camera may be mounted on the vehicle, and the monocular vision sensor may be a part of the monocular camera and may acquire an image in front of the vehicle through the monocular vision sensor. The vehicle-mounted terminal can be electrically connected with the monocular vision sensor to acquire a first image acquired by the monocular vision sensor.
and the travelable area detection model is obtained by training according to the second images, the lane lines and the inference model of the obstacles.
The second image may be an image acquired by a monocular vision sensor in advance, and specifically may be a training image used for training an inference model for recognizing lane departure and obstacles, and the second image marks lane lines and obstacles in advance. The travelable region detection model may be trained based on the plurality of second images and the label data in each second image, and a specific training process is described in detail in the following embodiments.
In the embodiment of the invention, the acquired first image can be input into the travelable region detection model in a single-frame mode, after inference of the travelable region detection model, a travelable region detection result in the first image of the current frame can be finally obtained, and control of automatic driving can be assisted based on the travelable region detection result.
According to the embodiment of the invention, the travelable area of the first image is obtained through inference by inputting the first image acquired by the monocular vision sensor into the travelable area detection model obtained through the inference model training based on the plurality of second images, the lane line and the obstacle. In this way, since the first image acquired by the monocular vision sensor may include image data at a longer distance, the distance of travelable region detection may be increased. In addition, the second image does not need to be subjected to the marking of the drivable area, so that the data marking cost is effectively reduced.
Further, the training process of the travelable region detection model may be set according to actual needs, for example, in this embodiment, before the step 101, a process of training the travelable region detection model is further included, specifically including:
inputting the plurality of second images into a reasoning model of a lane line and an obstacle to obtain lane line data and obstacle data of each second image;
determining travelable region labeling data of each second image according to lane line data and obstacle data;
and training to obtain a travelable region detection model based on the second image and travelable region labeling data of the second image.
In an embodiment of the present invention, the lane line data includes coordinate information of a lane line, and the obstacle data includes coordinate information of an obstacle. Specifically, the lane line data and the obstacle data correspond to each second image. The obstacle may include any one or more obstacles such as a person and a vehicle, and is not further limited herein.
In this embodiment, the travelable region labeling data may be obtained by performing subsequent processing according to the lane line data and the obstacle data in the second image. For example, in an optional embodiment, a vehicle driving area may be determined based on the lane line data, an area other than the obstacle area in the vehicle driving area is calculated to obtain a drivable area, the area of the obstacle is determined according to the obstacle data, and the coordinate information of the drivable area is the drivable area labeling data.
After determining the travelable region labeling data of each second image, training may be performed according to the second images and the corresponding travelable region labeling data, and finally, a travelable region detection model is obtained.
In the embodiment, the marking data of the travelable area can be obtained by performing self-supervision learning based on the reasoning model of the lane line and the obstacle, so that the travelable area is not required to be manually marked by a user, and the marking cost is reduced.
Further, based on the above-described embodiment, in the present embodiment, the travelable region detection problem can be converted into a regression problem, that is, the feature map is obtained by extracting the image features using the deep neural network, and then the nearest boundary of the non-travelable region on each column in the feature map is regressed. Specifically, the obtaining of the travelable region detection model based on the second image and the travelable region labeling data training of the second image includes:
extracting image features of the second image by using a deep neural network to obtain a feature map;
performing regression processing on the nearest boundary point of the non-driving area on each column in the feature map based on the marking data of the driving area of the second image and the loss function correspondingly set for each dimension of the feature map to obtain the driving area detection model;
wherein the loss functions include boundary point position loss functions, boundary point position offset loss functions, and boundary point class loss functions.
In this embodiment, the feature map is a three-dimensional feature map, and in other embodiments, feature maps with more dimensions may be further set, where each dimension sub-table is correspondingly set with different loss functions. For example, a first dimension may set a boundary point position penalty function, a second dimension may set a boundary point position offset penalty function, and a third dimension may set a boundary point class penalty function. The boundary point position loss function and the boundary point position offset loss function can be calculated by adopting Euclidean distance, and the boundary point type loss function can be calculated by using Sigmoid cross entropy. The boundary point position loss function is used for calculating free space point loss (free space point loss), the boundary point position offset loss function is used for calculating free space point offset loss (free space point offset loss), and the boundary point position offset loss function is used for calculating free space point class loss (free space point class loss).
Fig. 2 is a schematic diagram of a second image, as shown in fig. 2. In fig. 2, the lower edge line of fig. 2 is the X axis, and the right side of the left edge line is the positive axis; the left edge line is the Y-axis and the upper side of the lower edge line is the positive axis. The above-mentioned nearest boundary point of the no-travel area on each row means a coordinate point closest to a target coordinate point of the row among the no-travel area coordinate points of each row, and the target coordinate point is a coordinate point whose Y axis takes a value of 0.
Since the boundary point class loss function is set in the present embodiment, it is possible to define that the loss weights of the boundaries of the regions of different classes are different, thereby increasing the accuracy of detection of the travelable region. As shown in fig. 2, the lane lines, the obstacles a, and the obstacles B are classified. If the lane line is marked as a first type boundary line, the boundary of the obstacle A is marked as a second type boundary line, and the boundary of the obstacle C is marked as a third type boundary line. The class penalty function may have different penalty weights for different classes of boundary lines at the boundary points.
As shown in fig. 3, the second image 301 may be a 1920 × 640 image, which is passed through the full convolution neural network 302 to obtain a 120 × 40 × 3 feature map 303. Wherein the characteristic map sets corresponding loss functions for different dimensions. For example, the first dimension 3041 of the feature map is set with a boundary point position loss function, the second dimension 3042 of the feature map is set with a boundary point position offset loss function, and the first dimension 3043 of the feature map is set with a boundary point class loss function.
Further, the first image may be a complete or cut image of one frame acquired by the monocular vision sensor, and in order to increase the speed of the travelable region detection model for calculating the travelable region, in this embodiment, it is preferable that the cut image acquired by the monocular vision sensor is used as the first image. Specifically, the acquiring of the first image acquired by the monocular vision sensor includes:
acquiring an initial image acquired by a monocular vision sensor;
and selecting and cutting a Region Of Interest (ROI) Of the initial image to obtain the first image.
In the present embodiment, the ratio of cropping the initial image may be set according to the actual situation, and may be 1/2, for example, that is, the ratio of the size of the initial image to the size of the first image is 2: 1. Specifically, the ratio of the length to the width of the cropped first image to the initial image may be the same.
For example, the second image may be 960 × 320 image, which is subjected to a full convolution neural network to obtain a 60 × 20 × 3 feature map. Since the input image is cropped to 1/2 of the original image, the deconvolution operation of the feature extraction network can be reduced, and thus the speed of travelable region calculation can be increased.
It should be noted that, various optional implementations described in the embodiments of the present invention may be implemented in combination with each other or implemented separately, and the embodiments of the present invention are not limited thereto.
Referring to fig. 4, fig. 4 is a structural diagram of a vehicle-mounted terminal according to an embodiment of the present invention, and as shown in fig. 4, the vehicle-mounted terminal 400 includes:
an obtaining module 401, configured to obtain a first image acquired by a monocular vision sensor;
a first input module 402, configured to input the first image into a travelable region detection model, so as to obtain a travelable region of the first image;
and the travelable area detection model is obtained by training according to the second images, the lane lines and the inference model of the obstacles.
Optionally, the vehicle-mounted terminal 400 further includes:
the second input module is used for inputting the second images into a reasoning model of a lane line and an obstacle to obtain lane line data and obstacle data of each second image;
the determining module is used for determining the travelable region marking data of each second image according to the lane line data and the obstacle data;
and the training module is used for training to obtain a travelable region detection model based on the second image and travelable region labeling data of the second image.
Optionally, the training module includes:
the extraction unit is used for extracting the image characteristics of the second image by using a deep neural network to obtain a characteristic map;
the processing unit is used for carrying out regression processing on the nearest boundary point of the non-driving area on each column in the feature map based on the marking data of the driving area of the second image and the loss function correspondingly set for each dimension of the feature map to obtain the driving area detection model;
wherein the loss functions include boundary point position loss functions, boundary point position offset loss functions, and boundary point class loss functions.
Optionally, the obtaining module 401 includes:
the acquisition unit is used for acquiring an initial image acquired by the monocular vision sensor;
and the cutting unit is used for carrying out ROI (region of interest) selection and cutting on the initial image to obtain the first image.
Optionally, a ratio of the size of the initial image to the size of the first image is 2: 1.
The vehicle-mounted terminal provided by the embodiment of the invention can realize each process realized by the vehicle-mounted terminal in the method embodiments of fig. 1 to fig. 3, and is not described again to avoid repetition.
Fig. 5 is a schematic diagram of a hardware structure of a vehicle-mounted terminal for implementing various embodiments of the present invention.
The in-vehicle terminal 500 includes, but is not limited to: a radio frequency unit 501, a network module 502, an audio output unit 503, an input unit 504, a sensor 505, a display unit 506, a user input unit 507, an interface unit 508, a memory 509, a processor 510, and a power supply 511. Those skilled in the art will appreciate that the in-vehicle terminal structure shown in fig. 5 does not constitute a limitation of the in-vehicle terminal, and the in-vehicle terminal may include more or less components than those shown, or combine some components, or a different arrangement of components.
The processor 510 is configured to acquire a first image acquired by a monocular vision sensor; inputting the first image into a travelable region detection model to obtain a travelable region of the first image; and the travelable area detection model is obtained by training according to the second images, the lane lines and the inference model of the obstacles.
Optionally, the processor 510 is further configured to:
inputting the plurality of second images into a reasoning model of a lane line and an obstacle to obtain lane line data and obstacle data of each second image;
determining travelable region labeling data of each second image according to lane line data and obstacle data;
and training to obtain a travelable region detection model based on the second image and travelable region labeling data of the second image.
Optionally, the processor 510 is specifically configured to:
extracting image features of the second image by using a deep neural network to obtain a feature map;
performing regression processing on the nearest boundary point of the non-driving area on each column in the feature map based on the marking data of the driving area of the second image and the loss function correspondingly set for each dimension of the feature map to obtain the driving area detection model;
wherein the loss functions include boundary point position loss functions, boundary point position offset loss functions, and boundary point class loss functions.
Optionally, the processor 510 is specifically configured to:
acquiring an initial image acquired by a monocular vision sensor;
and performing ROI (region of interest) selection and cutting on the initial image to obtain the first image.
Optionally, a ratio of the size of the initial image to the size of the first image is 2: 1.
According to the embodiment of the invention, the travelable area of the first image is obtained through inference by inputting the first image acquired by the monocular vision sensor into the travelable area detection model obtained through the inference model training based on the plurality of second images, the lane line and the obstacle. In this way, since the first image acquired by the monocular vision sensor may include image data at a longer distance, the distance of travelable region detection may be increased. In addition, the second image does not need to be subjected to the marking of the drivable area, so that the data marking cost is effectively reduced.
It should be understood that, in the embodiment of the present invention, the radio frequency unit 501 may be used for receiving and sending signals during a message sending and receiving process or a call process, and specifically, receives downlink data from a base station and then processes the received downlink data to the processor 510; in addition, the uplink data is transmitted to the base station. In general, radio frequency unit 501 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like. In addition, the radio frequency unit 501 can also communicate with a network and other devices through a wireless communication system.
The in-vehicle terminal provides wireless broadband internet access to the user through the network module 502, such as helping the user send and receive e-mails, browse webpages, access streaming media, and the like.
The audio output unit 503 may convert audio data received by the radio frequency unit 501 or the network module 502 or stored in the memory 509 into an audio signal and output as sound. Also, the audio output unit 503 may also provide audio output related to a specific function performed by the in-vehicle terminal 500 (e.g., a call signal reception sound, a message reception sound, etc.). The audio output unit 503 includes a speaker, a buzzer, a receiver, and the like.
The input unit 504 is used to receive an audio or video signal. The input Unit 504 may include a Graphics Processing Unit (GPU) 5041 and a microphone 5042, and the Graphics processor 5041 processes image data of a still picture or video obtained by an image capturing device (e.g., a camera) in a video capturing mode or an image capturing mode. The processed image frames may be displayed on the display unit 506. The image frames processed by the graphic processor 5041 may be stored in the memory 509 (or other storage medium) or transmitted via the radio frequency unit 501 or the network module 502. The microphone 5042 may receive sounds and may be capable of processing such sounds into audio data. The processed audio data may be converted into a format output transmittable to a mobile communication base station via the radio frequency unit 501 in case of the phone call mode.
The in-vehicle terminal 500 further includes at least one sensor 505, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor that adjusts the brightness of the display panel 5061 according to the brightness of ambient light, and a proximity sensor that turns off the display panel 5061 and/or a backlight when the in-vehicle terminal 500 is moved to the ear. As one of the motion sensors, the accelerometer sensor can detect the magnitude of acceleration in each direction (generally three axes), detect the magnitude and direction of gravity when stationary, and can be used to identify the vehicle-mounted terminal attitude (such as horizontal and vertical screen switching, related games, magnetometer attitude calibration), vibration identification related functions (such as pedometer, tapping), and the like; the sensors 505 may also include fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., which are not described in detail herein.
The display unit 506 is used to display information input by the user or information provided to the user. The Display unit 506 may include a Display panel 5061, and the Display panel 5061 may be configured in the form of a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), or the like.
The user input unit 507 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the in-vehicle terminal. Specifically, the user input unit 507 includes a touch panel 5071 and other input devices 5072. Touch panel 5071, also referred to as a touch screen, may collect touch operations by a user on or near it (e.g., operations by a user on or near touch panel 5071 using a finger, stylus, or any suitable object or attachment). The touch panel 5071 may include two parts of a touch detection device and a touch controller. The touch detection device detects the touch direction of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch sensing device, converts the touch information into touch point coordinates, sends the touch point coordinates to the processor 510, and receives and executes commands sent by the processor 510. In addition, the touch panel 5071 may be implemented in various types such as a resistive type, a capacitive type, an infrared ray, and a surface acoustic wave. In addition to the touch panel 5071, the user input unit 507 may include other input devices 5072. In particular, other input devices 5072 may include, but are not limited to, a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which are not described in detail herein.
Further, the touch panel 5071 may be overlaid on the display panel 5061, and when the touch panel 5071 detects a touch operation thereon or nearby, the touch operation is transmitted to the processor 510 to determine the type of the touch event, and then the processor 510 provides a corresponding visual output on the display panel 5061 according to the type of the touch event. Although in fig. 5, the touch panel 5071 and the display panel 5061 are two independent components to implement the input and output functions of the vehicle-mounted terminal, in some embodiments, the touch panel 5071 and the display panel 5061 may be integrated to implement the input and output functions of the vehicle-mounted terminal, and is not limited herein.
The interface unit 508 is an interface for connecting an external device to the in-vehicle terminal 500. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device having an identification module, an audio input/output (I/O) port, a video I/O port, an earphone port, and the like. The interface unit 508 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the in-vehicle terminal 500 or may be used to transmit data between the in-vehicle terminal 500 and the external device.
The memory 509 may be used to store software programs as well as various data. The memory 509 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the cellular phone, and the like. Further, the memory 509 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device.
The processor 510 is a control center of the in-vehicle terminal, connects various parts of the entire in-vehicle terminal by various interfaces and lines, and performs various functions of the in-vehicle terminal and processes data by operating or executing software programs and/or modules stored in the memory 509 and calling data stored in the memory 509, thereby performing overall monitoring of the in-vehicle terminal. Processor 510 may include one or more processing units; preferably, the processor 510 may integrate an application processor, which mainly handles operating systems, user interfaces, application programs, etc., and a modem processor, which mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into processor 510.
The in-vehicle terminal 500 may further include a power supply 511 (such as a battery) for supplying power to each component, and preferably, the power supply 511 may be logically connected to the processor 510 through a power management system, so that functions of managing charging, discharging, and power consumption are implemented through the power management system.
In addition, the in-vehicle terminal 500 includes some functional modules that are not shown, and are not described in detail herein.
Preferably, an embodiment of the present invention further provides a vehicle-mounted terminal, which includes a processor 510, a memory 509, and a computer program that is stored in the memory 509 and can be run on the processor 510, and when the computer program is executed by the processor 510, the processes of the above-mentioned drivable area detection method embodiment are implemented, and the same technical effects can be achieved, and in order to avoid repetition, details are not described here again.
An embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program implements each process of the above-mentioned method for detecting a drivable area, and can achieve the same technical effect, and in order to avoid repetition, details are not repeated here. The computer-readable storage medium may be a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment of the present invention.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a U disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (8)
1. A travelable region detection method, comprising:
acquiring a first image acquired by a monocular vision sensor;
inputting the first image into a travelable region detection model to obtain a travelable region of the first image;
the travelable area detection model is obtained by training according to a plurality of second images and inference models of lane lines and obstacles;
the method further comprises, before the acquiring the first image acquired based on the monocular vision sensor:
inputting the plurality of second images into a reasoning model of a lane line and an obstacle to obtain lane line data and obstacle data of each second image;
determining travelable region labeling data of each second image according to lane line data and obstacle data;
training to obtain a travelable region detection model based on the second image and travelable region labeling data of the second image;
the training of the travelable region labeling data based on the second image and the second image to obtain a travelable region detection model comprises:
extracting image features of the second image by using a deep neural network to obtain a feature map;
performing regression processing on the nearest boundary point of the non-driving area on each column in the feature map based on the marking data of the driving area of the second image and the loss function correspondingly set for each dimension of the feature map to obtain the driving area detection model;
wherein the loss functions include boundary point position loss functions, boundary point position offset loss functions, and boundary point class loss functions.
2. The method of claim 1, wherein said acquiring a first image acquired by a monocular vision sensor comprises:
acquiring an initial image acquired by a monocular vision sensor;
and performing ROI (region of interest) selection and cutting on the initial image to obtain the first image.
3. The method of claim 2, wherein a ratio of the size of the initial image to the size of the first image is 2: 1.
4. A vehicle-mounted terminal characterized by comprising:
the acquisition module is used for acquiring a first image acquired by the monocular vision sensor;
the first input module is used for inputting the first image into a travelable region detection model to obtain a travelable region of the first image;
the travelable area detection model is obtained by training according to a plurality of second images and inference models of lane lines and obstacles;
the vehicle-mounted terminal further includes:
the second input module is used for inputting the second images into a reasoning model of a lane line and an obstacle to obtain lane line data and obstacle data of each second image;
the determining module is used for determining the travelable region marking data of each second image according to the lane line data and the obstacle data;
the training module is used for training to obtain a travelable region detection model based on the second image and travelable region labeling data of the second image;
the training module comprises:
the extraction unit is used for extracting the image characteristics of the second image by using a deep neural network to obtain a characteristic map;
the processing unit is used for carrying out regression processing on the nearest boundary point of the non-driving area on each column in the feature map based on the marking data of the driving area of the second image and the loss function correspondingly set for each dimension of the feature map to obtain the driving area detection model;
wherein the loss functions include boundary point position loss functions, boundary point position offset loss functions, and boundary point class loss functions.
5. The vehicle-mounted terminal according to claim 4, wherein the obtaining module comprises:
the acquisition unit is used for acquiring an initial image acquired by the monocular vision sensor;
and the cutting unit is used for carrying out ROI (region of interest) selection and cutting on the initial image to obtain the first image.
6. The in-vehicle terminal according to claim 5, wherein a ratio of a size of the initial image to a size of the first image is 2: 1.
7. A vehicle-mounted terminal characterized by comprising a processor, a memory and a computer program stored on the memory and executable on the processor, the computer program, when executed by the processor, implementing the steps of the travelable area detection method according to any of claims 1 to 3.
8. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the travelable region detection method according to any of claims 1-3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910218775.9A CN109977845B (en) | 2019-03-21 | 2019-03-21 | Driving region detection method and vehicle-mounted terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910218775.9A CN109977845B (en) | 2019-03-21 | 2019-03-21 | Driving region detection method and vehicle-mounted terminal |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109977845A CN109977845A (en) | 2019-07-05 |
| CN109977845B true CN109977845B (en) | 2021-08-17 |
Family
ID=67079862
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910218775.9A Active CN109977845B (en) | 2019-03-21 | 2019-03-21 | Driving region detection method and vehicle-mounted terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109977845B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111105590A (en) * | 2019-12-27 | 2020-05-05 | 深圳前海微众银行股份有限公司 | Alarm method and device |
| CN111231977B (en) * | 2020-01-14 | 2022-04-08 | 浙江吉利汽车研究院有限公司 | Vehicle speed determination method and device, vehicle and storage medium |
| CN111552289B (en) * | 2020-04-28 | 2021-07-06 | 苏州高之仙自动化科技有限公司 | Detection method, virtual radar device, electronic apparatus, and storage medium |
| CN113705272A (en) * | 2020-05-20 | 2021-11-26 | 华为技术有限公司 | Method, device, equipment and storage medium for detecting travelable area |
| CN112793567A (en) * | 2021-01-14 | 2021-05-14 | 史鹏飞 | Driving assistance method and system based on road condition detection |
| CN113052009A (en) * | 2021-03-02 | 2021-06-29 | 北京经纬恒润科技股份有限公司 | Vehicle anti-collision method, model training method and device |
| CN116052122B (en) * | 2023-01-28 | 2023-06-27 | 广汽埃安新能源汽车股份有限公司 | Method and device for detecting drivable space, electronic equipment and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103679127A (en) * | 2012-09-24 | 2014-03-26 | 株式会社理光 | Method and device for detecting drivable area of road pavement |
| CN107389084A (en) * | 2017-06-09 | 2017-11-24 | 深圳市速腾聚创科技有限公司 | Planning driving path planing method and storage medium |
| CN107545263A (en) * | 2017-08-02 | 2018-01-05 | 清华大学 | A kind of object detecting method and device |
| CN108345875A (en) * | 2018-04-08 | 2018-07-31 | 北京初速度科技有限公司 | Wheeled region detection model training method, detection method and device |
| CN108470138A (en) * | 2018-01-24 | 2018-08-31 | 博云视觉（北京）科技有限公司 | Method for target detection and device |
| CN109117690A (en) * | 2017-06-23 | 2019-01-01 | 百度在线网络技术（北京）有限公司 | Drivable region detection method, device, equipment and storage medium |
| CN109472309A (en) * | 2018-11-12 | 2019-03-15 | 南京烽火星空通信发展有限公司 | A kind of X-ray screening machine picture object detecting method |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP6637399B2 (en) * | 2016-09-30 | 2020-01-29 | 株式会社デンソー | Area recognition apparatus and area recognition method |
| CN106485233B (en) * | 2016-10-21 | 2020-01-17 | 深圳地平线机器人科技有限公司 | Method and device for detecting travelable area and electronic equipment |
| CN109117691A (en) * | 2017-06-23 | 2019-01-01 | 百度在线网络技术（北京）有限公司 | Drivable region detection method, device, equipment and storage medium |
| CN108197669B (en) * | 2018-01-31 | 2021-04-30 | 北京陌上花科技有限公司 | Feature training method and device of convolutional neural network |
| CN109344921B (en) * | 2019-01-03 | 2019-04-23 | 湖南极点智能科技有限公司 | A kind of image-recognizing method based on deep neural network model, device and equipment |
-
2019
- 2019-03-21 CN CN201910218775.9A patent/CN109977845B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103679127A (en) * | 2012-09-24 | 2014-03-26 | 株式会社理光 | Method and device for detecting drivable area of road pavement |
| CN107389084A (en) * | 2017-06-09 | 2017-11-24 | 深圳市速腾聚创科技有限公司 | Planning driving path planing method and storage medium |
| CN109117690A (en) * | 2017-06-23 | 2019-01-01 | 百度在线网络技术（北京）有限公司 | Drivable region detection method, device, equipment and storage medium |
| CN107545263A (en) * | 2017-08-02 | 2018-01-05 | 清华大学 | A kind of object detecting method and device |
| CN108470138A (en) * | 2018-01-24 | 2018-08-31 | 博云视觉（北京）科技有限公司 | Method for target detection and device |
| CN108345875A (en) * | 2018-04-08 | 2018-07-31 | 北京初速度科技有限公司 | Wheeled region detection model training method, detection method and device |
| CN109472309A (en) * | 2018-11-12 | 2019-03-15 | 南京烽火星空通信发展有限公司 | A kind of X-ray screening machine picture object detecting method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109977845A (en) | 2019-07-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109977845B (en) | Driving region detection method and vehicle-mounted terminal | |
| CN107749194B (en) | Lane changing assisting method and mobile terminal | |
| WO2016041340A1 (en) | An indication method and mobile terminal | |
| CN107977652B (en) | Method for extracting screen display content and mobile terminal | |
| CN108153422B (en) | Display object control method and mobile terminal | |
| CN108846663B (en) | Two-dimensional code adjusting method and device and mobile terminal | |
| CN107826109B (en) | Lane keeping method and apparatus | |
| CN109495641B (en) | Reminding method and mobile terminal | |
| CN110110571B (en) | Code scanning method and mobile terminal | |
| CN110807405A (en) | Detection method of candid camera device and electronic equipment | |
| CN109784234B (en) | Right-angled bend identification method based on forward fisheye lens and vehicle-mounted equipment | |
| CN109993234B (en) | Unmanned driving training data classification method and device and electronic equipment | |
| CN112927539B (en) | Mapping method and device for automatic parking | |
| CN109657643A (en) | A kind of image processing method and device | |
| CN107798662B (en) | Image processing method and mobile terminal | |
| CN111432122B (en) | Image processing method and electronic equipment | |
| CN109711477B (en) | Automatic driving model training method and device | |
| CN109189517B (en) | Display switching method and mobile terminal | |
| CN107730030B (en) | Path planning method and mobile terminal | |
| CN110046569B (en) | Unmanned driving data processing method and device and electronic equipment | |
| CN108550182B (en) | Three-dimensional modeling method and terminal | |
| CN110647892A (en) | Violation alarm prompting method, terminal and computer readable storage medium | |
| CN112464831B (en) | Video classification method, training method of video classification model and related equipment | |
| CN110126829B (en) | Torque filter coefficient determining method, vehicle-mounted terminal and vehicle | |
| CN109855643B (en) | Lane guiding method and navigation equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |