CN115661238B - Method and device for generating travelable region, electronic equipment and computer readable medium - Google Patents
Method and device for generating travelable region, electronic equipment and computer readable medium Download PDFInfo
- Publication number
- CN115661238B CN115661238B CN202211553876.XA CN202211553876A CN115661238B CN 115661238 B CN115661238 B CN 115661238B CN 202211553876 A CN202211553876 A CN 202211553876A CN 115661238 B CN115661238 B CN 115661238B
- Authority
- CN
- China
- Prior art keywords
- driving
- driving scene
- marginal
- scene
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Abstract
The embodiment of the disclosure discloses a travelable region generation method, a travelable region generation device, an electronic device and a computer-readable medium. One embodiment of the method comprises: receiving driving scene data of a target vehicle and a driving scene label set corresponding to the driving scene data; inputting a driving scene image sequence and a driving scene label set included in driving scene data into a pre-trained edge driving scene generation model to obtain an edge driving scene; inputting at least one edge driving image included in an edge driving scene into a driving image recognition model trained in advance to obtain a driving image recognition result; responding to the driving image recognition result to represent that the target vehicle is in an abnormal state, and acquiring the driving data of the target vehicle; and inputting the driving data of the target vehicle and at least one marginal driving image included in the marginal driving scene into the self-defined model to obtain a travelable area. This embodiment improves the safety of the vehicle running.
Description
Technical Field
The embodiment of the disclosure relates to the technical field of computers, in particular to a travelable area generation method and device, electronic equipment and a computer-readable medium.
Background
When determining a travelable region of a vehicle, a travelable region generation model is usually trained by using a large number of marginal driving scenes, and how to generate marginal driving scenes becomes an important research topic. At present, when an edge driving scene is generated, the method generally adopts the following steps: and (3) realizing an edge driving scene by using a 3D simulation technology, and training a travelable area generation model by using the edge driving scene.
However, when the edge driving scene is generated in the above manner, there are often the following technical problems:
firstly, a large difference exists between an edge driving scene realized by using a 3D simulation technology and a real edge driving scene, so that an obstacle exists in a travelable region generated by a trained travelable region generation model, and the safety of vehicle travel is low.
Secondly, when a fixed tag set is used as a drive to generate an edge driving scene, all possible edge driving scenes cannot be generated, so that a trained travelable region generating model cannot accurately identify a traveling environment when a vehicle travels, and the traveling safety of the vehicle is low.
Thirdly, when the pre-trained diffusion model is used to generate the edge driving scene, the generated edge driving scene needs to be a high-resolution image or video, so that a function with high complexity needs to be used in the diffusion model, and a long time needs to be consumed to generate the edge driving scene.
Disclosure of Invention
This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
Some embodiments of the present disclosure propose a travelable region generation method, apparatus, electronic device, and computer-readable medium to solve one or more of the technical problems set forth in the background section above.
In a first aspect, some embodiments of the present disclosure provide a travelable region generation method, including: receiving driving scene data of a target vehicle and a driving scene label set corresponding to the driving scene data, wherein the driving scene data comprises a driving scene image sequence, and the driving scene image sequence comprises at least one driving scene image; inputting a driving scene image sequence and the driving scene label set included in the driving scene data into a pre-trained edge driving scene generation model to obtain an edge driving scene, wherein the edge driving scene includes at least one edge driving image; inputting at least one edge driving image included in the edge driving scene into a driving image recognition model trained in advance to obtain a driving image recognition result; responding to the driving image recognition result to represent that the target vehicle is in an abnormal state, and acquiring the driving data of the target vehicle; and inputting the driving data of the target vehicle and at least one marginal driving image included in the marginal driving scene into a user-defined model to obtain a travelable area.
In a second aspect, some embodiments of the present disclosure provide a travelable region generating apparatus, the apparatus comprising: a receiving unit configured to receive driving scene data of a target vehicle and a driving scene tag set corresponding to the driving scene data, wherein the driving scene data comprises a driving scene image sequence including at least one driving scene image; a first input unit configured to input a driving scene image sequence and the driving scene label set included in the driving scene data into a pre-trained marginal driving scene generation model to obtain a marginal driving scene, wherein the marginal driving scene includes at least one marginal driving image; the second input unit is configured to input at least one edge driving image included in the edge driving scene into a driving image recognition model trained in advance to obtain a driving image recognition result; an acquisition unit configured to acquire driving data of the target vehicle in response to the driving image recognition result representing that the target vehicle is in an abnormal state; and the third input unit is configured to input the driving data of the target vehicle and at least one marginal driving image included in the marginal driving scene into a custom model to obtain a drivable area.
In a third aspect, some embodiments of the present disclosure provide an electronic device, comprising: one or more processors; a storage device having one or more programs stored thereon, which when executed by one or more processors, cause the one or more processors to implement the method described in any of the implementations of the first aspect.
In a fourth aspect, some embodiments of the present disclosure provide a computer readable medium on which a computer program is stored, wherein the program, when executed by a processor, implements the method described in any of the implementations of the first aspect.
The above embodiments of the present disclosure have the following beneficial effects: the above embodiments of the present disclosure have the following advantages: by the travelable region generation method of some embodiments of the present disclosure, the safety of vehicle travel is improved. Specifically, the reason why the safety of the vehicle running is low is that: the edge driving scene realized by using the 3D simulation technology has a large difference with the real edge driving scene, so that obstacles exist in the travelable region generated by the trained travelable region generating model, and the safety of vehicle traveling is low. Based on this, the travelable region generation method of some embodiments of the present disclosure, first, receives driving scene data of a target vehicle and a driving scene tag set corresponding to the driving scene data. Thus, data support can be provided for generating the edge driving scene. Secondly, inputting the driving scene image sequence and the driving scene label set included in the driving scene data into a pre-trained marginal driving scene generation model to obtain a marginal driving scene. Therefore, the driving scene label can be used as a drive to guide the construction of the edge driving scene, and the real edge driving scene is generated by using the real image, so that the driving-capable area generating model can be trained by using the generated real edge driving scene, the trained driving-capable area generating model can accurately generate the driving-capable area of the vehicle, and the driving safety of the vehicle is improved. Then, inputting at least one edge driving image included in the edge driving scene into a driving image recognition model trained in advance to obtain a driving image recognition result; and acquiring the driving data of the target vehicle in response to the driving image recognition result representing that the target vehicle is in an abnormal state. Thus, data support is provided for generating the travelable region. And finally, inputting the driving data of the target vehicle and at least one marginal driving image included in the marginal driving scene into a user-defined model to obtain a drivable area. Therefore, the driving feasible region of the vehicle can be accurately generated by the trained driving feasible region generating model through the generated real edge driving scene, and the driving safety of the vehicle is improved.
Drawings
The above and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. It should be understood that the drawings are schematic and that elements and elements are not necessarily drawn to scale.
FIG. 1 is a flow diagram of some embodiments of a travelable region generation method according to the present disclosure;
FIG. 2 is a schematic block diagram of some embodiments of a travelable region generation apparatus according to the present disclosure;
FIG. 3 is a schematic block diagram of an electronic device suitable for use in implementing some embodiments of the present disclosure.
Detailed Description
Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is to be understood that the disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and the embodiments of the disclosure are for illustration purposes only and are not intended to limit the scope of the disclosure.
It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings. The embodiments and features of the embodiments in the present disclosure may be combined with each other without conflict.
It should be noted that the terms "first", "second", and the like in the present disclosure are only used for distinguishing different devices, modules or units, and are not used for limiting the order or interdependence relationship of the functions performed by the devices, modules or units.
It is noted that references to "a" or "an" in this disclosure are intended to be illustrative rather than limiting, and that those skilled in the art will appreciate that references to "one or more" are intended to be exemplary and not limiting unless the context clearly indicates otherwise.
The names of messages or information exchanged between devices in the embodiments of the present disclosure are for illustrative purposes only, and are not intended to limit the scope of the messages or information.
The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
Fig. 1 illustrates a flow 100 of some embodiments of a travelable region generation method according to the present disclosure. The travelable region generation method comprises the following steps:
In some embodiments, an executive of the travelable region generation method may receive driving scenario data for a target vehicle and a set of driving scenario tags corresponding to the driving scenario data. In practice, the execution subject may receive driving scene data of the target vehicle and a driving scene tag set corresponding to the driving scene data, which are sent by the associated user terminal. The associated user terminal may be a terminal having a marginal driving scene generation authority, which is connected to the execution main body in a wired or wireless manner. The driving scene data includes a driving scene image sequence including at least one driving scene image. The driving scene image in the driving scene image sequence may be the driving scene image captured by the driving scene capturing device connected to the target vehicle by wire or infinitely. The driving scene collecting device may be a device for collecting driving scene information. For example, the driving scene capture device may be a camera or a video camera. The target vehicle may be a vehicle for driving scene acquisition. The driving scene tag in the driving scene tag set may be a preset tag for characterizing a certain feature in the driving scene data. For example, the driving scenario tags may include, but are not limited to: cement road surface, expressway.
Optionally, the driving scenario tag set may be generated by:
firstly, receiving label category information of a target driving scene.
In some embodiments, the execution main body block may receive target driving scene tag category information sent by the target terminal. The target terminal may be a terminal of a worker connected to the execution main body by wire or wirelessly. The target driving scene label category information may be used to represent a category of a driving scene label corresponding to the driving scene data. For example, the target driving scenario tag category information may be, but is not limited to, an environment category and a driving road surface category.
And secondly, selecting preset driving scene labels corresponding to the target driving scene label category information in preset label quantity from a preset driving scene label set as a target driving scene label set.
In some embodiments, the executing entity may randomly select, from a preset driving scene tag set, each preset driving scene tag corresponding to the target driving scene tag category information, which is a preset number of tags, as the target driving scene tag set. The preset driving scene tag in the preset driving scene tag set may be a preset driving scene tag. The preset driving scene label corresponds to the driving scene label category information. The number of the preset tags may be a preset number of selected preset driving scene tags.
And thirdly, sequencing all target driving scene labels included in the target driving scene label set according to a preset sequencing mode to generate a target driving scene label sequence.
In some embodiments, the executing subject may sort the target driving scene tags included in the target driving scene tag set in a preset sorting manner, so as to generate a target driving scene tag sequence. The preset sorting mode may be sorting the target driving scene labels according to a generated time sequence.
And fourthly, determining whether the target driving scene label sequence meets a preset label generation condition.
In some embodiments, the executing entity may determine whether the target driving scenario tag sequence satisfies a preset tag generation condition. The preset tag survival condition may be that a markov one-time transition probability of each adjacent two target driving scene tags in the target driving scene tag sequence is greater than or equal to a preset markov one-time transition probability, and a markov chain probability of the target driving scene tag sequence is greater than or equal to the preset markov chain probability. In practice, the Markov chain probability and Markov one-time transition probability may be determined using state transition probability formulas. The preset markov one-time transition probability may be a preset markov one-time transition probability of two target driving scene tags. For example, the preset markov one-time transition probability may be 85%. The preset markov chain probability may be a preset markov chain probability of a tag sequence of a target driving scene. For example, the above-mentioned preset markov chain probability may be 60%.
And fifthly, in response to the fact that the target driving scene label sequence meets the preset label generation condition, determining the target driving scene label sequence as a driving scene label set.
In some embodiments, the executing entity may determine the target driving scenario tag sequence as the driving scenario tag set in response to determining that the target driving scenario tag sequence satisfies the preset tag generation condition.
And sixthly, training a travelable region generation model by using the determined driving scene label set.
In some embodiments, the executive body may train the travelable region generation model using the determined driving scenario label set.
The technical content in the first step to the fifth step is taken as an invention point of the present disclosure, and a second technical problem mentioned in the background art is solved, namely that "when a fixed tag set is used as a driver to generate an edge driving scene, all possible edge driving scenes cannot be generated, so that a trained travelable region generation model cannot accurately identify a traveling environment when a vehicle travels, and thus the safety of vehicle traveling is low". Factors that cause low safety in vehicle travel tend to be as follows: when a fixed label set is used as a drive to generate an edge driving scene, all possible edge driving scenes cannot be generated, so that a trained travelable region generation model cannot accurately identify a traveling environment when a vehicle travels, and the safety of vehicle traveling is low. If the above-mentioned factors are solved, the effect of improving the safety of the vehicle running can be achieved. To achieve this, first, target driving scene tag category information is received; and selecting each preset driving scene label corresponding to the target driving scene label category information from a preset driving scene label set as a target driving scene label set. Therefore, all driving scene labels corresponding to the driving scene label category can be selected. Secondly, sequencing all target driving scene labels included in the target driving scene label set according to a preset sequencing mode to generate a target driving scene label sequence; and determining whether the target driving scene label sequence meets a preset label generation condition. Therefore, whether the selected driving scene labels meet the conditions or not can be determined by determining the Markov probability, and each driving scene label meeting the conditions is determined as a driving scene label set. Finally, in response to the fact that the target driving scene label sequence meets the preset label generation condition, determining the target driving scene label sequence as a driving scene label set; a travelable region generative model is trained using the determined set of driving scenario labels. Therefore, different driving scene label sets meeting the conditions can be selected as driving to generate the edge driving scene, the driving environment of the vehicle during driving can be accurately identified by the trained driving area generating model, and the driving safety of the vehicle is improved.
In some embodiments, the executing subject may input the driving scene image sequence included in the driving scene data and the driving scene tag set to a pre-trained marginal driving scene generation model, so as to obtain a marginal driving scene. Wherein the edge driving scene generation model may be a pre-trained diffusion model. Here, the diffusion model may be a markov process. The above markov process corresponds to a state transition equation. The markov process described above may include a noise addition process, a mapping process, and an encoding process. The above-described noise adding process may be adding noise generated in advance to the input driving scene image. The above-mentioned pre-generated noise may be a random variable satisfying a gaussian distribution. The mapping process may be mapping each driving scenario tag in the driving scenario tag set into a continuous vector. The encoding process may be to encode each driving scene tag in the continuous vector by an encoding equation. The above coding equation may be a preset coding equation. For example, the encoding equation may be an EMB encoding equation.
In an optional implementation of some embodiments, the edge driving scenario generation model may be a predefined model. The pre-defined model comprises an input layer, a pixel space layer, a semantic training layer and a noise countermeasure layer. The input layer is used for inputting a driving scene image and a driving scene label set. The pixel space layer may be configured to receive the driving scene image output by the input layer, extract pixels in the received driving scene image, and output pixel features. The semantic training layer is used for receiving the driving scene label set output by the input layer, extracting semantic features in the driving scene label set as semantic drive and outputting the semantic features. The noise countermeasure layer can be used for receiving the pixel characteristics and the semantic characteristics output by the pixel space layer and the semantic training layer, learning noise countermeasure is conducted on the pixel characteristics and the semantic characteristics, and the pixel characteristics and the semantic characteristics after the learning noise countermeasure are output. The noise countermeasure layer includes a predetermined number of convolution layers. Each convolutional layer of the predetermined number of convolutional layers comprises a 3*3 convolutional kernel, a batch normalization layer and a linear correction unit layer. The pixel space layer can also receive pixel characteristics and semantic characteristics output by the noise countermeasure layer after the noise countermeasure is learned, and the edge driving scene is output by enabling the pixel characteristics and the semantic characteristics after the noise countermeasure is learned to pass through a preset function. The predetermined function may be a predetermined loss function. For example, the above-described loss function may be a cross-entropy loss function.
The optional technical content in the step 102 is an inventive point of the present disclosure, and solves the technical problems mentioned in the background art, that is, when the edge driving scene is generated by using the pre-trained diffusion model, the generated edge driving scene needs to be a high-resolution image or video, so that a function with higher complexity needs to be used in the diffusion model, and a longer time needs to be consumed to generate the edge driving scene. The factors that cause the long time consumption for generating the edge driving scene are as follows: when the edge driving scene is generated by using the pre-trained diffusion model, the generated edge driving scene needs to be a high-resolution image or video, so that a function with higher complexity needs to be used in the diffusion model, and the edge driving scene needs to be generated in a longer time. If the above factors are solved, the effect of reducing the time for generating the marginal driving scene can be achieved. To achieve this effect, the present disclosure uses a predefined model, and performs noise countermeasure by converting the driving scene image into low-dimensional pixel features through a pixel space layer, so that the computational complexity can be reduced, and the time for generating an edge driving scene can be reduced.
Optionally, the above-mentioned edge driving scenario generation model may be obtained by training through the following steps:
first, a sample set is obtained.
In some embodiments, the execution subject may obtain a set of samples. The samples in the sample set comprise a sample driving scene image sequence, a sample driving scene label set and sample edge driving scenes corresponding to the sample driving scene image sequence and the sample driving scene label set.
And secondly, selecting samples from the sample set.
In some embodiments, the execution entity may select a sample from the set of samples. Here, the execution body may randomly select a sample from the sample set.
And thirdly, inputting the samples into an initial network model to obtain an edge driving scene corresponding to the samples.
In some embodiments, the executive body may input the sample into an initial network model, and obtain an edge driving scene corresponding to the sample. The initial neural network may be a generative model capable of obtaining an edge driving scene according to the driving scene image sequence and the driving scene label set. The initial neural network may be a generative model. For example, the initial neural network model may be a diffusion model.
And fourthly, determining a loss value between the marginal driving scene and the sample marginal driving scene included by the sample.
In some embodiments, the performing subject may determine a loss value between the marginal driving scenario and a sample marginal driving scenario included in the sample. In practice, first, the similarity between the edge driving scene and the sample edge driving scene can be determined using the OpenCV-based image similarity method. Second, a difference between a preset similarity and the above-mentioned similarity may be determined as a loss value between the above-mentioned marginal driving scene and the sample marginal driving scene included in the above-mentioned sample. The preset similarity may be a preset similarity. For example, the preset similarity may be 1.
And fifthly, responding to the loss value being more than or equal to a preset threshold value, and adjusting the network parameters of the initial network model.
In some embodiments, the executing entity may adjust the network parameter of the initial network model in response to the loss value being greater than or equal to a preset threshold. Here, the setting of the preset threshold is not limited. For example, the loss value and a preset threshold may be differenced to obtain a loss difference. On the basis, the error value is transmitted from the last layer of the model to the front by using methods such as back propagation, random gradient descent and the like so as to adjust the parameter of each layer. Of course, according to the requirement, a network freezing (dropout) method may also be adopted, and network parameters of some layers are kept unchanged and are not adjusted, which is not limited in any way.
Optionally, in response to that the loss value is smaller than the preset threshold, determining the initial network model as a marginal driving scene generation model.
In some embodiments, the executing entity may determine the initial network model as a marginal driving scenario generation model in response to the loss value being less than the preset threshold.
In some embodiments, the executing subject may input at least one marginal driving image included in the marginal driving scene into a driving image recognition model trained in advance, so as to obtain a driving image recognition result. The driving image recognition model may be a classifier model trained in advance for recognizing the driving state of the target vehicle in the edge driving image. The driving image recognition model takes at least one edge driving image as input and takes a driving image recognition result as output.
And 104, responding to the driving image recognition result representing that the target vehicle is in an abnormal state, and acquiring the driving data of the target vehicle.
In some embodiments, the executing body may acquire the driving data of the target vehicle in response to the driving image recognition result indicating that the target vehicle is in an abnormal state. Wherein the abnormal state may be the presence of an obstacle in front of the target vehicle. The driving data includes a traveling speed and a traveling performance value of the target vehicle. The above-described running performance value may characterize the running performance of the target vehicle. For example, the above-described running performance value may be an engine power value of the target vehicle.
And 105, inputting the driving data of the target vehicle and at least one marginal driving image included in the marginal driving scene into the custom model to obtain a drivable area.
In some embodiments, the execution subject may input the driving data of the target vehicle and at least one marginal driving image included in the marginal driving scene into a custom model, so as to obtain a travelable region. The custom model may be a neural network model for determining a travelable area of the vehicle, which is trained in advance. Here, the above-mentioned custom model may include an obstacle recognition submodel and a travelable region recognition submodel. The obstacle recognition submodel may be a model for recognizing an obstacle in the edge driving image. The obstacle recognition submodel may include a feature extraction layer, a pooling layer, and an output layer. The feature extraction layer is used for extracting the obstacle features in the edge driving image and outputting a feature map. And the pooling layer takes the feature map as an input and is used for fusing the extracted same obstacle features so as to reduce the dimension of the obstacle features. The output layer is used for outputting obstacle information in the edge driving image. The drivable region identifier model may be a neural network model that takes the driving data of the target vehicle and the obstacle information in the edge driving image as inputs, determines the coordinates of the obstacle in the edge driving image, determines a region in the edge driving image where the obstacle does not exist as a drivable region, and outputs the drivable region.
As an example, the above travelable region identifier sub-model may include, but is not limited to, at least one of: FCN (full volumetric Networks) model, resnet (Residual neural Network) model, VGG (Visual Geometry Group Network) model, google net (deep neural Network) model, and the like.
Alternatively, after step 105, the target vehicle is controlled to move according to the travelable region.
In some embodiments, the execution subject may control the target vehicle to move according to the travelable region. In practice, the target vehicle may be controlled to move in the travelable area by means of a wired connection or a wireless connection.
The above embodiments of the present disclosure have the following beneficial effects: the above embodiments of the present disclosure have the following advantages: by the travelable region generation method of some embodiments of the present disclosure, the safety of vehicle travel is improved. Specifically, the reason why the safety of the vehicle running is low is that: the edge driving scene realized by using the 3D simulation technology has a large difference with the real edge driving scene, so that obstacles exist in the travelable region generated by the trained travelable region generating model, and the safety of vehicle traveling is low. Based on this, the travelable region generation method of some embodiments of the present disclosure, first, receives driving scene data of a target vehicle and a driving scene tag set corresponding to the driving scene data. Thus, data support can be provided for generating the edge driving scene. Secondly, inputting the driving scene image sequence and the driving scene label set included in the driving scene data into a pre-trained marginal driving scene generation model to obtain a marginal driving scene. Therefore, the driving scene label can be used as a drive to guide the construction of the edge driving scene, and the real edge driving scene is generated by using the real image, so that the driving-capable area generating model can be trained by using the generated real edge driving scene, the trained driving-capable area generating model can accurately generate the driving-capable area of the vehicle, and the driving safety of the vehicle is improved. Then, inputting at least one edge driving image included in the edge driving scene into a driving image recognition model trained in advance to obtain a driving image recognition result; and responding to the driving image recognition result to represent that the target vehicle is in an abnormal state, and acquiring the driving data of the target vehicle. Thus, data support is provided for generating the travelable region. And finally, inputting the driving data of the target vehicle and at least one marginal driving image included in the marginal driving scene into a user-defined model to obtain a drivable area. Therefore, the driving feasible region of the vehicle can be accurately generated by the trained driving feasible region generating model through the generated real edge driving scene, and the driving safety of the vehicle is improved.
With further reference to fig. 2, as an implementation of the methods illustrated in the above figures, the present disclosure provides some embodiments of a travelable region generation apparatus, which correspond to those method embodiments illustrated in fig. 1, and which may be particularly applicable in various electronic devices.
As shown in fig. 2, a travelable region generation apparatus 200 of some embodiments includes: a receiving unit 201, a first input unit 202, a second input unit 203, an acquisition unit 204, and a third input unit 205. Wherein the receiving unit 201 is configured to receive driving scene data of a target vehicle and a driving scene tag set corresponding to the driving scene data, wherein the driving scene data comprises a driving scene image sequence comprising at least one driving scene image; the first input unit 202 is configured to input a driving scene image sequence included in the driving scene data and the driving scene tag set to a pre-trained marginal driving scene generation model, so as to obtain a marginal driving scene, where the marginal driving scene includes at least one marginal driving image; the second input unit 203 is configured to input at least one edge driving image included in the edge driving scene into a driving image recognition model trained in advance, so as to obtain a driving image recognition result; the obtaining unit 204 is configured to obtain driving data of the target vehicle in response to the driving image recognition result indicating that the target vehicle is in an abnormal state; the third input unit 205 is configured to input the driving data of the target vehicle and at least one marginal driving image included in the marginal driving scene into a custom model, resulting in a travelable region.
It is to be understood that the units described in the travelable region generation apparatus 200 correspond to the respective steps in the method described with reference to fig. 1. Thus, the operations, features and resulting beneficial effects described above with respect to the method are also applicable to the travelable region generation apparatus 200 and the units included therein, and are not described in detail herein.
Referring now to FIG. 3, a block diagram of an electronic device 300 suitable for use in implementing some embodiments of the present disclosure is shown. The electronic device shown in fig. 3 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.
As shown in fig. 3, the electronic device 300 may include a processing means 301 (e.g., a central processing unit, a graphics processor, etc.) that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM) 302 or a program loaded from a storage means 308 into a Random Access Memory (RAM) 303. In the RAM 303, various programs and data necessary for the operation of the electronic apparatus 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input/output (I/O) interface 305 is also connected to bus 304.
Generally, the following devices may be connected to the I/O interface 305: input devices 306 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; an output device 307 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage devices 308 including, for example, magnetic tape, hard disk, etc.; and a communication device 309. The communication means 309 may allow the electronic device 300 to communicate wirelessly or by wire with other devices to exchange data. While fig. 3 illustrates an electronic device 300 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided. Each block shown in fig. 3 may represent one device or may represent multiple devices, as desired.
In particular, according to some embodiments of the present disclosure, the processes described above with reference to the flow diagrams may be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In some such embodiments, the computer program may be downloaded and installed from a network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. The computer program, when executed by the processing apparatus 301, performs the above-described functions defined in the methods of some embodiments of the present disclosure.
It should be noted that the computer readable medium described in some embodiments of the present disclosure may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In some embodiments of the disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
In some embodiments, the clients, servers may communicate using any currently known or future developed network Protocol, such as HTTP (HyperText Transfer Protocol), and may interconnect with any form or medium of digital data communication (e.g., a communications network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network.
The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: the method comprises the steps of receiving driving scene data of a target vehicle and a driving scene label set corresponding to the driving scene data, wherein the driving scene data comprise a driving scene image sequence, and the driving scene image sequence comprises at least one driving scene image. And inputting a driving scene image sequence and the driving scene label set which are included in the driving scene data into a pre-trained edge driving scene generation model to obtain an edge driving scene, wherein the edge driving scene includes at least one edge driving image. And inputting at least one edge driving image included in the edge driving scene into a driving image recognition model trained in advance to obtain a driving image recognition result. And acquiring the driving data of the target vehicle in response to the driving image recognition result representing that the target vehicle is in an abnormal state. And inputting the driving data of the target vehicle and at least one marginal driving image included in the marginal driving scene into a user-defined model to obtain a travelable area.
Computer program code for carrying out operations for embodiments of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in some embodiments of the present disclosure may be implemented by software, and may also be implemented by hardware. The described units may also be provided in a processor, and may be described as: a processor includes a receiving unit, a first input unit, a second input unit, an obtaining unit, and a third input unit. Here, the names of these units do not constitute a limitation of the unit itself in some cases, and for example, the receiving unit may also be described as a "unit that receives driving scene data of the target vehicle and a driving scene tag set corresponding to the above driving scene data".
The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems on a chip (SOCs), complex Programmable Logic Devices (CPLDs), and the like.
The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention in the embodiments of the present disclosure is not limited to the specific combination of the above-mentioned features, but also encompasses other embodiments in which any combination of the above-mentioned features or their equivalents is made without departing from the inventive concept as defined above. For example, the above features and (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure are mutually replaced to form the technical solution.
Claims (7)
1. A travelable region generation method, comprising:
receiving driving scene data of a target vehicle and a driving scene tag set corresponding to the driving scene data, wherein the driving scene data comprises a driving scene image sequence comprising at least one driving scene image;
inputting a driving scene image sequence and the driving scene label set included in the driving scene data into a pre-trained marginal driving scene generation model to obtain a marginal driving scene, wherein the marginal driving scene includes at least one marginal driving image;
inputting at least one edge driving image included in the edge driving scene into a driving image recognition model trained in advance to obtain a driving image recognition result;
responding to the driving image recognition result to represent that the target vehicle is in an abnormal state, and acquiring driving data of the target vehicle;
and inputting the driving data of the target vehicle and at least one marginal driving image included in the marginal driving scene into a user-defined model to obtain a drivable area.
2. The method of claim 1, wherein the method further comprises:
and controlling the target vehicle to move according to the travelable area.
3. The method of claim 1, wherein the edge driving scenario generative model is trained by:
obtaining a sample set, wherein samples in the sample set comprise a sample driving scene image sequence and a sample driving scene label set, and sample edge driving scenes corresponding to the sample driving scene image sequence and the sample driving scene label set;
selecting a sample from the set of samples;
inputting the sample into an initial network model to obtain an edge driving scene corresponding to the sample;
determining a loss value between the marginal driving scenario and a sample marginal driving scenario comprised by the sample;
and adjusting the network parameters of the initial network model in response to the loss value being greater than or equal to a preset threshold value.
4. The method of claim 3, wherein the method further comprises:
and determining the initial network model as a marginal driving scene generation model in response to the loss value being smaller than the preset threshold value.
5. A travelable region generation apparatus comprising:
a receiving unit configured to receive driving scene data of a target vehicle and a driving scene tag set corresponding to the driving scene data, wherein the driving scene data comprises a driving scene image sequence including at least one driving scene image;
a first input unit configured to input a driving scene image sequence and the driving scene label set included in the driving scene data into a pre-trained marginal driving scene generation model to obtain a marginal driving scene, wherein the marginal driving scene includes at least one marginal driving image;
the second input unit is configured to input at least one edge driving image included in the edge driving scene into a driving image recognition model trained in advance to obtain a driving image recognition result;
an acquisition unit configured to acquire driving data of the target vehicle in response to the driving image recognition result representing that the target vehicle is in an abnormal state;
and the third input unit is configured to input the driving data of the target vehicle and at least one marginal driving image included in the marginal driving scene into a custom model to obtain a drivable area.
6. An electronic device, comprising:
one or more processors;
a storage device having one or more programs stored thereon;
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-4.
7. A computer-readable medium, on which a computer program is stored, wherein the program, when executed by a processor, implements the method of any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211553876.XA CN115661238B (en) | 2022-12-06 | 2022-12-06 | Method and device for generating travelable region, electronic equipment and computer readable medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211553876.XA CN115661238B (en) | 2022-12-06 | 2022-12-06 | Method and device for generating travelable region, electronic equipment and computer readable medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115661238A CN115661238A (en) | 2023-01-31 |
| CN115661238B true CN115661238B (en) | 2023-03-07 |
Family
ID=85019833
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211553876.XA Active CN115661238B (en) | 2022-12-06 | 2022-12-06 | Method and device for generating travelable region, electronic equipment and computer readable medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115661238B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111366917A (en) * | 2020-03-13 | 2020-07-03 | 北京百度网讯科技有限公司 | Method, device and equipment for detecting travelable area and computer readable storage medium |
| CN114359869A (en) * | 2021-12-31 | 2022-04-15 | 中国第一汽车股份有限公司 | Method and device for detecting boundary on vehicle driving area |
| CN115018909A (en) * | 2022-08-03 | 2022-09-06 | 禾多科技（北京）有限公司 | Method and device for generating travelable region, electronic equipment and computer readable medium |
-
2022
- 2022-12-06 CN CN202211553876.XA patent/CN115661238B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111366917A (en) * | 2020-03-13 | 2020-07-03 | 北京百度网讯科技有限公司 | Method, device and equipment for detecting travelable area and computer readable storage medium |
| CN114359869A (en) * | 2021-12-31 | 2022-04-15 | 中国第一汽车股份有限公司 | Method and device for detecting boundary on vehicle driving area |
| CN115018909A (en) * | 2022-08-03 | 2022-09-06 | 禾多科技（北京）有限公司 | Method and device for generating travelable region, electronic equipment and computer readable medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115661238A (en) | 2023-01-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108520220B (en) | Model generation method and device | |
| CN109816589B (en) | Method and apparatus for generating cartoon style conversion model | |
| CN108427939B (en) | Model generation method and device | |
| CN109740018B (en) | Method and device for generating video label model | |
| CN112699991A (en) | Method, electronic device, and computer-readable medium for accelerating information processing for neural network training | |
| CN111523640B (en) | Training method and device for neural network model | |
| CN112132847A (en) | Model training method, image segmentation method, device, electronic device and medium | |
| EP3710993B1 (en) | Image segmentation using neural networks | |
| WO2020093724A1 (en) | Method and device for generating information | |
| CN112650841A (en) | Information processing method and device and electronic equipment | |
| CN113705515B (en) | Training of semantic segmentation model and generation method and device of high-precision map lane line | |
| CN115540893B (en) | Vehicle path planning method and device, electronic equipment and computer readable medium | |
| CN115546293B (en) | Obstacle information fusion method and device, electronic equipment and computer readable medium | |
| CN114627331A (en) | Model training method and device | |
| CN113673446A (en) | Image recognition method and device, electronic equipment and computer readable medium | |
| CN110633597A (en) | Driving region detection method and device | |
| CN115661238B (en) | Method and device for generating travelable region, electronic equipment and computer readable medium | |
| CN116164770A (en) | Path planning method, path planning device, electronic equipment and computer readable medium | |
| CN115273224A (en) | High-low resolution bimodal distillation-based video human body behavior identification method | |
| CN115082306A (en) | Image super-resolution method based on blueprint separable residual error network | |
| CN112215853A (en) | Image segmentation method and device, electronic equipment and computer readable medium | |
| CN114782290B (en) | Disparity map correction method, device, equipment and computer readable medium | |
| CN115565152B (en) | Traffic sign extraction method integrating vehicle-mounted laser point cloud and panoramic image | |
| CN111311616B (en) | Method and apparatus for segmenting an image | |
| CN117557822A (en) | Image classification method, apparatus, electronic device, and computer-readable medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |