WO2023019507A1

WO2023019507A1 - Road image generation method and apparatus based on deep learning, and device and storage medium

Info

Publication number: WO2023019507A1
Application number: PCT/CN2021/113511
Authority: WO
Inventors: 金晨; 卢红喜; 李国庆; 衡阳; 杜濠杰
Original assignee: 浙江吉利控股集团有限公司; 宁波吉利汽车研究开发有限公司
Priority date: 2021-08-19
Filing date: 2021-08-19
Publication date: 2023-02-23
Also published as: CN117616425A

Abstract

Disclosed in the present application are a road image generation method and apparatus based on deep learning, and a device and a storage medium. The method comprises: acquiring a road image to be processed; performing environmental influence removal processing on the road image to be processed, so as to obtain a basic road image; and respectively performing injection processing on the basic road image according to environmental influence feature information corresponding to different environmental influences, so as to generate environmental influence road images under different environmental influences. In comparison with the prior art where a high-precision map is directly made according to road images which are collected in real time in good weather, thus resulting in there being relatively few environmental influence image samples for making the high-precision map, in the present application, a basic road image of a road image, and environmental influence feature information under different environmental influences are acquired, the basic road image being a road image without an environmental influence, and environmental influence road images under different environmental influences are then generated according to the environmental influence feature information and the basic road image, thereby realizing diversification of the environmental influence road images.

Description

Road image generation method, device, equipment and storage medium based on deep learning

technical field

The present application relates to the technical field of image processing, in particular to a method, device, device and storage medium for generating road images based on deep learning.

Background technique

With the development and technological iteration of autonomous driving, the mass production of autonomous driving is getting closer. At the same time, in order for the automatic driving system to better understand its own posture during automatic operation, and perform precise behavior planning and body control, it is very important to produce high-precision maps. In the prior art, road images are only collected in real time by a single sensor such as laser or vision or multi-sensor fusion (vision, laser, millimeter wave, etc.) under good weather conditions, and high-precision maps are produced directly based on the collected road images, resulting in high production costs. The environmental impact of precision maps is less on road images.

The above content is only used to assist in understanding the technical solution of the present application, and does not mean that the above content is admitted as prior art.

technical solution

The main purpose of this application is to provide a method, device, device and storage medium for generating road images based on deep learning, aiming to solve the technical problem of how to obtain road images affected by the environment under different environmental influences.

In order to achieve the above object, the present application provides a method for generating a road image based on deep learning, the method for generating a road image based on deep learning includes the following steps:

Obtain road images to be processed;

performing environmental impact removal processing on the road image to be processed to obtain a basic road image;

Injection processing is performed on the basic road image according to the environmental impact feature information corresponding to different environmental impacts, so as to generate environmental impact road images under different environmental impacts.

In one embodiment, the step of performing environmental impact removal processing on the road image to be processed to obtain a basic road image includes:

The road image to be processed is input into a preset environment stripping model to obtain a basic road image, and the preset environment stripping model is obtained by training a first initial neural network model.

In one embodiment, before the step of inputting the road image to be processed into the preset environment stripping model and obtaining the basic road image, it further includes:

Obtain multiple road image samples;

Carry out environmental marking on each road image sample respectively, and obtain multiple marked image samples;

selecting a plurality of road image training samples from the plurality of marked image samples according to preset rules;

The first initial neural network model is trained according to a plurality of road image training samples to obtain a preset environment stripping model.

In one embodiment, the step of acquiring multiple road image samples includes:

Obtain multiple road image samples to be processed;

Traversing the image pixel information corresponding to a plurality of road image samples to be processed, and using the traversed image pixel information as the current image pixel information;

When the pixel information of the current image satisfies a preset pixel condition, extracting missing road image samples corresponding to the pixel information of the current image from a plurality of road image samples to be processed;

After traversing the pixel information of multiple images, a missing road image sample set is established according to the extraction results;

A plurality of road image samples are determined according to the plurality of road image samples to be processed and the missing road image sample set.

In an embodiment, the step of determining a plurality of road image samples according to the plurality of road image samples to be processed and the missing road image sample set includes:

Carrying out repair processing on the missing road image samples in the missing road image sample set respectively to obtain a plurality of road image repair samples;

A plurality of road image samples are determined according to the plurality of road image repair samples and the plurality of road image samples to be processed.

In one embodiment, the step of repairing the missing road image samples in the missing road image sample set to obtain multiple road image repair samples includes:

Traverse the missing road image samples in the missing road image sample set, and use the traversed missing road image samples as the current missing road image samples;

determining missing feature information corresponding to the current missing road image sample;

performing repair processing on the currently missing road image sample according to the missing feature information to obtain a road image repair sample;

After the traversal of multiple image data is completed, multiple road image patch samples are obtained.

In one embodiment, the step of injecting the basic road image according to the environmental impact feature information corresponding to different environmental impacts to generate the environmental impact road images under different environmental impacts includes:

The environmental impact feature information corresponding to different environmental impacts and the basic road image are input into a preset environment injection model to generate environmental impact road images under different environmental impacts, and the preset environment injection model passes the second initial neuron The network model is trained to obtain.

In order to achieve the above purpose, the present application also provides a road image generation device based on deep learning, the road image generation device based on deep learning includes:

An acquisition module, configured to acquire road images to be processed;

A stripping module, configured to perform environmental impact removal processing on the road image to be processed to obtain a basic road image;

The injection module is configured to perform injection processing on the basic road image according to the environmental impact feature information corresponding to different environmental impacts, so as to generate environmental impact road images under different environmental impacts.

In order to achieve the above object, the present application also provides a road image generation device based on deep learning, which includes: a memory, a processor, and a road image based on deep learning that is stored on the memory and can run on the processor. A generation program, the deep learning-based road image generation program is configured to implement the steps of the above-mentioned deep learning-based road image generation method.

In addition, in order to achieve the above purpose, the present application also proposes a computer storage medium, on which a modeling program based on cross-features is stored, and when the modeling program based on cross-features is executed by a processor, the above-mentioned The steps of the above-mentioned modeling method based on cross features.

In addition, in order to achieve the above object, the present application also proposes a storage medium, on which a road image generation program based on deep learning is stored, and when the road image generation program based on deep learning is executed by a processor, the above-mentioned The steps of the road image generation method based on deep learning.

This application first obtains the road image to be processed, then performs environmental impact removal processing on the road image to be processed to obtain the basic road image, and finally injects the basic road image according to the environmental impact characteristic information corresponding to different environmental impacts to generate different environments The environment under influence influences the road image. Compared with the existing technology, which directly collects real-time road images with good weather to make high-precision maps, resulting in fewer samples of environmental impact images for making high-precision maps, and in this application, the basic road images corresponding to road images and different environmental impacts are acquired. Under the environmental impact feature information, the basic road image is a road image without environmental impact, and then the environmental impact road images under different environmental impacts are quickly generated according to the environmental impact feature information and the basic road image, thus realizing the diversification of environmental impact road images .

Description of drawings

Fig. 1 is a schematic structural diagram of a road image generation device based on deep learning of the hardware operating environment involved in the embodiment of the present application;

Fig. 2 is a schematic flow chart of the first embodiment of the method for generating road images based on deep learning in the present application;

FIG. 3 is a schematic diagram of the preset environment stripping model of the first embodiment of the deep learning-based road image generation method of the present application;

FIG. 4 is a schematic diagram of the preset environment injection model of the first embodiment of the deep learning-based road image generation method of the present application;

5 is a schematic flow diagram of the second embodiment of the method for generating road images based on deep learning in the present application;

6 is a schematic flow diagram of the third embodiment of the method for generating road images based on deep learning in the present application;

Fig. 7 is a structural block diagram of the first embodiment of the device for generating road images based on deep learning in the present application.

The realization, functional features and advantages of the present application will be further described in conjunction with the embodiments and with reference to the accompanying drawings.

Embodiment of this application

It should be understood that the specific embodiments described here are only used to explain the present application, not to limit the present application.

Referring to FIG. 1 , FIG. 1 is a schematic structural diagram of a deep learning-based road image generation device of a hardware operating environment involved in an embodiment of the present application.

As shown in Figure 1, the road image generation device based on deep learning may include: a processor 1001, such as a central processing unit (Central Processing Unit, CPU), communication bus 1002, user interface 1003, network interface 1004, memory 1005. Wherein, the communication bus 1002 is used to realize connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). Memory 1005 can be a high-speed random access memory (Random Access Memory, RAM), can also be a stable non-volatile memory (Non-Volatile Memory, NVM), such as disk storage. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001 .

Those skilled in the art can understand that the structure shown in Figure 1 does not constitute a limitation to the deep learning-based road image generation device, and may include more or less components than those shown in the illustration, or combine certain components, or have different Part placement.

As shown in FIG. 1 , the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a road image generation program based on deep learning.

In the road image generation device based on deep learning shown in Figure 1, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with users; The processor 1001 and memory 1005 can be set in the road image generation device based on deep learning, and the road image generation device based on deep learning calls the road image generation program based on deep learning stored in the memory 1005 through the processor 1001, and executes the application The embodiment provides a road image generation method based on deep learning.

The embodiment of the present application provides a method for generating a road image based on deep learning. Referring to FIG. 2 , FIG. 2 is a schematic flowchart of the first embodiment of the method for generating a road image based on deep learning in this application.

In this embodiment, the method for generating road images based on deep learning includes the following steps:

Step S10: Obtain the road image to be processed.

It is easy to understand that the execution subject of this embodiment may be a road image generation device based on deep learning with functions such as data processing, network communication, and program operation, or other computer devices with similar functions. This embodiment does not be restricted.

The road image to be processed is the road image collected in real time by the shooting device on the current vehicle. There is environmental impact feature information in the road image to be processed. It should be noted that the environmental impact feature information includes weather feature information, such as sunny weather feature information, rainy weather feature information information or fog feature information, etc., the environmental impact feature information also includes light feature information, such as early morning feature information, noon feature information, or evening feature information.

The weather characteristic information can be understood as the influence information of the weather on the captured image, such as clarity and blur. The light feature information may be understood as information about the influence of the light on the captured image, such as light intensity and image brightness. The weather or light is different, and the corresponding weather feature information or light feature information is different.

Assuming that the road image to be processed is captured by a shooting device at noon on a rainy day, the environmental impact feature information in the road image to be processed includes weather feature information and light feature information, where the weather feature information is rainy day feature information, and the light feature information is noon feature information. feature information, etc.

Step S20: Perform environmental impact removal processing on the road image to be processed to obtain a basic road image.

The basic road image is a road image without environmental impact. In order to obtain the basic road image accurately, the road image to be processed is processed to remove the environmental impact, and the processing method to obtain the basic road image can be input to the preset environment stripping In the model, the preset environment stripping model is used to perform stripping processing on the road image to be processed to output the basic road image, and the preset environment stripping model is obtained by training the first initial neural network model.

In order to be able to obtain the preset environment stripping model for image environment stripping processing, it is necessary to obtain multiple road image samples in advance, and then perform environment labeling on each road image sample to obtain multiple labeled image samples. The road image samples are marked for environmental comparison, and multiple road image training samples are selected from multiple marked image samples according to preset rules, and finally the first initial neural network model is trained according to the multiple road image training samples to obtain the preset environment stripping Model. The preset rule is an image selection ratio set by the user, which can be 70%, or 60%.

In a specific implementation, the multiple road image samples include multiple sunny road images, multiple rainy road images, multiple foggy road images, multiple early morning road images, multiple noon road images and multiple evening road images, according to the environment Comparing and marking the road image samples, 70% of the road image samples can be extracted from the multiple marked road image samples, that is, the marked image samples, as multiple road image training samples, and multiple road image samples can be determined according to the remaining marked image samples. Road image validation sample.

It should be noted that the first initial neural network model is trained according to multiple road image training samples to obtain the initial environment stripping model, and then multiple road image verification samples need to be input into the initial environment stripping model for verification. , using the initial environmental stripping model as the default environmental stripping model.

In this embodiment, it is also necessary to obtain multiple road image test samples, and input the multiple road image test samples into the preset environment stripping model for testing. The proportion of the test set and the test set can be divided according to 6:2:2; when the data volume of the obtained road image samples is large, the proportion of the training set, the verification set and the test set can be divided according to 98:1:1, etc.

It should be understood that the preset environment stripping model can output the environmental information of the image to be processed and the basic image, where the environmental information includes the environmental impact name and the corresponding environmental impact feature information, in this embodiment, the output environmental impact name can be and the corresponding environmental impact feature information to generate a weather feature database, in which there are environmental impact feature information corresponding to different environmental impacts.

In a specific implementation, image loopback detection can be performed on multiple road image samples to be processed, missing road image samples can be extracted from multiple road image samples to be processed, and then the missing road image samples can be repaired respectively to obtain multiple road images Repair the samples, and finally replace the corresponding missing road image samples with the plurality of road image repair samples, and determine the plurality of road image samples according to the plurality of road image repair samples and the plurality of road image samples to be processed.

The missing road image samples in the missing road image sample set are respectively repaired, and the processing method of obtaining multiple road image repair samples can be to traverse the missing road image samples in the missing road image sample set, and the traversed missing road image samples As the current missing road image sample, then determine the missing feature information corresponding to the current missing road image sample, and then repair the current missing road image sample according to the missing feature information to obtain the road image repair sample, and finally complete the traversal of multiple image data After that, multiple road image inpainting samples are obtained.

Referring to FIG. 3, FIG. 3 is a schematic diagram of the preset environment stripping model based on the first embodiment of the deep learning road image generation method of the present application. In FIG. 3, multiple road image samples are collected by the shooting device, and the multiple road image samples to be processed include Multiple road images in sunny days, road images in rainy days, road images in foggy days, road images in early morning, road images in noon and road images in evening, etc., and then perform image loopback on multiple road image samples to be processed Detection, extracting missing road image samples from multiple road image samples to be processed, and then repairing the missing road image samples respectively to obtain multiple road image repair samples, repairing multiple road image samples and multiple road image samples to be processed The sample determines multiple road image samples, and performs environmental labeling on multiple road image samples according to the weather impact comparison, and then builds a stripped data set based on the marked road image samples, where the stripped data set includes multiple road image training samples and multiple road images. Image verification sample.

It should be noted that the first initial neural network model is trained according to multiple road image training samples to obtain the initial environment stripping model, and then multiple road image verification samples need to be input into the initial environment stripping model for verification. , to obtain the preset environmental stripping model.

In this embodiment, the missing road image is obtained, and then the corresponding standard road image is obtained according to the current missing feature information of the missing road image, the standard road image is a road image without environmental impact corresponding to the missing road image with complete pixel information, Then, the missing road image is repaired according to the road image without environmental impact, and the road image repair sample is obtained. The weather influence information of the road image repair sample is consistent with the weather influence information of the missing road image, and the road image repair sample has complete pixels Road imagery for information and weather-affected information.

Step S30: Perform injection processing on the basic road image according to the environmental impact feature information corresponding to different environmental impacts, so as to generate environmental impact road images under different environmental impacts.

Input the environmental impact feature information corresponding to different environmental impacts and the basic road image into the preset environment injection model to generate environmental impact road images under different environmental impacts. The preset environment injection model is trained by the second initial neural network model get.

In order to be able to obtain the preset environment injection model for image environment injection, it is necessary to obtain multiple road image samples and weather feature databases in advance, select multiple road image training samples from multiple road image samples according to preset rules, and The remaining multiple road image samples are used as multiple road image verification samples, and finally the second initial neural network model is trained according to the multiple road image training samples and the weather feature database, and the trained neural network model is trained according to the multiple road image verification samples. The initial neural network model is verified. After the trained initial neural network model is successfully verified, the trained initial neural network model is injected into the model as a preset environment, and then multiple road image test samples are used to test the preset environment injection model , after the test is successful, the preset environmental injection model will output the road image of environmental impact under different environmental influences. The default rule is the user-defined image selection ratio, which can be 70%, and can also be Sixty percent etc.

Referring to Fig. 4, Fig. 4 is a schematic diagram of the preset environment injection model based on the first embodiment of the deep learning road image generation method of the present application. In Fig. 4, multiple road image samples are collected by the shooting device, and the multiple road image samples to be processed include Multiple road images in sunny days, road images in rainy days, road images in foggy days, road images in early morning, road images in noon and road images in evening, etc., according to multiple road image samples, determine the injection data set, inject The data set includes multiple road image training samples and multiple road image verification samples, and then the second initial neural network model is trained according to the multiple road image training samples and the weather feature database, and the training is completed according to the multiple road image verification samples. After the initial neural network model after training is successfully verified, the initial neural network model after training is injected into the model as a preset environment.

Assuming that the environmental impact characteristic information under different environmental influences is foggy weather characteristic information, early morning characteristic information, rainy weather characteristic information and noon characteristic information respectively, the foggy weather characteristic information, early morning characteristic information, rainy day characteristic information and noon characteristic information and the basic road image Input into the default environment injection module, then obtain the road image of the foggy morning feature information, the road image of the foggy noon feature information, the road image of the rainy morning feature information, the road image of the rainy noon feature information and the rainy evening feature information road image.

In this embodiment, first obtain the road image to be processed, then perform environmental impact removal processing on the road image to be processed to obtain the basic road image, and finally perform injection processing on the basic road image according to the environmental impact feature information corresponding to different environmental impacts, To generate environmental impact road images under different environmental influences. Compared with the prior art, which directly collects real-time road images with good weather to make high-precision maps, resulting in fewer samples of environmental impact images for making high-precision maps, and in this embodiment, the basic road images corresponding to road images and different environmental Environmental impact feature information under the impact, the basic road image is a road image without environmental impact, and then according to the environmental impact feature information and the basic road image to generate environmental impact road images under different environmental impacts, thus realizing the diversification of environmental impact road images , thus enriching the types of high-precision maps.

Referring to FIG. 5 , FIG. 5 is a schematic flowchart of a second embodiment of the deep learning-based road image generation method of the present application.

Based on the first embodiment above, in this embodiment, the step S20 further includes:

Step S201: Input the road image to be processed into a preset environment stripping model to obtain a basic road image. The preset environment stripping model is obtained by training a first initial neural network model.

In order to be able to obtain the preset environment stripping model for image environment stripping processing, it is necessary to obtain multiple road image samples in advance, and then perform environment labeling on each road image sample to obtain multiple labeled image samples. The road image samples are marked for environmental comparison, and multiple road image training samples are selected from multiple marked image samples according to preset rules, and finally the first initial neural network model is trained according to the multiple road image training samples to obtain the preset environment stripping Model.

Assume that multiple road image samples are image A, image B, image C, image D, image E, and image F. If image A is a sunny road image, image B is a rainy road image, and image C is a foggy road image, image D is the road image in the morning, image E is the road image at noon, and image F is the road image in the evening, then the environment of image A is marked as sunny, the environment of image B is marked as rainy, the environment of image C is marked as foggy, and the environment of image D The environment labeled early morning, image E is labeled noon and the environment of image F is labeled evening.

The preset rule is an image selection ratio set by the user, which can be 70%, or 60%.

The processing method of obtaining multiple road image samples can be to obtain multiple road image samples to be processed, traverse the image pixel information corresponding to the multiple road image samples to be processed, and use the traversed image pixel information as the current image pixel information, and When the pixel information of the current image satisfies the preset pixel conditions, extract the missing road image samples corresponding to the pixel information of the current image from multiple road image samples to be processed, and establish the missing road image according to the extraction results after traversing the pixel information of multiple images The sample set is to determine a plurality of road image samples according to the plurality of road image samples to be processed and the missing road image sample set. The preset pixel condition can be understood as missing pixel information in the image and the like.

Referring to FIG. 6 , FIG. 6 is a schematic flowchart of a third embodiment of a method for generating a road image based on deep learning in the present application.

Based on the first embodiment above, in this embodiment, the step S30 further includes:

Step S301: Input the environmental impact feature information corresponding to different environmental impacts and the basic road image into the preset environment injection model to generate environmental impact road images under different environmental impacts. The second initial neural network model is obtained by training.

The environmental impact feature information corresponding to different environmental impacts can be obtained from the weather feature database, which includes multiple environmental impact names and corresponding environmental impact feature information, where there is a one-to-one correspondence between the environmental impact names and the environmental impact feature information .

Input the environmental impact characteristic information and basic road image corresponding to different environmental impacts into the preset environment injection model to generate environmental impact road images under different environmental impacts. The preset environment injection model is trained by the second initial neural network model get.

In order to be able to obtain the preset environment injection model for image environment injection, it is necessary to obtain multiple road image samples and weather feature databases in advance, select multiple road image training samples from multiple road image samples according to preset rules, and The remaining multiple road image samples are used as multiple road image verification samples, and finally the second initial neural network model is trained according to the multiple road image training samples and the weather feature database, and the trained neural network model is trained according to the multiple road image verification samples. The initial neural network model is verified. After the trained initial neural network model is successfully verified, the trained initial neural network model is injected into the model as a preset environment, and the output result of the preset environment injection model is the environment under different environmental influences. Affecting the road image, the default rule is the user-defined image selection ratio, which can be 70%, 60%, etc.

In a specific implementation, the multiple road image samples include multiple sunny road images, multiple rainy road images, multiple foggy road images, multiple early morning road images, multiple noon road images and multiple evening road images, which can be obtained from Seventy percent of the road image samples are extracted from the multiple road image samples as multiple road image training samples, and multiple road image verification samples are determined according to the remaining marked image samples.

Assuming that the environmental impact feature information under different environmental influences is sunny day feature information, noon feature information, rainy day feature information and evening feature information, the sunny day feature information, noon feature information, rainy day feature information, evening feature information and basic road image input to The preset environment is injected into the module to obtain road images of sunny noon characteristic information, sunny evening characteristic information road images, rainy noon characteristic information road images, and rainy evening characteristic information road images.

In this embodiment, the environmental impact feature information and basic road images corresponding to different environmental impacts are input into the preset environment injection model to generate environmental impact road images under different environmental impacts. The initial neural network model is obtained by training. Compared with the existing technology that directly collects road images with environmental influences through shooting equipment in real time, this method results in fewer road images under different environmental influences and increases the work of image capture. In this embodiment, the environmental impact feature information corresponding to different environmental impacts and the basic road image are input into the preset environment injection model, so that the preset environment injection model superimposes the environmental impact feature information on the basic road image, thereby obtaining different The environment under the influence of the environment affects the road image, which in turn reduces the workload of image capture.

In addition, the embodiment of the present application also proposes a storage medium on which a deep learning-based road image generation program is stored. When the deep learning-based road image generation program is executed by a processor, the depth-based Learn the steps of a road image generation method.

Referring to FIG. 7, FIG. 7 is a structural block diagram of the first embodiment of the device for generating road images based on deep learning in the present application.

As shown in Figure 7, the road image generation device based on deep learning proposed in the embodiment of the present application includes:

An acquisition module 7001, configured to acquire road images to be processed.

The stripping module 7002 is configured to perform environmental impact removal processing on the road image to be processed to obtain a basic road image.

In a specific implementation, the multiple road image samples include multiple sunny road images, multiple rainy road images, multiple foggy road images, multiple early morning road images, multiple noon road images and multiple evening road images, according to the environment Comparing and marking the road image samples, 70% of the road image samples can be extracted from the multiple marked road image samples, that is, the marked image samples, as multiple road image training samples, and the remaining marked image samples can be used as multiple road image samples. Road image validation sample.

The injection module 7003 is configured to perform injection processing on the basic road image according to the environmental impact feature information corresponding to different environmental impacts, so as to generate environmental impact road images under different environmental impacts.

In order to be able to obtain the preset environment injection model for image environment injection, it is necessary to obtain multiple road image samples in advance, and then train and verify the second initial neural network model according to the multiple road image samples, and finally the initial training after training After the neural network model is successfully verified, the trained initial neural network model is used as the preset environment injection model, and the output result of the preset environment injection model is the environmental impact road image under different environmental influences.

For other embodiments or specific implementations of the device for generating road images based on deep learning in this application, reference may be made to the above-mentioned method embodiments, which will not be repeated here.

It should be noted that, as used herein, the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or system comprising a set of elements includes not only those elements, It also includes other elements not expressly listed, or elements inherent in the process, method, article, or system. Without further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article or system comprising that element.

The serial numbers of the above embodiments of the present application are for description only, and do not represent the advantages and disadvantages of the embodiments.

Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware, but in many cases the former is better implementation. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product in essence or the part that contributes to the prior art, and the computer software product is stored in a storage medium (such as read-only memory/random access memory, magnetic disk, optical disk), including several instructions to enable a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present application.

The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structures or equivalent process transformations made by using the description of the application and the accompanying drawings are directly or indirectly used in other related technical fields. , are all included in the patent protection scope of the present application in the same way.

Claims

A method for generating road images based on deep learning, characterized in that, the method for generating road images based on deep learning comprises the following steps:

Obtain road images to be processed;

performing environmental impact removal processing on the road image to be processed to obtain a basic road image;

Injection processing is performed on the basic road image according to the environmental impact feature information corresponding to different environmental impacts, so as to generate environmental impact road images under different environmental impacts.
The method according to claim 1, wherein the step of performing environmental impact removal processing on the road image to be processed to obtain a basic road image comprises:

The road image to be processed is input into a preset environment stripping model to obtain a basic road image, and the preset environment stripping model is obtained by training a first initial neural network model.
The method according to claim 2, wherein, before the step of obtaining the basic road image, the step of inputting the road image to be processed into the preset environment stripping model further includes:

Obtain multiple road image samples;

Carry out environmental marking on each road image sample respectively, and obtain multiple marked image samples;

selecting a plurality of road image training samples from the plurality of marked image samples according to preset rules;

The first initial neural network model is trained according to a plurality of road image training samples to obtain a preset environment stripping model.
The method according to claim 3, wherein the step of acquiring a plurality of road image samples comprises:

Obtain multiple road image samples to be processed;

Traversing the image pixel information corresponding to a plurality of road image samples to be processed, and using the traversed image pixel information as the current image pixel information;

When the pixel information of the current image satisfies a preset pixel condition, extracting missing road image samples corresponding to the pixel information of the current image from a plurality of road image samples to be processed;

After traversing the pixel information of multiple images, a missing road image sample set is established according to the extraction results;

A plurality of road image samples are determined according to the plurality of road image samples to be processed and the missing road image sample set.
The method according to claim 4, wherein the step of determining a plurality of road image samples according to the plurality of road image samples to be processed and the missing road image sample set comprises:

Carrying out repair processing on the missing road image samples in the missing road image sample set respectively to obtain a plurality of road image repair samples;

A plurality of road image samples are determined according to the plurality of road image repair samples and the plurality of road image samples to be processed.
The method according to claim 5, wherein the step of repairing the missing road image samples in the missing road image sample set to obtain a plurality of road image repair samples includes:

Traverse the missing road image samples in the missing road image sample set, and use the traversed missing road image samples as the current missing road image samples;

determining missing feature information corresponding to the current missing road image sample;

performing repair processing on the currently missing road image sample according to the missing feature information to obtain a road image repair sample;

After the traversal of multiple image data is completed, multiple road image patch samples are obtained.
The method according to any one of claims 1-6, characterized in that, according to the environmental impact feature information corresponding to different environmental impacts, the basic road image is injected into the basic road image to generate environmental impacts under different environmental impacts The steps of the road image include:

The environmental impact feature information corresponding to different environmental impacts and the basic road image are input into a preset environment injection model to generate environmental impact road images under different environmental impacts, and the preset environment injection model passes the second initial neuron The network model is trained to obtain.
A road image generation device based on deep learning, characterized in that the road image generation device based on deep learning includes:

An acquisition module, configured to acquire road images to be processed;

A stripping module, configured to perform environmental impact removal processing on the road image to be processed to obtain a basic road image;

The injection module is configured to perform injection processing on the basic road image according to the environmental impact feature information corresponding to different environmental impacts, so as to generate environmental impact road images under different environmental impacts.
A road image generation device based on deep learning, characterized in that the device includes: a memory, a processor, and a road image generation program based on deep learning that is stored on the memory and can run on the processor, the The road image generation program based on deep learning is configured to implement the steps of the method for generating road images based on deep learning according to any one of claims 1 to 7.
A storage medium, characterized in that the storage medium is stored with a road image generation program based on deep learning, and when the road image generation program based on deep learning is executed by a processor, it realizes any one of claims 1 to 7 The steps of the road image generation method based on deep learning.