WO2022206158A1 - 一种图像生成方法、装置、设备及存储介质 - Google Patents

一种图像生成方法、装置、设备及存储介质 Download PDF

Info

Publication number
WO2022206158A1
WO2022206158A1 PCT/CN2022/074085 CN2022074085W WO2022206158A1 WO 2022206158 A1 WO2022206158 A1 WO 2022206158A1 CN 2022074085 W CN2022074085 W CN 2022074085W WO 2022206158 A1 WO2022206158 A1 WO 2022206158A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
replaced
preset
target
scene
Prior art date
Application number
PCT/CN2022/074085
Other languages
English (en)
French (fr)
Inventor
程光亮
石建萍
安井裕司
松永英树
冨手要
Original Assignee
商汤集团有限公司
本田技研工业株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 商汤集团有限公司, 本田技研工业株式会社 filed Critical 商汤集团有限公司
Priority to JP2023559731A priority Critical patent/JP2024511635A/ja
Publication of WO2022206158A1 publication Critical patent/WO2022206158A1/zh
Priority to US18/472,702 priority patent/US20240013348A1/en

Links

Images

Classifications

    • G06T5/77—
    • G06T5/70—
    • G06T3/04—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformation in the plane of the image
    • G06T3/40—Scaling the whole image or part thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image

Definitions

  • the embodiments of the present disclosure relate to the technical field of intelligent driving, and relate to, but are not limited to, an image generation method, apparatus, device, and storage medium.
  • embodiments of the present disclosure provide an image generation technical solution.
  • An embodiment of the present disclosure provides an image generation method, the method includes: detecting a lighting lamp on a target object in an original image; converting the original image into an image to be processed in a specific scene; , determine that the lighting lamp whose working state does not match the scene information of the specific scene is the object to be replaced; according to the scene information and the object to be replaced, determine the lighting that includes the object to be replaced whose working state matches the scene information Replace the image; use the replacement image to replace the image of the area occupied by the object to be replaced in the image to be processed to generate a target image.
  • the determining, according to the scene information and the object to be replaced includes a replacement image of the object to be replaced whose working state matches the scene information, comprising: determining attribute information of the object to be replaced ; in the preset image library, search for a target preset image that includes a preset object whose attributes match the attribute information of the object to be replaced; determine the found target preset image as the replacement image; wherein, The working state of the preset object in the replacement image matches the scene information. In this way, the fidelity and rationality of the replaced target image can be improved.
  • the method further includes: determining a distance between the object to be replaced and a device that captures the original image; and searching a preset image library for a data including attributes and the object to be replaced.
  • the target preset image of the preset object whose attribute information matches including: according to the category information of the object to be replaced and the determined distance, in the preset image library, searching for the attribute including the attribute and the attribute of the object to be replaced Information that matches the target preset image of the preset object.
  • the method in response to the target preset image not being found in the preset image library, the method further includes: generating a preset object whose attributes match the attribute information of the object to be replaced the target preset image; the generated target preset image is determined as the replacement image. In this way, when the target preset image cannot be found, the target preset image is automatically generated, which can improve the accuracy of the replacement image.
  • the generating the target preset image including the preset object whose attributes match the attribute information of the object to be replaced includes: according to the relationship between the object to be replaced and the device that collects the original image The working parameters of the preset object are determined; according to the determined working parameters, a target preset image including the preset object is generated. In this way, the non-illuminated vehicle lamp image in the image to be processed at night is replaced with the illuminated replacement image, so that the generated target image is more realistic.
  • the determining the distance between the object to be replaced and the device that captures the original image includes: determining size information of the object to be replaced according to category information of the object to be replaced; The size information and the size of the object to be replaced in the image to be processed determine the distance between the device that captures the original image and the object to be replaced. In this way, by analyzing the size of the object to be replaced, the distance between the acquisition device and the object can be obtained more accurately.
  • the method further includes: storing the target preset image in the preset image library.
  • the preset image library can be enriched, and it is convenient to search for the target preset image in the preset image library subsequently.
  • the working parameter of the preset object is the illumination intensity of the lighting lamp; the working parameter of the preset object is determined according to the distance between the object to be replaced and the device that captures the original image parameters, including: determining the illumination intensity of the lighting lamp when it is in the activated state; determining the illumination intensity of the illumination lamp that matches the distance according to the distance and the illumination intensity when the illumination lamp is in the activated state;
  • the generating the target preset image including the preset object according to the determined working parameters includes: generating the preset object including the preset object with the illumination intensity matching the distance according to the illumination intensity matching the distance target preset image. In this way, the matching degree between the finally obtained replacement object and the scene information can be improved.
  • using the replacement image to replace the image of the area occupied by the object to be replaced in the image to be processed to generate a target image includes: determining size information of the replacement image; determining the the area of the area occupied by the object to be replaced in the image to be processed; according to the area, adjust the size information of the replacement image to obtain an adjusted image; use the adjusted image to replace the object to be replaced
  • the image of the occupied area is used to generate a candidate image; the candidate image is smoothed to generate the target image. In this way, the generated target image is more reasonable and clear.
  • the target objects include traveling equipment and street lamps.
  • An embodiment of the present disclosure provides an image generation apparatus, the apparatus includes: a lighting detection module configured to detect lighting on a target object in an original image; an image conversion module configured to convert the original image into a specific scene
  • the object determination module is configured to, in the to-be-processed image, determine the lighting lamp whose working state does not match the scene information of the specific scene as the object to be replaced;
  • the replacement image determination module is configured to The scene information and the object to be replaced are determined to include a replacement image of the object to be replaced whose working state matches the scene information;
  • the image generation module is configured to use the replacement image to replace the to-be-replaced image in the to-be-processed image. Replace the image of the area occupied by the object to generate the target image.
  • An embodiment of the present disclosure provides a computer storage medium, where computer-executable instructions are stored thereon, and after the computer-executable instructions are executed, the image generation method described above can be implemented.
  • An embodiment of the present disclosure provides an electronic device, the electronic device includes a memory and a processor, where computer-executable instructions are stored in the memory, and the processor can implement the above-mentioned when executing the computer-executable instructions on the memory.
  • the image generation method described above is described above.
  • An embodiment of the present disclosure provides a computer program, including computer-readable code, when the computer-readable code is executed in an electronic device, a processor in the electronic device executes the image configured to realize any one of the above-mentioned images Generate method.
  • Embodiments of the present disclosure provide an image generation method, device, device, and storage medium.
  • an original image is converted into an image to be processed in a specific scene;
  • the lighting lamp whose scene information does not match is used as the object to be replaced, and according to the scene information and the object to be replaced, a replacement image including the object to be replaced whose working state matches the scene information is determined; finally, the replacement image is used to replace the object to be replaced in the image to be processed.
  • the image of the area occupied by the object is replaced to generate the target image; in this way, the working state of the object in the target image is matched with the scene information, so that the generated target image is more in line with the real scene.
  • FIG. 1A is a schematic diagram of a system architecture to which a trajectory prediction method according to an embodiment of the present disclosure can be applied;
  • FIG. 1B is a schematic diagram of an implementation flowchart of an image generation method according to an embodiment of the present disclosure
  • FIG. 2 is a schematic flowchart of another implementation of the image generation method provided by the embodiment of the present disclosure.
  • 3A is a schematic diagram of the composition and structure of an image generation system provided by an embodiment of the present disclosure.
  • 3B is a schematic diagram of an application scenario of an image generation method according to an embodiment of the present disclosure.
  • FIG. 4 is an implementation framework structure diagram of an image generation method provided by an embodiment of the present disclosure.
  • FIG. 5 is a schematic diagram of another application scenario of the image generation method according to an embodiment of the present disclosure.
  • FIG. 6 is a schematic structural composition diagram of an image generating apparatus according to an embodiment of the present disclosure.
  • FIG. 7 is a schematic structural diagram of a computer device according to an embodiment of the present disclosure.
  • first ⁇ second ⁇ third is only used to distinguish similar objects, and does not represent a specific ordering of objects. It is understood that “first ⁇ second ⁇ third” is used in Where permitted, the specific order or sequence may be interchanged to enable embodiments of the disclosure described in some embodiments to be practiced in sequences other than those illustrated or described in some embodiments.
  • Gaussian blur It is a low-pass filter for the image.
  • the so-called “blur” can be understood as taking the average value of surrounding pixels for each pixel.
  • Autonomous vehicle A vehicle that contains sensors that sense the surrounding environment.
  • the vehicle coordinate system is fixed on the autonomous vehicle, wherein the x-axis is the direction of the car's advancing direction, the y-axis points to the left side of the vehicle's advancing direction, and the z-axis is perpendicular to the ground, which conforms to the right-handed coordinate system.
  • the origin of the coordinate system is on the ground below the midpoint of the rear axle.
  • the device provided by the embodiment of the present disclosure may be implemented as a notebook computer, a tablet computer, a desktop computer, a camera, and a mobile device (eg, a personal digital computer) with an image acquisition function.
  • a mobile device eg, a personal digital computer
  • Various types of user terminals such as assistants, dedicated messaging devices, portable game devices) can also be implemented as servers.
  • exemplary applications when the device is implemented as a terminal or a server will be described.
  • the method can be applied to a computer device, and the functions implemented by the method can be realized by calling a program code by a processor in the computer device.
  • the program code can be stored in a computer storage medium.
  • the computer device includes at least a processor and a storage medium. medium.
  • FIG. 1A is a schematic diagram of a system architecture to which an image generation method according to an embodiment of the present disclosure can be applied; as shown in FIG. 1A , the system architecture includes an image acquisition device 131 , a network 132 and an image generation terminal 133 .
  • the image capture device 131 and the image generation terminal 133 may establish a communication connection through the network 132, and the image capture device 131 reports the captured original image (or, the image generation terminal 133) to the image generation terminal 133 through the network 202.
  • the original image collected by the vehicle terminal 131 is automatically acquired), and the image generation terminal 133 responds to the received original image, first, detects the lighting in the original image, and converts the image into a to-be-processed image in a certain scene; then , in the image to be processed, find the lighting lamps with unreasonable working status; finally, replace the image area corresponding to the unreasonable lighting lamps with the replacement image whose working status matches the scene information, so as to generate the target image, and in the image generation
  • the target image is output on the image display interface of the terminal 133 . In this way, the working state of the object in the target image is matched with the scene information, so that the generated target image is more in line with the real scene.
  • the image capture device 131 may be a capture device including a camera or the like.
  • the image generation terminal 133 may include a computer device with a certain computing capability, for example, the computer device includes a terminal device or a server or other processing device.
  • the network 132 can be wired or wireless. Wherein, when the image generation terminal 133 is a server, the image acquisition device 131 can communicate with the server through a wired connection, such as data communication through a bus; when the image generation terminal 133 is a terminal device, the image acquisition device 131 can be wirelessly connected The connection method is connected to the image generation terminal 133 for communication, and further data communication is performed.
  • the image generation terminal 133 may be a vision processing device with a video capture module, or a host with a camera.
  • the image generation method of the embodiment of the present application may be executed by the image generation terminal 133 , and the above-mentioned system architecture may not include the network 132 and the image acquisition device 131 .
  • FIG. 1B is a schematic diagram of the implementation flow of the image generation method according to the embodiment of the present disclosure, as shown in FIG. 1B , and described in conjunction with the steps shown in FIG. 1B :
  • Step S101 detecting the lighting lamp on the target object in the original image.
  • the original image may be an image collected in any scene, may be an image with complex screen content, or may be an image with simple screen content, for example, an image of a street scene collected in the middle of the night, or an image collected during the day images of street scenes, etc.
  • the target object is an object with lighting such as a vehicle or a street lamp.
  • the target objects include devices with variable working states such as driving equipment and street lights, and devices with variable working states, including: movable devices with at least two working states, such as vehicles with various functions (such as trucks, cars, motorcycles, bicycles, etc.), vehicles with various wheel numbers (such as four-wheeled vehicles, two-wheeled vehicles, etc.) and any movable equipment (such as robots, aircraft, blind guides, smart furniture equipment or smart toys, etc.), etc. Or fixed equipment with at least two working states, such as various road lighting lights (such as high pole street lights, mid pole lights, road lights, garden lights, lawn lights or landscape lights, etc.).
  • the image to be processed is a road image of a night scene, wherein the device with a variable working state is a light on a vehicle traveling on the road.
  • Step S102 converting the original image into an image to be processed in a specific scene.
  • the scene information of a specific scene may include the level of light in the scene, the location where the scene is located, the objects in the scene, and the like.
  • the scene information includes: the brightness of the street, the location of the street, and objects such as vehicles and street lights on the street.
  • the image to be processed may be an image in a specific scene, where the specific scene may be any set scene. For example, a late night scene, an evening scene, or an early morning scene, etc.
  • step S102 converting the original image into an image to be processed in a specific scene can be achieved by the following steps:
  • the first step is to get the original image.
  • the original image is an image collected in any scene, for example, a road image collected during the day or a road image collected at night.
  • the scene information of the original image is determined.
  • a trained discriminator is used to determine whether the scene information of the image is scene information of a specific scene. For example, the scene information of a specific scene is a night scene, and the discriminator determines whether the scene information of the image is a night scene.
  • the third step when the scene information does not match the scene information of the specific scene, convert the scene information of the original image according to the scene information of the specific scene to obtain a converted image.
  • the scene information of the original image is converted into the scene information of the specific scene, that is, the original image is converted into the image of the specific scene, so as to Get the converted image.
  • the scene information of a specific scene is the night scene
  • the original image is the image collected in the daytime scene
  • the original image collected in the daytime scene is converted into the image of the night scene
  • the corresponding original image can be generated by inputting the original image into the generator. Image to be processed at night.
  • the original image is determined as the to-be-processed image if the scene information matches the scene information of the specific scene.
  • the scene information is the same as or very similar to the scene information of the specific scene, it means that the original image has the scene information of the specific scene, so the original image can be used as the image to be processed without performing image conversion on the original image.
  • the scene information of the specific scene is a late-night scene
  • the original image is an image collected in a scene where night falls
  • the scene is similar to the specific scene and is night
  • the original image is determined as the image to be processed.
  • the converted image is determined as the image to be processed.
  • the scene information of the original image is converted, so as to obtain An image to process with scene information for a specific scene.
  • Step S103 in the image to be processed, it is determined that the lighting lamp whose working state does not match the scene information of the specific scene is the object to be replaced.
  • the working state of the lighting lamp is associated with the scene information of the specific scene, and it can be understood that the working state of the lighting lamp should change with the change of the scene information of the specific scene. For example, if the scene information of a specific scene is converted from a daytime scene to a nighttime scene, the working state of the lighting also changes accordingly.
  • the image to be processed is a road image with a night scene, and the lights are the lights of the vehicle running in the image, in a real situation, the lights (such as tail lights) of the vehicle in the night scene should be In the light-emitting state, if the vehicle lamp is in the unlit state in the image to be processed, the vehicle lamp of the vehicle is determined to be the object to be replaced.
  • the lights of the vehicle in the daytime scene should be in an unlit state. If the lights in the image to be processed are in the lighted state, then the lights of the vehicle are determined to be the objects to be replaced. Or, if the object to be replaced is a street lamp, then in the actual scene, the street lamp in the night scene should be in a lighting state, and the street lamp in the daytime scene should be in an unlit state.
  • the entire target object in the to-be-processed image including the lighting lamp whose working state does not match the scene information of the specific scene may also be used as the to-be-replaced object.
  • the scene information of a specific scene is a night scene
  • the target object is a vehicle
  • the vehicle includes lights in an unlit state, and the vehicle can be used as the object to be replaced.
  • Step S104 according to the scene information and the object to be replaced, determine a replacement image including the object to be replaced whose working state matches the scene information.
  • the working state of the object to be replaced included in the replacement image matches the scene information, that is, the working state of the lighting lamp included in the replacement image matches the scene information. If the working state of the object to be replaced in the image to be processed does not match the scene information, it can be understood that the working state of the object to be replaced is not a reasonable state in the scene.
  • the scene information is a night scene
  • the object to be replaced is a car lamp.
  • the working state of the object to be replaced is an unlit state, which means that the working state of the object to be replaced does not match the scene information.
  • the working state of the object to be replaced in the replacement image matches the scene information, and it can be understood that the working state of the object to be replaced in the replacement image is a reasonable state in the scene. For example, if the scene information is a night scene, and the object to be replaced is a car lamp, then the working state of the object to be replaced in the replacement image is a lighting state, and correspondingly, the replacement image is the collected image of the vehicle lamp in the lighting state.
  • Step S105 the replacement image is used to replace the image of the area occupied by the object to be replaced in the image to be processed to generate a target image.
  • the object to be replaced whose working state does not match the scene information is determined, and the area occupied by the object to be replaced in the image to be processed is determined;
  • the replacement image corresponding to the replacement object replaces the image in the area, and performs smoothing on the replaced image to generate the target image.
  • the image to be processed is a road image with a night scene, and the lights whose working status changes with the scene are the lights in the vehicle;
  • the state is not illuminated, that is, the working state does not match the night scene, in this case, according to the specifications of the lamp, look for the preset image in the preset image library that includes the same specification as the lamp.
  • the replacement image wherein, the working state of the object to be replaced in the replacement image matches the scene, that is, the preset working state of the vehicle lamp in the replacement image is the light-emitting state.
  • the replacement image is used to replace the image of the region where the vehicle light is located in the image to be processed, thereby generating the target image.
  • the working state of the vehicle lights in the generated target image is a lighting state, which matches the night scene, thereby making the generated target image more realistic.
  • the object to be replaced whose working state does not match the scene information is determined in the image to be processed; then, the image of the area occupied by the object to be replaced is replaced by the replacement image, so that the generated target image is Including the object to be replaced whose working state matches the scene information, thereby making the generated target image more vivid.
  • a replacement image including an object to be replaced whose working state matches the scene information is stored in a preset image library, which can be used to render the object to be replaced in real time, thereby generating a target image for animation playback,
  • the implementation process is as follows.
  • the object to be replaced is a vehicle lamp as an example for description:
  • the vehicle lights refer to lighting lights on the vehicle, which may be headlights, fog lights, reversing lights, license plate lights, and so on.
  • the working modes of the headlights include at least: flashing mode, high beam mode and low beam mode.
  • the working mode is a preset working mode
  • the preset working mode can be set to a flashing mode, that is, if the working mode is a flashing mode, in the preset image library, it is determined that the vehicle light changes with time series during the flashing process.
  • a target preset image is obtained, and a set of target preset images is obtained.
  • the current object to be replaced is rendered in real time according to a plurality of target preset images that change in time series, a target animation of the object to be replaced in a preset working mode is generated, and the generated target animation is played in an animation format.
  • the vehicle lamp can emit light of various colors and work through a variety of lighting modes, for example, by Blink mode works.
  • the embodiment of the present disclosure can not only output a still target image, but also can adopt a target image in an animation format. Since the replacement image including the object whose working state matches the scene information is stored in the preset image library, the object to be replaced can be rendered through the preset image that changes with the time series, so as to generate and play the target animation.
  • the target object is determined in the image to be processed, and the lighting lamp whose working state does not match the scene information in the target object is used as the object to be replaced. The following steps are implemented:
  • the first step is to determine the lighting lamps with variable working states in the image to be processed.
  • a lighting lamp whose working state is associated with the scene information is determined. Although the working state of the lighting lamp is associated with the scene information, it does not necessarily match the scene information.
  • the second step determine the lighting lamp whose working state does not match the scene information, as the object to be replaced.
  • the object to be replaced is a lamp in the movable device whose working state does not match the scene information.
  • the object to be replaced is a lamp in the vehicle. Lights whose working status does not match the scene information (such as headlights, front lights or tail lights, etc.).
  • the target object is a fixed device, the object to be replaced is a lamp whose working state does not match the scene information in the fixed device; for example, if the target object is a high-pole street lamp, then the object to be replaced is a high-pole street lamp whose working state and scene information do not match. matching lights.
  • the target object may also be a fixed lighting device, and the lighting lamp whose working state does not match the scene information is the lighting lamp, that is, the object to be replaced is the lighting lamp on the fixed lighting device; in the image to be processed , the working state of the illuminating lamp may be emitting light or not emitting light, that is, the illuminating lamp in the image to be processed may be in a light-on state or a light-off state.
  • the scene information of the image to be processed is a daytime scene
  • the image to be processed is collected by nighttime
  • the image of the daytime scene obtained after the original image in the street scene is scene converted. If the street lights on the street are off in the original image, and after the image conversion is performed, the image to be processed is an image of a daytime scene, and the street lights do not need to be on; If the scene information of the to-be-processed image is a night scene, for example, the to-be-processed image is an image of a night scene obtained by performing scene conversion on the original image of the street scene collected during the day.
  • the image to be processed is an image of a night scene, and the street lights need to be on;
  • An image is replaced, and the replacement image replaces the image of the area occupied by the object to be replaced in the image to be processed to generate a target image.
  • the target object may be a traveling device, and the lighting lamp in the traveling device whose working state does not match the scene information is the lighting lamp of the traveling device.
  • the traveling device is a vehicle
  • the object to be replaced is Car lights: In the image to be processed, the working state of the lights on the vehicle can be on or off, that is, the lights in the to-be-processed image can be on or off.
  • the driving device as a vehicle and the object to be replaced as a taillight
  • the scene information of the image to be processed is a daytime scene
  • the image to be processed is a daytime scene obtained by converting the original image in the street scene collected at night. image of the scene.
  • the image to be processed is an image of a daytime scene, and the headlights do not need to be on; so in this case, There is no need to replace the headlights on the image to be processed; if the scene information of the image to be processed is a night scene, for example, the image to be processed is a night scene obtained by converting the original image of the street scene collected during the day.
  • the image to be processed is an image of a night scene, and the headlights need to be on;
  • the replacement image of the headlamp in the lighted state is used to replace the image of the area occupied by the object to be replaced in the to-be-processed image to generate the target image.
  • a target object with multiple working states is determined in the to-be-processed image, and a lighting lamp whose working state does not match the scene information in the target object is replaced, so that the replaced target image It matches the scene information better, and the picture content is more reasonable and vivid.
  • the image to be replaced may be obtained from a preset image library, or the image to be replaced may be generated by analyzing the attribute information of the object to be replaced, that is, in step S104, it may be determined in the following two ways Image to be replaced, where:
  • Manner 1 In the preset image library, search for a replacement image including a preset object matching the object to be replaced.
  • the working state of the preset object stored in the preset image library matches the scene information. For example, if the scene information is a night scene, then the preset image library stores the images of the lights of the car in the lighting state.
  • the target preset image matching the object to be replaced can be understood as a target preset image with at least the same specification or the same type as the object to be replaced. For example, if the object to be replaced is a car lamp, the target preset image is a car lamp image with the same specification as the car lamp.
  • a replacement image can be searched in the preset image library through the following process, as shown in FIG. 2 , which is another schematic flowchart of the implementation of the image generation method provided by the embodiment of the present disclosure, step S104 may be It is implemented through steps S201 to S203, and is described in conjunction with the steps shown in FIG. 2:
  • Step S201 determining attribute information of the object to be replaced.
  • the trained second neural network can be used to find the replacement image, so as to improve the accuracy of determining the replacement image, that is, the attribute information of the object to be replaced is determined through the second neural network.
  • the attribute information of the object to be replaced is information used to represent the specification and type of the object to be replaced, and to describe the object to be replaced; for example, the object to be replaced is a car lamp, and the attribute information is the specification of the car lamp, the specific The types of lights and the left and right sides of the lights, etc.; among them, the types of lights include: rear position lights, left tail lights, right tail lights, headlights, etc.
  • Step S202 in the preset image library, search for a target preset image including a preset object whose attributes match the attribute information of the object to be replaced.
  • a preset object whose attribute information is the same as that of the object to be replaced is searched; then, a target preset image including the preset object is searched in the preset image library.
  • Step S203 determining the found target preset image as the replacement image.
  • the working state of the preset object in the replacement image matches the scene information.
  • the objects to be replaced are left and right tail lamps
  • a preset image library is searched for a target preset image that includes the tail lamps in the lighting state that have the same attribute information as the vehicle lamps. Because the left taillight and the right taillight are the same in appearance and shape, the determined target preset image may be one frame image, or may be two symmetrical images that are the same as the left taillight and the right taillight respectively.
  • the target preset image is a frame of image
  • the target preset image is used to replace the left and right taillights respectively, so that the left and right taillights in the replaced target image are replaced. still symmetrical.
  • the object to be replaced is a headlight or a rear light
  • the position information of the object to be replaced is used to determine whether it is a headlight or a rear light, In order to make the replaced target image more reasonable and realistic.
  • searching for the target preset image in the preset image library can be achieved through the following process:
  • the distance between the object to be replaced and the device for capturing the original image is determined.
  • the image to be processed is obtained by converting the original image, that is, the image to be processed is obtained by performing image conversion on the original image; for example, the original image is a street image collected in a daytime scene, and the daytime scene is converted into a nighttime scene scene, get the image to be processed.
  • the distance can be calculated by determining the size of the object to be replaced, and then based on the size, the number of pixels the object to be replaced occupies in the image to be processed, and the focal length of the device that captured the original image.
  • the size information of the object to be replaced is determined according to the category information of the object to be replaced, and the device for collecting the original image is determined according to the size information of the object to be replaced and the size of the object to be replaced in the image to be processed. The distance from the object to be replaced.
  • the size information of the object to be replaced includes the length, width, height, and the like of the object to be replaced.
  • the category information of the object to be replaced includes: the brand or type of the object to be replaced, etc., taking the target object as a driving device as an example, the category information of the target object includes: car, Suburban Utility Vehicle (SUV) , vans, small trucks, large trucks, buses, buses or vans, etc.; then the objects to be replaced are cars, SUVs, vans, small trucks, large trucks, buses, buses or vans, etc. Lights whose state does not match the scene information for a specific scene.
  • SUV Suburban Utility Vehicle
  • the category information of the target object includes: various types of street lights, construction site lighting, searchlights, architectural lighting, marine lights or civil lights; then the objects to be replaced are various types of street lights, construction site lighting Lamps, searchlights, architectural lights, marine lights, or civil lights whose working state does not match the scene information of a specific scene.
  • the specification of the target object can be determined, thereby obtaining the size information of the target object. For example, it is determined that the category information of the target object is a van, and it is further determined which type and specification of the van it is, so as to obtain the width of the van.
  • the target object as a van after the true width of the van is determined, the number of pixels corresponding to the true width of the object to be replaced (that is, the lights of the van) in the image to be processed can be determined; then, combined with the focal length of the image acquisition device (eg, the focal length of the camera), the distance between the image capture device and the object to be replaced is estimated by the quotient of the number of pixels and the focal length.
  • different categories of objects to be replaced have different working parameters at different distances. Therefore, by combining the category of the object to be replaced with the distance, the target preset is searched in the preset image library. image, which can improve the accuracy of the found target preset image. Take the target object as a fixed lighting device, the object to be replaced is a lamp on the fixed lighting device, or, take the target object as a traveling device, and the object to be replaced is a lamp on the traveling device, as an example to illustrate, the image The distance between the acquisition device and the lighting lamp is different, and the lighting intensity of the lighting lamp is also different. Based on the lighting intensity at this distance, look for the lighting lamp with the same lighting intensity in the preset image library, and preset the target with the lighting lamp. image, as a replacement image.
  • a target preset image including a preset object whose attributes match the attribute information of the object to be replaced may be generated, and the generated target preset image , as a replacement image, and update the preset image library based on the generated target preset image, which can be achieved by the following process:
  • the distance between the object to be replaced and the device that captures the original image is determined.
  • the determined distance is: the distance between the lighting lamp of the traveling device and the device for capturing the original image.
  • the traveling device is a vehicle
  • the determined distance is the distance between the headlights of the vehicle and the image acquisition device.
  • the working parameters of the preset object are determined.
  • the working parameters of the preset object include: various data of the preset object during normal operation, and the category of the parameters corresponds to the category of the preset object, including: the working power and working intensity of the preset object, etc. ;
  • the preset object is a lighting lamp (a lighting lamp of a traveling equipment or a lighting lamp of a fixed lighting equipment)
  • the working parameters of the lighting lamp include at least the light intensity of the lighting lamp, and the distance is different, and the lighting intensity of the lighting lamp is different, The larger the distance, the weaker the light intensity of the lighting lamp, that is, the smaller the working parameters.
  • a target preset image including a preset object is generated.
  • a target preset image with an object in a working state corresponding to the working parameters may be generated according to the working parameters to obtain a replacement image.
  • the preset object is an illumination lamp and the working parameter is the illumination intensity of the illumination lamp, then according to the illumination intensity, a replacement image of the object to be replaced with the illumination intensity is generated.
  • a replacement image of the object to be replaced for the illumination intensity can be generated.
  • the replacement image and the corresponding relationship between the replacement image and the distance may be stored in the preset image library to obtain an updated preset image library.
  • the size of the area occupied by the object to be replaced in the image to be processed is determined, and then the size of the replacement image is adjusted based on the size, so that the adjusted replacement image is the same as the object to be replaced.
  • the size of the occupied area fits, so that the adjusted replacement image is used to replace the image of the area occupied by the object to be replaced, so that the generated target image has a higher quality.
  • the image to be processed is a road image of a night scene
  • the target object is a vehicle running on the road
  • the object to be replaced is an unlit headlight in the target object, for example, the unlit taillight of the vehicle in the night scene image
  • a replacement image of the vehicle lamp with the light intensity matching the distance is generated, and the replacement image is used to replace the vehicle tail lamp in the image to be processed.
  • the non-illuminated headlight image in the image to be processed at night is replaced with an illuminated replacement image, so that the generated target image is more realistic.
  • the size of the device is determined by determining the category information of the object to be replaced, so that the distance between the device and the image acquisition device can be determined.
  • the corresponding relationship between the distances can be determined in the relationship correspondence table, and the working parameter of the distance matching can be determined, thereby generating a target preset image with the object in the working state corresponding to the working parameter.
  • the correspondence between the distance and the working parameters, the correspondence between the category information and size information of the object to be replaced, and the generated target preset image are all stored in the preset image library to update the preset image library. Therefore, when the target preset image needs to be searched from the preset image library again, more abundant preset objects can be provided for selection, thereby improving the accuracy of the selected target preset image.
  • the process of generating the target preset image including the preset object by determining the working parameters of the preset object can be realized by the following steps:
  • the first step is to determine the light intensity when the light is on.
  • the illuminating lamp may be an illuminating lamp of any specification, for example, a street lamp with a lower power or a searchlight with a higher power, then determine the light intensity of the illuminating lamp in the activated state.
  • the second step according to the distance and the light intensity when the light is in the activated state, determine the light intensity of the light that matches the distance.
  • the corresponding relationship between the light intensity when the lighting lamp is in the activated state and the distance is determined, and then, based on the corresponding relationship, the light intensity of the lighting light at different distances is determined.
  • the corresponding relationship between the illumination intensity and the distance is that the larger the distance, the smaller the illumination intensity; in other words, due to the different distances between the image acquisition device and the object to be replaced, the collected illumination intensity of the object to be replaced is different, and the illumination intensity is different.
  • the intensity is inversely proportional to the distance, that is, the greater the distance between the image acquisition device and the object to be replaced, the smaller the collected illumination intensity of the object to be replaced.
  • a correspondence table representing the correspondence between light intensity and distance can be created.
  • the table matching the distance can be found from the table. the light intensity.
  • a target preset image including the preset object with the illumination intensity matched with the distance is generated.
  • an image of a preset object with the illumination intensity is generated by rendering.
  • determining the replacement image including the object to be replaced whose working state matches the scene information is implemented, and the attribute information of the object to be replaced can be determined through the second neural network, and the preset image Find the target preset image in the library that includes the preset object with the same attribute information; if the target preset image cannot be found in the preset image library, then by comprehensively considering the category information of the object to be replaced and the device and image acquisition The distance between the devices is determined, and the working parameters of the preset object matching the distance are determined; thereby generating a target preset image including the preset object with the working parameters, and by storing the target preset image in the preset image library The preset image library is updated, thereby improving the match between the finally obtained replacement object and the scene information.
  • the above-mentioned process of determining the replacement object may be implemented by a neural network, where the neural network includes a first neural network and a second neural network, and the first neural network is used to determine the category information of the target object;
  • the network according to the category information, determines the lighting lamps in the target object whose working state does not match the scene information, and then the object to be replaced can be obtained.
  • the implementation process is as follows:
  • the first neural network may be any type of neural network, such as a convolutional neural network, a residual network, and the like. Input the image to be processed into the trained first neural network, and the first neural network outputs the detection frame and category of the target object.
  • the training process of the first neural network can be implemented by the following steps:
  • the training image is input into the first neural network to be trained, and the first position information of the target object in the to-be-trained image is predicted.
  • the first neural network to be trained is trained by using a large number of training images, that is, a large number of training images are input into the first neural network to be trained to predict the position and category of the target object in the to-be-trained image.
  • the first prediction loss of the first position information is determined according to the marked position information of the target object in the training image.
  • the first prediction loss is determined by using the difference between the labeled position information of the target object in the training image and the first position information of the target object.
  • the third step is to adjust the network parameters of the first neural network to be trained according to the first prediction loss to obtain the first neural network.
  • the accuracy of each predicted first position information is determined by combining the labeled position information of the target object, and the accuracy is fed back to the neural network, so that the neural network can adjust the network such as weight parameters. parameters to improve the accuracy of neural network detection.
  • the first prediction loss is the cross-entropy loss of positive samples and negative samples.
  • the parameters such as the weight of the neural network are adjusted by the prediction loss, so that the adjusted neural network prediction result is more accurate.
  • the above process is a process of training the first neural network. Based on the predicted position of the target object and the marked position of the target object, multiple iterations are performed to make the first prediction of the first position information output by the trained first neural network.
  • the loss satisfies the convergence condition, so that the accuracy of the target object detected by the first neural network is higher.
  • the process of determining the object to be replaced in the target object is as follows:
  • the category information of the target object is determined through the first neural network.
  • the category information of the target object is predicted through the first neural network
  • the category information is input into the second neural network.
  • the object to be replaced in the target object is determined according to the category information.
  • the second neural network may be a trained network for predicting objects to be replaced in the target object, and the network may be any type of neural network. By inputting the category information of the target object into the second neural network, the object to be replaced that is associated with the scene information of the specific scene in the target object can be predicted.
  • the training process of the second neural network can be implemented by the following steps:
  • the category information to which the target object in the training image belongs is marked to obtain the marked training image.
  • the marked image is input into the second neural network to be trained, and the second position information of the object to be replaced of the target object is predicted according to the marked category information.
  • the second neural network to be trained is used to predict the position of the object to be replaced in the target object.
  • the marked image is input into the second neural network to be trained, so as to predict the second position information of the object to be replaced, that is, to predict the position of the object to be replaced in the target object.
  • the second prediction loss of the second position information is determined according to the marked position information of the object to be replaced of the target object.
  • the second prediction loss may be a loss function of the same type as the first prediction loss, eg, a cross-entropy loss function.
  • the fourth step is to adjust the network parameters of the second neural network to be trained according to the second prediction loss to obtain the second neural network.
  • the network parameters of the second neural network to be trained include weights of neurons in the neural network, and the like. Parameters such as the weight of the second neural network to be trained are adjusted by using the second prediction loss, so that the detection result of the adjusted second neural network is more accurate.
  • the above process is a process of training the second neural network. Based on the category information of the target object, multiple iterations are performed, so that the second prediction loss of the predicted position information of the object to be replaced, which is output by the trained second neural network, is satisfied. Convergence conditions, so that the accuracy of the object to be replaced output by the second neural network is higher.
  • the replacement image is used to replace the image of the area occupied by the object to be replaced in the image to be processed to generate the target image, which can be achieved through the following process:
  • the size information of the replacement image is determined.
  • the first neural network is used to output the detection frame of the object to be replaced, and the area of the detection frame can be used as the area of the area occupied by the object to be replaced in the image to be processed.
  • the size information of the replacement image is adjusted to obtain an adjusted image.
  • the size information of the replacement image is adjusted according to the area of the area occupied by the object to be replaced in the image to be processed, to obtain an adjusted image, so that the size information of the adjusted image is consistent with the size of the area. Size fits.
  • the adjusted image is used to replace the image of the area occupied by the object to be replaced to generate a candidate image.
  • the image of the area occupied by the object to be replaced is replaced by the adjusted image, thereby obtaining the replaced image, that is, the candidate image.
  • the scene information of a specific scene is a night scene
  • the image to be processed is a road image with a night scene
  • the object to be replaced is a car lamp whose working state does not match the night scene, that is, a car lamp in an unlit state , indicating that the way the lamp is presented in the image is unreasonable
  • the target preset image that is, the replacement image found from the preset image library
  • the adjusted image can be obtained by adjusting the size of the image, so that the image of the area occupied by the object to be replaced is replaced by the adjusted image, and the target image including the lights in the light-emitting state is generated.
  • the candidate image is smoothed to generate the target image.
  • the area in the candidate image where the replacement operation occurs may be smoothed to eliminate noise in the area, or the entire candidate image may be smoothed to denoise the entire image , so as to obtain the target image, which makes the generated target image more reasonable and clear.
  • Embodiments of the present disclosure provide a method for adding tail lights in a night scene based on image generation, target detection, and tail light matching, so that vehicles in the generated night image are more realistic.
  • the embodiments of the present disclosure can be applied to more image generation fields. For example, the addition of vehicle headlights and street lights in night scenes makes the generation of taillights in the generated images more realistic.
  • FIG. 3A is a schematic diagram of the composition and structure of an image generation system (for generating an image to be processed) provided by an embodiment of the present disclosure, and the following description is made in conjunction with FIG. 3A :
  • the image generation system includes: a generator 301 and a discriminator 302 .
  • the image to be processed during the day (corresponding to the original image in the above-mentioned embodiment, such as the image to be processed during the day 321 in FIG. 3B ) is used as input, and is input from the input terminal 303 to the generator 301;
  • both the night scene image collected in the night scene and the generated night scene image are input into the discriminator 302 .
  • the discriminator 302 is used to distinguish whether the image of the night scene is from the real night scene image or the generated night scene image, that is, the real image 305 and the converted image 306 are obtained respectively;
  • fine-grained object detection and annotation are performed on the vehicles in the image, that is, the vehicles in the image are framed by a rectangular frame and marked with Category information for each vehicle (eg, car, SUV, van, pickup, truck, bus, bus, van, etc.).
  • Category information for each vehicle eg, car, SUV, van, pickup, truck, bus, bus, van, etc.
  • the taillights of each vehicle are marked, that is, a rectangular frame is used to mark the position of the taillights.
  • FIG. 4 is an implementation frame structure diagram of the image generation method provided by the embodiment of the present disclosure. The following description is made in conjunction with FIG. 4 :
  • the image acquisition module 401 is configured to perform target detection on the original image and use a labeling frame to label the target.
  • the vehicle detection network training module 402 is configured to use the marked frame and the classification of vehicles to train a corresponding detection network to obtain a vehicle detection network (corresponding to the first neural network in the above embodiment).
  • the vehicle detection result module 403 is configured to use the vehicle detection network to detect the original image and obtain the vehicle detection result (for example, obtain the coordinates of the upper left corner and the lower right corner of the vehicle in the image coordinate system (that is, the rectangular frame marking the area where the vehicle is located)) , and obtain the category to which the detected vehicle belongs, that is, the vehicle type 404, so as to prepare for the matching of headlights when the image is generated.
  • the vehicle detection result for example, obtain the coordinates of the upper left corner and the lower right corner of the vehicle in the image coordinate system (that is, the rectangular frame marking the area where the vehicle is located)
  • the category to which the detected vehicle belongs that is, the vehicle type 404
  • the cropping module 409 is configured to crop the position corresponding to the rectangular frame from the original image based on the rectangular frame for detecting the vehicle in the vehicle detection result, and determine the lamp information of the vehicle in the rectangular frame.
  • the vehicle lamp detection network training module 405 is configured to train the vehicle lamp detection network (corresponding to the second neural network in the above embodiment) according to the vehicle lamp information of the vehicle in the rectangular frame.
  • the vehicle lamp detection result output module 406 is configured to detect the vehicle lamp in the original image through the vehicle lamp detection network to obtain the vehicle lamp detection result.
  • the vehicle lamp detection result includes the position of the vehicle lamp corresponding to the vehicle and the size of the vehicle lamp.
  • the vehicle lamp detection network is not limited to a specific network.
  • the vehicle lamp matching and replacing module 408 is configured to, after detecting the position and size of the vehicle lamp corresponding to the vehicle in the original image and the sub-category corresponding to the vehicle, use the vehicle tail lamp found in the vehicle lamp library 407 and the original image
  • the matched sample taillight map performs light-matching replacement of the taillights in the original image.
  • an image matching method is used to search the corresponding vehicle light library 407 for a sample of a taillight in a night scene of a vehicle of the corresponding category, and the taillight is expanded and retracted to obtain a sample taillight that includes the same size as the taillight of the vehicle.
  • a sample taillight map i.e. a target preset image that matches the taillights of this vehicle.
  • the final result output module 410 is configured to use the sample tail light image to replace the area where the tail light is located in the original image, and use smoothing technology to smooth the surrounding image of the replaced tail light area, so as to obtain the target image, that is, the final night zone. Image result of glowing car taillights.
  • FIG. 5 is a schematic diagram of another application scenario of the image generation method according to the embodiment of the present disclosure, wherein the original image
  • the image 501 (corresponding to the original image in the above embodiment) is an image collected in a daytime scene, and the generation process of the target image is as follows:
  • a sample taillight map including sample taillights matching the taillights in the night scene images 511 , 512 and 513 is determined from the preset image library, and the sample taillight maps are replaced with the taillights in the night scene images 511 , 512 and 513 In the area where it is located, the night scene images 521, 522 and 523 after adding the luminous taillights, that is, the target image, are obtained in sequence.
  • the vehicle type is sub-classified while the vehicle is detected, then the vehicle taillight detection result is performed to obtain the position and size of the vehicle taillight, and finally, according to the vehicle type classification result, the taillight position and size, and match the sample taillights in the taillight library to obtain the final image with taillights; in this way, it can not only make the addition of taillights more natural, but also convert the image of the daytime scene into the vehicle taillight of the night scene image. Bright, more in line with the authenticity of the night scene.
  • FIG. 6 is a schematic structural diagram of an image generating apparatus according to an embodiment of the present disclosure.
  • the apparatus 600 includes:
  • an illumination lamp detection module 601, configured to detect an illumination lamp on the target object in the original image
  • An image conversion module 602 configured to convert the original image into an image to be processed in a specific scene
  • the object determination module 603 is configured to determine, in the to-be-processed image, a lighting lamp whose working state does not match the scene information of the specific scene as an object to be replaced;
  • the replacement image determination module 604 is configured to determine, according to the scene information and the object to be replaced, a replacement image including the object to be replaced whose working state matches the scene information;
  • the target image generation module 605 is configured to use the replacement image to replace the image of the area occupied by the object to be replaced in the image to be processed to generate a target image.
  • the replacement image determination module 604 includes:
  • an attribute determination submodule configured to determine the attribute information of the object to be replaced
  • a target preset image search submodule configured to search a preset image library for a target preset image including a preset object whose attribute matches the attribute information of the object to be replaced;
  • the replacement image determination submodule is configured to determine the found target preset image as the replacement image; wherein, the working state of the preset object in the replacement image matches the scene information.
  • the device further includes:
  • a distance determination module configured to determine the distance between the object to be replaced and the device for collecting the original image
  • the target preset image search sub-module is further configured as:
  • a target preset image including a preset object whose attributes match the attribute information of the object to be replaced is searched for.
  • the device in response to the target preset image not being found in the preset image library, the device further includes:
  • the target preset image generation module is configured to generate a target preset image including a preset object whose attributes match the attribute information of the object to be replaced; and determine the generated target preset image as the replacement image.
  • the target preset image generation module includes:
  • a working parameter determination sub-module configured to determine the working parameters of the preset object according to the distance between the object to be replaced and the device that collects the original image
  • the target preset image generation sub-module is configured to generate a target preset image including the preset object according to the determined working parameters.
  • the distance determination module includes:
  • a size information determining sub-first module configured to determine size information of the object to be replaced according to the category information of the object to be replaced;
  • the distance determination submodule is configured to determine the distance between the device for capturing the original image and the object to be replaced according to the size information and the size of the object to be replaced in the image to be processed.
  • the device after generating the target preset image, the device further includes:
  • An image library updating module configured to store the target preset image in the preset image library.
  • the working parameter of the preset object is the light intensity of the lighting lamp
  • the working parameter determination sub-module includes: a first unit for determining light intensity, configured to determine the light intensity when the lighting lamp is in an activated state; a second unit for determining light intensity, configured to determine the light intensity according to the distance and the lighting lamp Intensity of illumination when in the activated state, determining the intensity of illumination of the lighting lamp that matches the distance;
  • the target preset image generating sub-module is further configured to: generate a target preset image including the preset object with the illumination intensity matching the distance according to the illumination intensity matching the distance.
  • the target image generation module 605 includes:
  • a size information determining sub-second module configured to determine size information of the replacement image
  • an area determination submodule configured to determine the area of the area occupied by the object to be replaced in the image to be processed
  • a replacement image adjustment submodule configured to adjust the size information of the replacement image according to the area to obtain an adjusted image
  • a candidate image generation sub-module configured to replace the image of the area occupied by the object to be replaced with the adjusted image to generate a candidate image
  • the target image generation sub-module is configured to perform smoothing processing on the candidate image to generate the target image.
  • the target objects include traveling equipment and street lamps.
  • the above-mentioned image generation method is implemented in the form of a software function module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.
  • the technical solutions of the embodiments of the present disclosure essentially or the parts that make contributions to the prior art can be embodied in the form of a software product, and the computer software product is stored in a storage medium and includes several instructions for A computer device (which may be a terminal, a server, etc.) is caused to execute all or part of the methods described in the various embodiments of the present disclosure.
  • the aforementioned storage medium includes: a U disk, a mobile hard disk, a read only memory (Read Only Memory, ROM), a magnetic disk or an optical disk and other media that can store program codes.
  • ROM Read Only Memory
  • an embodiment of the present disclosure further provides a computer program product, wherein the computer program product includes computer-executable instructions, and after the computer-executable instructions are executed, the steps in the image generation method provided by the embodiment of the present disclosure can be implemented.
  • the embodiments of the present disclosure further provide a computer storage medium, where computer-executable instructions are stored on the computer storage medium, and when the computer-executable instructions are executed by a processor, the image generation methods provided by the above embodiments are implemented. step.
  • an embodiment of the present disclosure provides a computer device.
  • FIG. 7 is a schematic structural diagram of a computer device according to an embodiment of the present disclosure. As shown in FIG.
  • the computer device 700 includes: a processor 701 , at least one communication bus, Communication interface 702 , at least one external communication interface and memory 703 .
  • the communication interface 702 is configured to realize the connection communication between these components.
  • the communication interface 702 may include a display screen, and the external communication interface may include a standard wired interface and a wireless interface.
  • the processor 701 is configured to execute an image processing program in the memory, so as to implement the image generation method provided by the above embodiments.
  • the disclosed apparatus and method may be implemented in other manners.
  • the device embodiments described above are only illustrative.
  • the division of the units is only a logical function division. In actual implementation, there may be other division methods.
  • multiple units or components may be combined, or Can be integrated into another system, or some features can be ignored, or not implemented.
  • the coupling, or direct coupling, or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be electrical, mechanical or other forms. of.
  • each functional unit in each embodiment of the present disclosure may be all integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above integration
  • the unit can be implemented either in the form of hardware or in the form of hardware plus software functional units.
  • the aforementioned program can be stored in a computer-readable storage medium, and when the program is executed, the execution includes: The steps of the above method embodiments; and the aforementioned storage medium includes: a removable storage device, a read only memory (Read Only Memory, ROM), a magnetic disk or an optical disk and other media that can store program codes.
  • ROM Read Only Memory
  • the above-mentioned integrated units of the present disclosure are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.
  • the technical solutions of the embodiments of the present disclosure essentially or the parts that make contributions to the prior art can be embodied in the form of a software product, and the computer software product is stored in a storage medium and includes several instructions for A computer device (which may be a personal computer, a server, or a network device, etc.) is caused to execute all or part of the methods described in the various embodiments of the present disclosure.
  • the aforementioned storage medium includes various media that can store program codes, such as a removable storage device, a ROM, a magnetic disk, or an optical disk.
  • Embodiments of the present disclosure provide an image generation method, apparatus, device, and storage medium, wherein the lighting on a target object in an original image is detected; the original image is converted into an image to be processed in a specific scene; In the image to be processed, it is determined that the lighting lamp whose working state does not match the scene information of the specific scene is the object to be replaced; A replacement image of the object to be replaced; the replacement image is used to replace the image of the area occupied by the object to be replaced in the image to be processed to generate a target image.

Abstract

一种图像生成方法，包括：检测原始图像中的目标对象上的照明灯；将原始图像转换为特定场景下的待处理图像；在待处理图像中，确定工作状态与特定场景的场景信息不匹配的照明灯为待替换对象；根据场景信息以及待替换对象，确定包括工作状态与场景信息相匹配的待替换对象的替换图像；采用替换图像替换待处理图像中待替换对象所占据的区域的图像，生成目标图像。还公开了一种图像生成装置、设备及存储介质。

Description

一种图像生成方法、装置、设备及存储介质
相关申请的交叉引用
本公开基于申请号为202110351897.2、申请日为2021年3月31日的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此引入本公开作为参考。
技术领域
本公开实施例涉及智能驾驶技术领域，涉及但不限于一种图像生成方法、装置、设备及存储介质。
背景技术
在相关技术的图像生成方法中，当需要将一幅白天自动驾驶场景图像通过图像生成的方式转化为夜晚场景的图像时，由于转换后的图像的车尾灯也是暗的，使得生成的图像的真实性不高。
发明内容
有鉴于此，本公开实施例提供一种图像生成技术方案。
本公开实施例的技术方案是这样实现的：
本公开实施例提供一种图像生成方法，所述方法包括：检测原始图像中的目标对象上的照明灯；将所述原始图像转换为特定场景下的待处理图像；在所述待处理图像中，确定工作状态与所述特定场景的场景信息不匹配的照明灯为待替换对象；根据所述场景信息以及所述待替换对象，确定包括工作状态与所述场景信息相匹配的待替换对象的替换图像；采用所述替换图像替换所述待处理图像中所述待替换对象所占据的区域的图像，生成目标图像。
在一些实施例中，所述根据所述场景信息以及所述待替换对象，确定包括工作状态与所述场景信息相匹配的待替换对象的替换图像，包括：确定所述待替换对象的属性信息；在预设图像库中，查找包括属性与所述待替换对象的属性信息相匹配的预设对象的目标预设图像；将查找到的目标预设图像，确定为所述替换图像；其中，所述预设对象在所述替换图像中的工作状态与所述场景信息相匹配。如此，能够提高替换后的目标图像的逼真度和合理性。
在一些实施例中，所述方法还包括：确定所述待替换对象与采集所述原始图像的装置之间的距离；所述在预设图像库中，查找包括属性与所述待替换对象的属性信息相匹配的预设对象的目标预设图像，包括：根据所述待替换对象的类别信息以及确定的距离，在所述预设图像库中，查找包括属性与所述待替换对象的属性信息相匹配的预设对象的目标预设图像。如此，通过将待替换对象的类别与距离相结合，在预设图像库中查找目标预设图像，能够提高查找到的目标预设图像的准确度。
在一些实施例中，响应于在所述预设图像库中查找不到所述目标预设图像，所述方法还包括：生成包括属性与所述待替换对象的属性信息相匹配的预设对象的目标预设图像；将生成的目标预设图像，确定为所述替换图像。如此，在查找不到目标预设图像的情况下，自动生成目标预设图像，能够提高替换图像的精确性。
在一些实施例中，所述生成包括属性与所述待替换对象的属性信息相匹配的预设对象的目标预设图像，包括：根据所述待替换对象与采集所述原始图像的装置之间的距离，确定所述预设对象的工作参数；根据确定的工作参数，生成包括所述预设对象的目标预设图像。如此，这样，将夜晚待处理图像中的未发光的车灯图像替换为发光的替换图像，从而使得生成的目标图像更加逼真。
在一些实施例中，所述确定所述待替换对象与采集所述原始图像的装置之间的距离，包括：根据所述待替换对象的类别信息，确定所述待替换对象的尺寸信息；根据所述尺寸信息以及所述待处理图像中所述待替换对象的尺寸，确定采集所述原始图像的装置与所述待替换对象之间的距离。如此，通过分析待替换对象的尺寸，能够更加精确地得到采集装置与对象之间的距离。
在一些实施例中，在生成所述目标预设图像之后，所述方法还包括：将所述目标预设图像存储在所述预设图像库中。这样，能够丰富预设图像库，便于后续在预设图像库中查找目标预设图像。
在一些实施例中，所述预设对象的工作参数为照明灯的光照强度；所述根据所述待替换对象与采集所述原始图像的装置之间的距离，确定所述预设对象的工作参数，包括：确定所述照明灯处于启动状态时的光照强度；根据所述距离以及所述照明灯处于启动状态时的光照强度，确定所述照明灯的与所述距离相匹配的光照强度；所述根据确定的工 作参数，生成包括所述预设对象的目标预设图像，包括：根据所述与距离相匹配的光照强度，生成包括具有与距离相匹配的光照强度的所述预设对象的目标预设图像。如此，能够提高最终得到的替换对象是与场景信息之间的匹配度。
在一些实施例中，所述采用所述替换图像替换所述待处理图像中所述待替换对象所占据的区域的图像，生成目标图像，包括：确定所述替换图像的尺寸信息；确定所述待替换对象在所述待处理图像中所占据的区域的面积；根据所述面积，对所述替换图像的尺寸信息进行调整，得到已调整图像；采用所述已调整图像替换所述待替换对象所占据的区域的图像，生成候选图像；对所述候选图像进行平滑处理，生成所述目标图像。如此，使得生成的目标图像更加的合理且清晰。
在一些实施例中，所述目标对象包括行驶设备、路灯。
本公开实施例提供一种图像生成装置，所述装置包括：照明灯检测模块，配置为检测原始图像中的目标对象上的照明灯；图像转换模块，配置为将所述原始图像转换为特定场景下的待处理图像；对象确定模块，配置为在所述待处理图像中，确定工作状态与所述特定场景的场景信息不匹配的照明灯为待替换对象；替换图像确定模块，配置为根据所述场景信息以及所述待替换对象，确定包括工作状态与所述场景信息相匹配的待替换对象的替换图像；图像生成模块，配置为采用所述替换图像替换所述待处理图像中所述待替换对象所占据的区域的图像，生成目标图像。
本公开实施例提供一种计算机存储介质，所述计算机存储介质上存储有计算机可执行指令，该计算机可执行指令被执行后，能够实现上述所述的图像生成方法。
本公开实施例提供一种电子设备，所述电子设备包括存储器和处理器，所述存储器上存储有计算机可执行指令，所述处理器运行所述存储器上的计算机可执行指令时可实现上述所述的图像生成方法。
本公开实施例提供一种计算机程序，包括计算机可读代码，当所述计算机可读代码在电子设备中运行时，所述电子设备中的处理器执行配置为实现上述任意一项所述的图像生成方法。
本公开实施例提供一种图像生成方法、装置、设备及存储介质，首先将原始图像转换为特定场景下的待处理图像；然后，将待处理图像中目标对象上的、工作状态与特定场景的场景信息不匹配的照明灯作为待替换对象，并根据场景信息以及待替换对象，确定包括工作状态与场景信息相匹配的待替换对象的替换图像；最后，采用该替换图像替换待处理图像中待替换对象所占据的区域的图像，以生成目标图像；如此，使得目标图像中的对象的工作状态与场景信息相匹配，从而使得生成的目标图像更加符合真实场景。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，这些附图示出了符合本公开实施例的实施例，并与说明书一起用于说明本公开实施例的技术方案。
图1A为可以应用本公开实施例的轨迹预测方法的一种系统架构示意图；
图1B为本公开实施例图像生成方法的实现流程示意图；
图2为本公开实施例提供的图像生成方法的另一实现流程示意图；
图3A为本公开实施例提供的图像生成系统的组成结构示意图；
图3B为本公开实施例图像生成方法的应用场景示意图；
图4为本公开实施例提供的图像生成方法的实现框架结构图；
图5为本公开实施例图像生成方法的另一应用场景示意图；
图6为本公开实施例图像生成装置结构组成示意图；
图7为本公开实施例计算机设备的组成结构示意图。
具体实施方式
为使本公开实施例的目的、技术方案和优点更加清楚，下面将结合本公开实施例中的附图，对发明的具体技术方案做进一步详细描述。以下实施例用于说明本公开，但不用来限制本公开的范围。
在以下的描述中，涉及到“一些实施例”，其描述了所有可能实施例的子集，但是可以理解，“一些实施例”可以是所有可能实施例的相同子集或不同子集，并且可以在不冲突的情况下相互结合。
在以下的描述中，所涉及的术语“第一\第二\第三”仅仅是区别类似的对象，不代表针对对象的特定排序，可以理解地，“第一\第二\第三”在允许的情况下可以互换特定的顺序或先后次序，以使在一些实施例中描述的本公开实施 例能够以除了在一些实施例中图示或描述的以外的顺序实施。
除非另有定义，本文所使用的所有的技术和科学术语与属于本公开的技术领域的技术人员通常理解的含义相同。本文中所使用的术语只是为了描述本公开实施例的目的，不是旨在限制本公开。
对本公开实施例进行进一步详细说明之前，对本公开实施例中涉及的名词和术语进行说明，本公开实施例中涉及的名词和术语适用于如下的解释。
1)高斯模糊：对于图像来说就是一个低通滤波器。所谓"模糊"，可以理解成每一个像素都取周边像素的平均值。
2)自主车辆(ego vehicle)：包含感知周围环境传感器的车辆。车辆坐标系固连在自主车辆上，其中，x轴为汽车前进的方向，y轴指向车辆前进方向的左侧，z轴垂直于地面向上，符合右手坐标系。坐标系原点位于后轴中点下方的大地上。
下面说明本公开实施例提供的图像生成的设备的示例性应用，本公开实施例提供的设备可以实施为具有图像采集功能的笔记本电脑，平板电脑，台式计算机，相机，移动设备(例如，个人数字助理，专用消息设备，便携式游戏设备)等各种类型的用户终端，也可以实施为服务器。下面，将说明设备实施为终端或服务器时示例性应用。
该方法可以应用于计算机设备，该方法所实现的功能可以通过计算机设备中的处理器调用程序代码来实现，当然程序代码可以保存在计算机存储介质中，可见，该计算机设备至少包括处理器和存储介质。
图1A为可以应用本公开实施例的图像生成方法的一种系统架构示意图；如图1A所示，该系统架构中包括：图像采集设备131、网络132和图像生成终端133。为实现支撑一个示例性应用，图像采集设备131和图像生成终端133可以通过网络132建立通信连接，图像采集设备131通过网络202向图像生成终端133上报采集到的原始图像(或者，图像生成终端133自动获取车辆终端131的采集到的原始图像)，图像生成终端133响应于接收到的原始图像，首先，检测原始图像中的照明灯，并将该图像转换为一定场景下的待处理图像；然后，在待处理图像中，找出工作状态不合理的照明灯；最后，采用工作状态与场景信息匹配的替换图像替换掉不合理的照明灯对应的图像区域，从而生成目标图像，并在图像生成终端133的图像显示界面上输出目标图像。如此，使得目标图像中的对象的工作状态与场景信息相匹配，从而使得生成的目标图像更加符合真实场景。
作为示例，图像采集设备131可以为包括摄像头等的采集设备。图像生成终端133可以包括具有一定计算能力的计算机设备，该计算机设备例如包括：终端设备或服务器或其它处理设备。网络132可以采用有线连接或无线连接方式。其中，在图像生成终端133为服务器时，图像采集设备131可以通过有线连接的方式与服务器通信连接，例如通过总线进行数据通信；在图像生成终端133为终端设备时，图像采集设备131可以通过无线连接的方式与图像生成终端133通信连接，进而进行数据通信。
或者，在一些场景中，图像生成终端133可以是带有视频采集模组的视觉处理设备，可以是带有摄像头的主机。这时，本申请实施例的图像生成方法可以由图像生成终端133执行，上述系统架构可以不包含网络132和图像采集设备131。
图1B为本公开实施例图像生成方法的实现流程示意图，如图1B所示，结合如图1B所示步骤进行说明：
步骤S101，检测原始图像中的目标对象上的照明灯。
在一些实施例中，原始图像可以是任意场景下采集的图像，可以是包括画面内容复杂的图像还可以是包括画面内容简单的图像，比如，在深夜采集的街道场景的图像，或者在白天采集的街道场景的图像等。目标对象为车辆或者路灯等具有照明灯的对象。目标对象包括行驶设备和路灯等工作状态可变的设备，工作状态可变的设备，包括：具有至少两种工作状态的可移动设备，比如，各种各样功能的车辆(如卡车、汽车、摩托车、自行车等)、各种轮数的车辆(如四轮车辆、两轮车辆等)和任意可移动设备(如机器人、飞行器、导盲器、智能家具设备或智能玩具等)等。或者具有至少两种工作状态的固定设备，比如，各种各样的道路照明灯(如高杆路灯、中杆灯、道路灯、庭院灯、草坪灯或景观灯等)。下面不妨以车辆为例进行说明。比如，待处理图像为夜晚场景的道路图像，其中，工作状态可变的设备为道路上行进的车辆上的照明灯。
步骤S102，将原始图像转换为特定场景下的待处理图像。
在一些实施例中，特定场景的场景信息可以包括场景中光线明暗程度、场景所在的位置以及场景中的对象等。比如，待处理图像是深夜场景下的街道图像，那么场景信息包括：该街道的明亮程度、该街道的位置以及街道上的车辆和路灯等对象。
在一些可能的实现方式中，待处理图像可以是特定场景下的图像，其中，特定场景可以是设定的任意场景。比如，深夜的场景、傍晚的场景或凌晨的场景等。
在步骤S102中，将原始图像转换为特定场景下的待处理图像可以通过以下步骤实现：
第一步，获取原始图像。
在一些可能的实现方式中，原始图像为任意场景下采集到的图像，比如，在白天采集的道路图像或者在夜晚采集的道路图像等。
第二步，确定原始图像的场景信息。
在一些可能的实现方式中，获取到原始图像之后，通过训练好的判别器，判断该图像的场景信息是否为特定场景的场景信息。比如，特定场景的场景信息为夜晚场景，通过判别器判断该图像的场景信息是否为夜晚场景。
第三步，在所述场景信息与所述特定场景的场景信息不匹配的情况下，根据所述特定场景的场景信息，对所述原始图像的场景信息进行转换，得到转换图像。
在一些可能的实现方式中，在场景信息与特定场景的场景信息相差较远的情况下，将原始图像的场景信息转换为特定场景的场景信息，即将原始图像转换为特定场景下的图像，从而得到转换图像。比如，特定场景的场景信息为夜晚场景，原始图像为白天场景下采集到的图像，那么将白天场景下采集的原始图像转换为夜晚场景的图像，可以通过将原始图像输入生成器，生成对应的夜晚待处理图像。
在一些实施例中，在场景信息与所述特定场景的场景信息相匹配的情况下，将所述原始图像确定为所述待处理图像。
比如，场景信息与特定场景的场景信息相同或极为相近的情况下，说明原始图像即具有特定场景的场景信息，所以不需要对原始图像进行图像转换，即可将原始图像作为待处理图像。在一个具体例子中，特定场景的场景信息为深夜场景，原始图像为夜幕降临的场景下采集到的图像，该场景与特定场景相似均为夜晚，那么将原始图像确定为待处理图像。
第四步，将转换图像确定为待处理图像。
通过上述第一步至第四步，在获取原始图像之后，通过对原始图像的场景信息进行判断，在场景信息不是特定场景的场景信息的情况下，对原始图像的场景信息进行转换，从而得到具有特定场景的场景信息的待处理图像。
步骤S103，在待处理图像中，确定工作状态与特定场景的场景信息不匹配的照明灯为待替换对象。
在一些实施例中，照明灯的工作状态与特定场景的场景信息相关联，可以理解为照明灯的工作状态应该随着特定场景的场景信息的变化而变化。比如，如果特定场景的场景信息从白天场景转换为夜晚场景，那么照明灯的工作状态也随着发生改变。在一个具体例子中，如果待处理图像为具有夜晚场景的道路图像，照明灯为该图像中运行的车辆的车灯，在真实情况下，夜晚场景的车辆的照明灯(如车尾灯)应该是处于发光状态，如果在待处理图像中车灯是处于未发光状态，那么确定该车辆的车灯为待替换对象。而白天场景的车辆的车灯应该是处于未发光状态，如果在待处理图像中车灯是处于发光状态，那么确定该车辆的车灯为待替换对象。或者，待替换对象为路灯，那么在实际场景中，夜晚场景的路灯应该是处于发光状态，而白天场景的路灯应该是处于未发光状态。
在其他实施例中，还可以将待处理图像中包括工作状态与所述特定场景的场景信息不匹配的照明灯的整个目标对象作为待替换对象。比如，特定场景的场景信息为夜晚场景，目标对象为车辆，该车辆中包括处于未发光状态的车灯，可以将该车辆作为待替换对象。
步骤S104，根据场景信息以及待替换对象，确定包括工作状态与场景信息相匹配的待替换对象的替换图像。
在一些实施例中，替换图像包括的待替换对象的工作状态与场景信息相匹配，即替换图像包括的照明灯的工作状态与场景信息相匹配。待处理图像中待替换对象的工作状态与场景信息不匹配，可以理解为是，待替换对象的工作状态不是该场景下合理的状态，比如，场景信息为夜晚场景，待替换对象为车灯，待替换对象的工作状态为未发光状态，即说明待替换对象的工作状态与场景信息不匹配。
替换图像中的待替换对象的工作状态与场景信息相匹配，可以理解为，替换图像中的待替换对象的工作状态是在场景下合理的状态。比如，场景信息为夜晚场景，待替换对象为车灯，那么替换图像中的待替换对象的工作状态为发光状态，对应的，替换图像即为采集的处于发光状态的车灯的图像。
步骤S105，采用所述替换图像替换待处理图像中待替换对象所占据的区域的图像，生成目标图像。
在一些实施例中，在待处理图像中，通过确定出工作状态与场景信息不匹配的待替换对象，并确定出待替换对象在待处理图像中占据的区域；采用工作状态与场景信息匹配的替换对象所对应的替换图像替换该区域的图像，并针对替换后的图像进行平滑处理，从而生成目标图像。
在一个具体例子中，以场景信息为夜晚场景为例，待处理图像为具有夜晚场景的道路图像，工作状态随着场景的变化而变化的照明灯为车辆中的车灯；在车灯的工作状态为未发光状态时，即该工作状态与夜晚场景不匹配，在这种情况下，按照该车灯的规格，在预设图像库中查找包括与该车灯的规格相同的预设车灯的替换图像；其中，替换图像中的待替换对象的工作状态与场景匹配，即在该替换图像中预设车灯的工作状态为发光状态。最后，采用该替换图像替换待处理图像中车灯所在的区域的图像，从而生成目标图像。如此，生成的目标图像中车灯的工作状态为发光状态，与夜晚场景相匹配，从而使得生成的目标图像更加逼真。
在本公开实施例中，通过在待处理图像中，确定出工作状态与场景信息不匹配的待替换对象；然后，利用替换图像替换待替换对象占据的区域的图像，从而使得生成的目标图像中包括工作状态匹配场景信息的待替换对象，进而使得生成的目标图像更加生动逼真。
在其他实施例中，将包括工作状态与所述场景信息相匹配的待替换对象的替换图像存储在预设图像库中，可以用来实时渲染待替换对象，从而可以生成动画播放的目标图像，提高生成的目标图像的丰富性，实现过程如下，在本公开实施例中以待替换对象为车灯为例进行说明：
首先，确定待替换对象的工作模式。
在一些可能的实现方式中，车灯是指车辆上的照明灯，可以是前照灯、吾雾灯、倒车灯、牌照灯等等。车灯的工作模式至少包括：闪烁模式、远光模式和近光模式等。
如果工作模式为预设工作模式，在预设图像库中，查找包括与所述待替换对象相匹配的对象，且按照所述预设工作模式随时间序列发生变化的多个目标预设图像，得到目预设图像集合。
在一些可能的实现方式中，预设工作模式可以设定为闪烁模式，即如果工作模式为闪烁模式，在预设图像库中，确定该车灯在闪烁的过程中随时间序列变化的多个目标预设图像，得到目标预设图像集合。
最后，按照随时间序列变化的多个目标预设图像，对当前的待替换对象进行实时渲染，生成待替换对象处于预设工作模式的目标动画，并以动画格式播放生成的目标动画。
在本公开实施例中，对于待替换对象，由于采用灯取代了发光二极管(Light Emitting Diode，LED)，所以车灯可以发出各种颜色的光以及，通过多种发光模式进行工作，比如，通过闪烁模式工作。在这种情况下，本公开实施例不仅能够输出静止的目标图像，还可以采用动画格式的目标图像。由于将包括工作状态与场景信息相匹配的对象的替换图像存储在预设图像库中，从而能够通过随时间序列变化的预设图像渲染待替换对象，以生成并播放目标动画。
在一些实施例中，为了提高确定待替换对象的准确度，通过在待处理图像中确定目标对象，并将该目标对象中工作状态与场景信息不匹配的照明灯，作为待替换对象，可以通过以下步骤实现：
第一步，确定待处理图像中的工作状态可变的照明灯。
在一些可能的实现方式中，在待处理图像中的目标对象中，确定出工作状态与所述场景信息具有关联关系的照明灯。该照明灯的工作状态虽与场景信息具有关联关系，但是与场景信息并不一定匹配。
第二步，确定工作状态与场景信息不匹配的照明灯，作为待替换对象。
在一些可能的实现方式中，如果目标对象为可移动设备，那么待替换对象为可移动设备中工作状态与场景信息不匹配的照明灯，比如，目标对象为车辆，那么待替换对象为车辆中工作状态与场景信息不匹配的车灯(如车头灯、前车灯或尾灯等)。如果目标对象为固定设备，那么待替换对象为固定设备中工作状态与场景信息不匹配的照明灯；比如，目标对象为高杆路灯，那么待替换对象为高杆路灯中工作状态与场景信息不匹配的灯。目标对象还可以为固定照明设备，该固定照明设备中工作状态与所述场景信息不匹配的照明灯为照明灯，即待替换对象为所述固定照明设备上的照明灯；在待处理图像中，该照明灯的工作状态可以是发光或者不发光，即待处理图像中的照明灯可以处于亮灯状态，也可以处于灯不亮状态。
在一些可能的实现方式中，以固定照明设备为高杆路灯，待替换对象为该路灯的灯具为例，如果待处理图像的场景信息为白天场景，比如，待处理图像为通过对夜晚采集的街道场景下的原始图像进行场景转换后得到的白天场景的图像。假如在原始图像中街道上的路灯是不亮的，而且进行图像转换后，待处理图像是白天场景的图像，路灯自然不需要处于亮灯状态；所以这种情况下，不需要对待处理图像上的路灯进行替换；如果待处理图像的场景信息为夜晚场景，比如，待处理图像为通过对白天采集的街道场景下的原始图像进行场景转换后得到的夜晚场景的图像。假如在原始图像中街道上的路灯是不亮的，进行图像转换后，待处理图像是夜晚场景的图像，路灯需要处于亮灯状态；这种情况下，需要确定包括处于亮灯状态的路灯的替换图像，将该替换图像替换待处理图像中待替换对象所占据的区域的图 像，生成目标图像。
在一些可能的实现方式中，目标对象可以为行驶设备，该行驶设备中工作状态与所述场景信息不匹配的照明灯为行驶设备的照明灯，比如，行驶设备为车辆，那么待替换对象为车灯；在待处理图像中，该车辆上的车灯的工作状态可以是发光或者不发光，即待处理图像中的车灯可以是处于亮灯状态，也可以是处于灯不亮状态。以行驶设备为车辆，待替换对象为车尾灯为例，如果待处理图像的场景信息为白天场景，比如，待处理图像为通过对夜晚采集的街道场景下的原始图像进行场景转换后得到的白天场景的图像。假如在原始图像中街道上的车辆的前照灯是不亮的，而且进行图像转换后，待处理图像是白天场景的图像，前照灯并不需要处于亮灯状态；所以这种情况下，不需要对待处理图像上的前照灯进行替换；如果待处理图像的场景信息为夜晚场景，比如，待处理图像为通过对白天采集的街道场景下的原始图像进行场景转换后得到的夜晚场景的图像，假如在原始图像中街道上的前照灯是不亮的，进行图像转换后，待处理图像是夜晚场景的图像，前照灯需要处于亮灯状态；这种情况下，需要确定包括处于亮灯状态的前照灯的替换图像，将该替换图像替换待处理图像中待替换对象所占据的区域的图像，生成目标图像。
在本公开实施例中，通过在待处理图像中确定出存在多种工作状态的目标对象，并针对该目标对象中工作状态与场景信息不匹配的照明灯进行替换，从而使得替换后的目标图像与场景信息更加匹配，画面内容更合理生动。
在一些实施例中，待替换图像可以是从预设图像库中查找得到，还可以是通过分析待替换对象的属性信息生成该待替换图像，即在步骤S104中，可以通过以下两种方式确定待替换图像，其中：
方式一：在预设图像库中，查找包括与待替换对象相匹配的预设对象的替换图像。
在一些实施例中，预设图像库中存储的预设对象的工作状态与场景信息相匹配。比如，场景信息为夜晚场景，那么预设图像库中存储的是处于发光状态的车灯图像。与待替换对象相匹配的目标预设图像，可以理解为是与待替换对象至少规格相同或类型相同的目标预设图像。比如，待替换对象为车灯，那么目标预设图像为与该车灯规格相同的车灯图像。
在一些可能的实现方式中，可以通过以下过程在预设图像库中查找替换图像，如图2所示，图2为本公开实施例提供的图像生成方法的另一实现流程示意图，步骤S104可以通过步骤S201至S203实现，结合图2所示的步骤进行说明：
步骤S201，确定待替换对象的属性信息。
在一些可能的实现方式中，可借助于训练好的第二神经网络来查找替换图像，从而提高确定替换图像的准确度，即通过第二神经网络，确定待替换对象的属性信息。其中，待替换对象的属性信息为用于表征待替换对象规格和类型等，描述待替换对象自身的信息；比如，待替换对象为车灯，属性信息为该车灯的规格、该车灯具体的种类和车灯的左右侧等；其中，车灯的种类包括：后位灯、左尾灯、右尾灯、前大灯等。
步骤S202，在预设图像库中，查找包括属性与待替换对象的属性信息相匹配的预设对象的目标预设图像。
在一些可能的实现方式中，首先，查找属性信息与待替换对象的属性信息相同的预设对象；然后，在预设图像库中，查找包括该预设对象的目标预设图像。
步骤S203，将查找到的目标预设图像，确定为所述替换图像。
在一些可能的实现方式中，预设对象在替换图像中的工作状态与场景信息相匹配。在一个具体例子中，待替换对象为左右两个尾灯，那么在预设图像库中查找包括与该车灯的属性信息相同的且处于发光状态的尾灯的目标预设图像。因为左尾灯和右尾灯在外观形状上相同，所以确定出的目标预设图像可以是一帧图像，也可以分别与左尾灯和右尾灯相同的两个对称图像。如果目标预设图像是一帧图像，那么在采用目标预设图像替换待替换对象占据的区域时，采用该目标预设图像按照左右尾灯的形状分别进行替换，使得替换后的目标图像中左右尾灯仍然对称。同理，如果待替换对象为前大灯或后位灯，那么当采用目标预设图像对待替换对象所在的区域进行替换时，通过待替换对象的位置信息确定是前大灯还是后位灯，以使得替换后的目标图像更加合理且逼真。
在方式一中，响应于待替换对象为目标对象中工作状态与所述场景信息不匹配的照明灯，在预设图像库中，查找目标预设图像，可以通过以下过程实现：
首先，确定所述待替换对象与采集原始图像的装置之间的距离。
在一些实施例中，待处理图像由原始图像转换得到，即待处理图像是通过对原始图像进行图像转换得到的；比如，原始图像为白天场景下采集的街道图像，将该白天场景转换为夜晚场景，得到待处理图像。首先，通过确定待替换对象的尺寸，然后，基于该尺寸、待替换对象在待处理图像中占据的像素数和采集原始图像的装置的焦距，可计算该距离。
在一可能的实现方式中，通过根据待替换对象的类别信息，确定待替换对象的尺寸信息，并根据待替换对象的尺寸信息以及待处理图像中待替换对象的尺寸，确定采集原始图像的装置与待替换对象之间的距离。
在一些可能的实现方式中，待替换对象的尺寸信息包括待替换对象的长度、宽度和高度等。待替换对象的类别信息包括：待替换对象所属的品牌或类型等，以目标对象为行驶设备为例进行说明，目标对象的类别信息包括：小轿车，城郊多用途汽车(Suburban Utility Vehicle，SUV)，面包车，小卡车，大卡车，大巴车，公交车或箱式货车等；那么待替换对象为小轿车，SUV，面包车，小卡车，大卡车，大巴车，公交车或箱式货车等中工作状态与特定场景的场景信息不匹配的车灯。以目标对象为固定照明设备，目标对象的类别信息包括：各种类型的路灯、工地照明灯、探照灯、建筑照明灯、船用灯或民用灯；那么待替换对象为各种类型的路灯、工地照明灯、探照灯、建筑照明灯、船用灯或民用灯中工作状态与特定场景的场景信息不匹配的灯具。
确定待替换对象的类别信息之后，即可确定目标对象的规格，从而得到目标对象的尺寸信息。比如，确定目标对象的类别信息为面包车，进一步确定是哪一款哪一规格的面包车，从而得到该面包车的宽度。以目标对象为面包车为例，确定出该面包车的真实宽度之后，即可确定待替换对象(即面包车车灯)的真实宽度在待处理图像中对应的像素数目；然后，结合图像采集装置的焦距(比如，相机的焦距)，通过像素数目与该焦距的商来估算图像采集装置与待替换对象之间的距离。
然后，根据所述待替换对象的类别信息以及确定的距离，在所述预设图像库中，查找包括属性与所述待替换对象的属性信息相匹配的预设对象的目标预设图像。
在一些实施例中，不同类别的待替换对象，在不同距离下，待替换对象的工作参数不同，所以，通过将待替换对象的类别与距离相结合，在预设图像库中查找目标预设图像，能够提高查找到的目标预设图像的准确度。以目标对象为固定照明设备，所述待替换对象为所述固定照明设备上的照明灯，或，以目标对象为行驶设备，待替换对象为行驶设备上的照明灯，为例进行说明，图像采集装置与照明灯的距离不同，照明灯的光照强度也不同，基于该距离下的光照强度，在预设图像库中查找与该光照强度相同的照明灯，将具有该照明灯的目标预设图像，作为替换图像。
方式二：响应于在预设图像库中查找不到目标预设图像，可以生成包括属性与待替换对象的属性信息相匹配的预设对象的目标预设图像，并将生成的目标预设图像，作为替换图像，并且基于生成的目标预设图像更新预设图像库，可以通过以下过程实现：
首先，确定所述待替换对象与采集所述原始图像的装置之间的距离。
在一些实施例中，如果目标对象为行驶设备，那么确定的距离为：行驶设备的照明灯与采集原始图像的装置之间的距离。比如，行驶设备为车辆，那么确定的距离为车辆的车灯和图像采集装置之间的距离。
其次，根据待替换对象与采集所述原始图像的装置之间的距离，确定预设对象的工作参数。
在一些实施例中，预设对象的工作参数包括：该预设对象在正常工作时的各项数据，参数的类别与预设对象的类别对应，包括：预设对象的工作功率和工作强度等；比如，预设对象为照明灯(行驶设备的照明灯或者固定照明设备的照明灯)，那么照明灯的工作参数至少包括该照明灯的光照强度，且距离不同，照明灯的光照强度不同，距离越大，照明灯的光照强度越弱，即工作参数越小。
再次，根据确定的工作参数，生成包括预设对象的目标预设图像。
在一些实施例中，在确定预设对象的工作参数之后，可以按照该工作参数，生成具有处于该工作参数对应的工作状态下的对象的目标预设图像，以得到替换图像。在一个具体例子中，如果预设对象为照明灯，工作参数为该照明灯的光照强度，那么根据该光照强度，生成具有该光照强度的待替换对象的替换图像。这里，确定该距离下的光照强度之后，即可生成用于该光照强度的待替换对象的替换图像。在确定出替换图像之后，可将替换图像，以及替换图像与距离之间的对应关系存储在预设图像库中，以得到更新的预设图像库。
再次，将待替换对象占据的区域替换为目标预设图像，生成目标图像。
在一些可能的实现方式中，生成替换图像之后，首先，确定待替换对象在待处理图像中占据的区域大小，然后基于该大小调整替换图像的尺寸，以使调整后的替换图像与待替换对象占据的区域大小契合，从而采用调整后的替换图像替换待替换对象所占据的区域的图像，使得生成的目标图像质量较高。在一个具体例子中，待处理图像为夜晚场景的道路图像，目标对象为道路上运行的车辆，待替换对象为目标对象中未发光的车灯，比如，夜景图像中车辆未发光的车尾灯，通过基于图像采集的装置与所述目标对象的照明灯之间的距离，生成具有该距离相匹配的光强度的车灯的替换图像，利用该替换图像替换待处理图像中的车尾灯。这样，将夜晚待处理图像中的未发光的车灯图像替换为发光 的替换图像，从而使得生成的目标图像更加逼真。
最后，将目标预设图像存储在预设图像库中。
在一些实施例中，通过确定待替换对象的类别信息，来确定该设备的尺寸，从而能够确定出该设备与图像采集装置之间的距离，基于此，按照距离与预设对象的工作参数之间的对应关系，可在关系对应表中确定出该距离匹配的工作参数，从而生成具有处于该工作参数对应的工作状态下的对象的目标预设图像。最后，将距离与工作参数之间的对应关系、待替换对象的类别信息与尺寸信息之间的对应关系和生成的目标预设图像，均存储在预设图像库中，以更新该预设图像库。从而当再次需要从预设图像库中查找目标预设图像时，能够提供更丰富的预设对象以供选择，进而提高选择的目标预设图像的准确度。
在一些可能的实现方式中，如果目标对象为行驶设备或固定照明设备，预设对象为固定照明设备上的照明灯或行驶设备上的照明灯，预设对象的工作参数为照明灯的光照强度，通过确定预设对象的工作参数，生成包括预设对象的目标预设图像的过程，可以通过以下步骤实现：
第一步，确定照明灯处于启动状态时的光照强度。
在一些实施例中，该照明灯可以是任意规格的照明灯，比如，功率较小的路灯或者功率较大的探照灯，那么确定该照明灯处于已启动状态的光照强度。
第二步，根据距离以及照明灯处于启动状态时的光照强度，确定照明灯的与距离相匹配的光照强度。
在一些实施例中，首先，确定照明灯处于启动状态时的光照强度与该距离之间的对应关系，然后，基于该对应关系，确定不同距离下照明灯的光照强度。光照强度与该距离之间的对应关系为，距离越大光照强度越小；换言之，由于图像采集的装置与待替换对象之间的距离不同，采集到的待替换对象的光照强度不同，该光照强度与距离成反比，即图像采集的装置与待替换对象之间的距离越大，采集到的待替换对象的光照强度越小。这样，通过测量多个距离与光照强度，可以创建表征光照强度与距离之间对应关系的关系对应表，在需要依据距离，确定照明灯的光照参数时，可以从该表中查找与距离相匹配的光照强度。
第三步，根据与距离相匹配的光照强度，生成包括具有与距离相匹配的光照强度的预设对象的目标预设图像。
在一些实施例中，按照该光照强度，渲染生成具有该光照强度的预设对象的图像，即目标预设图像。
在上述方式一和上述方式二中实现了“确定包括工作状态与场景信息相匹配的待替换对象的替换图像”，并且可通过第二神经网络确定出待替换对象的属性信息，在预设图像库中查找包括与该属性信息相同的预设对象的目标预设图像；如果在预设图像库中查找不到目标预设图像，那么通过综合考虑待替换对象的类别信息以及该设备与图像采集装置之间的距离，确定与该距离匹配的预设对象的工作参数；从而生成包括具有该工作参数的预设对象的目标预设图像，并通过将目标预设图像存储在预设图像库中更新该预设图像库，从而提高了最终得到的替换对象是与场景信息之间的匹配度。
在一些实施例中，上述确定替换对象的过程，可以通过神经网络来实现，该神经网络包括第一神经网络和第二神经网络，利用第一神经网络确定目标对象的类别信息；通过第二神经网络，根据类别信息，确定目标对象中工作状态与所述场景信息不匹配的照明灯，即可得到待替换对象，实现过程如下：
利用第一神经网络检测待处理图像中的目标对象。
在一些可能的实现方式中，第一神经网络可以是任意类型的神经网络，比如，卷积神经网络，或残差网络等。将待处理图像输入已训练的第一神经网络，第一神经网络输出目标对象的检测框和类别。
在一些实施例中，第一神经网络的训练过程可以通过以下步骤实现：
第一步，将训练图像输入待训练第一神经网络中，预测所述待训练图像中的目标对象的第一位置信息。
在一些可能的实现方式中，通过大量的训练图像对待训练第一神经网络进行训练，即将大量的训练图像输入待训练第一神经网络，以预测待训练图像中的目标对象的位置和类别。
第二步，根据所述训练图像中的目标对象的标注位置信息，确定第一位置信息的第一预测损失。
在一些可能的实现方式中，利用训练图像中目标对象的标注位置信息和目标对象的第一位置信息的差值，确定第一预测损失。
第三步，根据所述第一预测损失，对所述待训练第一神经网络的网络参数进行调整，得到所述第一神经网络。
在一些可能的实现方式中，通过结合目标对象的标注位置信息确定预测的每一个第一位置信息的准确度，将这一准确度反馈给神经网络，以使神经网络调整如权值参数等网络参数，从而提升神经网络检测的准确度。所述第一预测 损失为正样本和负样本的交叉熵损失。采用该预测损失对神经网络的权重等参数进行调整，从而使得调整后的神经网络预测结果更加准确。
上述过程为对第一神经网络进行训练的过程，基于目标对象的预测位置和目标对象的标注位置，进行多次迭代，以使训练后的第一神经网络输出的第一位置信息的第一预测损失满足收敛条件，从而使得该第一神经网络检测的目标对象的准确度更高。
基于对第一神经网络的训练过程，在目标对象中确定待替换对象的过程如下：
首先，通过所述第一神经网络确定所述目标对象的类别信息。
在一些可能的实现方式中，通过第一神经网络，预测出目标对象的类别信息之后，将该类别信息输入到第二神经网络中。
然后，通过第二神经网络，根据所述类别信息，确定所述目标对象中的待替换对象。
在一些可能的实现方式中，第二神经网络可以是用于预测目标对象中的待替换对象的已训练网络，该网络可以是任意类型的神经网络。将该目标对象的类别信息输入到第二神经网络中，即可预测出该目标对象中与特定场景的场景信息具有关联关系的待替换对象。
在一些实施例中，第二神经网络的训练过程可以通过以下步骤实现：
第一步，对所述训练图像中的目标对象所属的类别信息进行标注，得到已标注训练图像。
第二步，将所述已标注图像输入待训练第二神经网络，根据标注的类别信息，预测所述目标对象的待替换对象的第二位置信息。
在一些可能的实现方式中，待训练第二神经网络用于预测目标对象中的待替换对象的位置。将已标注图像输入到待训练第二神经网络，从而预测出待替换对象的第二位置信息，即预测目标对象中的待替换对象的位置。
第三步，根据所述目标对象的待替换对象的标注位置信息，确定所述第二位置信息的第二预测损失。
第二预测损失可以是与第一预测损失类型相同的损失函数，比如，交叉熵损失函数。
第四步，根据所述第二预测损失，对所述待训练第二神经网络的网络参数进行调整，得到所述第二神经网络。
在一些可能的实现方式中，待训练第二神经网络的网络参数包括神经网络中神经元权重等。采用该第二预测损失对待训练第二神经网络的权重等参数进行调整，从而使得调整后的第二神经网络的检测结果更加准确。
上述过程为对第二神经网络进行训练的过程，基于目标对象的类别信息，进行多次迭代，以使训练后的第二神经网络输出的预测的待替换对象的位置信息的第二预测损失满足收敛条件，从而使得该第二神经网络输出的待替换对象准确度更高。
在一些实施例中，采用所述替换图像替换所述待处理图像中所述待替换对象所占据的区域的图像，生成目标图像，可以通过以下过程实现：
首先，确定所述替换图像的尺寸信息。
其次，确定所述待替换对象在待处理图像中所占据的区域的面积。
在一些可能的实现方式中，利用第一神经网络输出待替换对象的检测框，可以将检测框的面积作为待替换对象在待处理图像中所占据的区域的面积。
再次，根据所述面积，对所述替换图像的尺寸信息进行调整，得到已调整图像。
在一些可能的实现方式中，按照待替换对象在待处理图像中所占据的区域的面积，对替换图像的尺寸信息进行调整，得到已调整图像，从而使得已调整图像的尺寸信息与该区域的大小相契合。
再次，采用所述已调整图像替换所述待替换对象所占据的区域的图像，生成候选图像。
在一些可能的实现方式中，在待处理图像中，采用已调整图像替换待替换对象所占据的区域的图像，从而得到替换后的图像，即候选图像。在一个具体例子中，如果特定场景的场景信息为夜晚场景，待处理图像为具有夜晚场景的道路图像，待替换对象为工作状态与夜晚场景不匹配的车灯，即处于未发光状态的车灯，说明该车灯在图像中的呈现方式并不合理，那么目标预设图像(即从预设图像库中查找到的替换图像)为包括处于发光状态的车灯的图像，通过对目标预设图像进行尺寸调整，即可得到已调整图像，从而采用已调整图像替换待替换对象所占据的区域的图像，生成包括处于发光状态的车灯的目标图像。
最后，对所述候选图像进行平滑处理，生成所述目标图像。
在一些可能的实现方式中，可以是通过对候选图像中发生替换操作的区域进行平滑处理，以消除图像在该区域的 噪声，还可以是对整个候选图像进行平滑处理，对整个图像进行降噪，从而得到目标图像；使得生成的目标图像更加的合理且清晰。
下面，将说明本公开实施例在一个实际的应用场景中的示例性应用，以针对白天场景下采集的道路图像转换为夜晚场景为例，即，以特定场景为夜晚场景，待处理图像为夜晚待处理图像，目标对象为车辆，待替换对象为车灯为例，进行说明。
本公开实施例提供一种基于图像生成、目标检测和尾灯匹配的夜晚场景车尾灯添加的方法，使得生成的夜晚图像中车辆更加逼真。本公开实施例可以应用于更多的图像生成领域。例如，夜晚场景车辆前照灯添加、路灯添加等，使得生成图像中的车尾灯生成更加真实。
图3A为本公开实施例提供的图像生成系统(用于生成待处理图像)的组成结构示意图，结合图3A进行以下说明：
本公开实施例提供的图像生成系统包括：生成器301和判别器302。其中，首先，将白天待处理图像(对应于上述实施例中的原始图像，如图3B中的白天待处理图像321)作为输入，从输入端303输入到生成器301；
其次，通过生成器301生成夜晚场景图像(对应于上述实施例中的待处理图像，如图3B中的夜晚场景图像322)，并将生成的夜晚场景图像通过输出端304输出到判别器302；
在一些可能的实现方式中，将在夜晚场景下采集到的夜晚场景图像和生成的夜晚场景图像均输入到判别器302中。
再次，通过判别器302来区分夜晚场景的图像是来自真实的夜晚场景图像还是生成的夜晚场景图像，即分别得到真实图像305和转换图像306；
最后，通过不断优化生成器和判别器的损失函数，使得生成器生成的夜晚场景更加真实。
在一些实施例中，通过图3B所示的图像生成系统生成夜晚场景数据之后，对图像中的车辆进行细粒度的目标检测标注，即，用矩形框将图像中的车辆框出，且标注了每辆车的类别信息(比如，小轿车，越野轿车，面包车，小卡车，大卡车，大巴车，公交车，箱式货车等)。此外，针对每一辆车，标注每辆车的尾灯(对应于上述实施例中的待替换对象)，即采用矩形框对车尾灯位置进行标记。同时，根据每个车型的不同，对真实夜晚场景中的车灯进行收集，即抠出夜晚场景下，采集的车辆图像中亮灯的区域，组成包括车灯数据的图像库(对应于上述实施例中的预设图像库)，用于后续对待替换对象进行匹配，以得到与待替换对象匹配的车尾灯(对应于上述实施例中与待替换对象相匹配的预设对象)。对车尾灯进行匹配的过程，如图4所示，图4为本公开实施例提供的图像生成方法的实现框架结构图，结合图4进行以下说明：
图像获取模块401，配置为通过对原始图像进行目标检测，采用标注框对目标进行标注。
车辆检测网络训练模块402，配置为利用该标注框和对车辆的分类，训练对应的检测网络，得到车辆检测网络(对应于上述实施例中的第一神经网络)。
车辆检测结果模块403，配置为利用车辆检测网络对原始图像进行检测，获得车辆检测结果(比如，获得车辆在图像坐标系下的左上角和右下角坐标(即，标注车辆所在区域的矩形框)，并获得检测的车辆所属的类别，即车辆类型404，进而为图像生成时车灯匹配做准备。
裁剪模块409，配置为基于车辆检测结果中对车辆进行检测的矩形框，从原始图像中裁剪出矩形框对应的位置，并确定矩形框中车辆的车灯信息。
车灯检测网络训练模块405，配置为根据矩形框中车辆的车灯信息训练车灯检测网络(对应于上述实施例中的第二神经网络)。
车灯检测结果输出模块406，配置为通过车灯检测网络对原始图像中的车灯进行检测，得到车灯检测结果。
在一些实施例中，该车灯检测结果中包括该车辆对应车灯的位置以及车灯的大小，在本公开实施例中，车灯检测网络也并不限定为具体网络。
车灯匹配替换模块408，配置为在检测到原始图像中车辆对应车灯位置及大小，以及该车辆所对应的细分类类别之后，利用在车灯库407中查找到的与原始图像的车尾灯匹配的样本尾灯图对原始图像中的车尾灯进行车灯匹配替换。
在一些实施例中，通过图像匹配的方法到对应的车灯库407中查找对应类别车辆夜晚场景下尾灯的样例，并将尾灯通过伸缩尺寸，得到包括与该车辆尾灯一样大小的样本尾灯的样本尾灯图(即与该车辆尾灯相匹配的目标预设图像)。
最终结果输出模块410，配置为利用该样本尾灯图替换原始图像中的车尾灯所在的区域，并使用平滑技术对替换车尾灯区域的周围图像进行平滑处理，从而得到目标图像，即最终的夜晚带发光车尾灯的图像结果。
在一些实施例中，利用车灯库中的尾灯图替换原始图像中的车尾灯之后的图像如图5所示，图5为本公开实施例图 像生成方法的另一应用场景示意图，其中，原始图像501(对应于上述实施例中的原始图像)为白天场景下采集的图像，目标图像的生成过程如下：
首先，通过对原始图像501进行目标检测，利用矩形框标出车辆502、503和504。
然后，对矩形框标出的区域(即车辆502、503和504)进行扣取并放大，并转换为夜晚场景图像，依次得到夜晚场景图像511、512和513(对应于上述实施例中的待处理图像)。
最后，从预设图像库中确定包括与夜晚场景图像511、512和513中的车尾灯相匹配的样本尾灯的样本尾灯图，将样本尾灯图替换夜晚场景图像511、512和513中的车尾灯所在的区域，依次得到加入发光尾灯后的夜晚场景图像521、522和523，即目标图像。
在本公开实施例中，首先，通过在车辆检测的同时对车型进行细分类，然后，通过车辆检测结果进行车尾灯检测，以获得车尾灯位置和大小，最后，根据车型分类结果，车尾灯位置以及大小，在尾灯库中匹配出样本尾灯，以获得最终的带车尾灯的图像；如此，不仅能够使得车尾灯的添加更加自然，而且将白天场景的图像转换为夜晚场景图像的车辆尾灯进行点亮，更加符合夜晚场景的真实性。
本公开实施例提供一种图像生成装置，图6为本公开实施例图像生成装置结构组成示意图，如图6所示，所述装置600包括：
照明灯检测模块601，配置为检测原始图像中的目标对象上的照明灯；
图像转换模块602，配置为将所述原始图像转换为特定场景下的待处理图像；
对象确定模块603，配置为在所述待处理图像中，确定工作状态与所述特定场景的场景信息不匹配的照明灯为待替换对象；
替换图像确定模块604，配置为根据所述场景信息以及所述待替换对象，确定包括工作状态与所述场景信息相匹配的待替换对象的替换图像；
目标图像生成模块605，配置为采用所述替换图像替换所述待处理图像中所述待替换对象所占据的区域的图像，生成目标图像。
在上述装置中，所述替换图像确定模块604，包括：
属性确定子模块，配置为确定所述待替换对象的属性信息；
目标预设图像查找子模块，配置为在预设图像库中，查找包括属性与所述待替换对象的属性信息相匹配的预设对象的目标预设图像；
替换图像确定子模块，配置为将查找到的目标预设图像，确定为所述替换图像；其中，所述预设对象在所述替换图像中的工作状态与所述场景信息相匹配。
在上述装置中，所述装置还包括：
距离确定还模块，配置为确定所述待替换对象与采集所述原始图像的装置之间的距离；
所述目标预设图像查找子模块，还配置为：
根据所述待替换对象的类别信息以及确定的距离，在所述预设图像库中，查找包括属性与所述待替换对象的属性信息相匹配的预设对象的目标预设图像。
在上述装置中，响应于在所述预设图像库中查找不到所述目标预设图像，所述装置还包括：
目标预设图像生成模块，配置为生成包括属性与所述待替换对象的属性信息相匹配的预设对象的目标预设图像；并将生成的目标预设图像，确定为所述替换图像。
在上述装置中，所述目标预设图像生成模块，包括：
工作参数确定子模块，配置为根据所述待替换对象与采集所述原始图像的装置之间的距离，确定所述预设对象的工作参数；
目标预设图像生成子模块，配置为根据确定的工作参数，生成包括所述预设对象的目标预设图像。
在上述装置中，所述距离确定模块，包括：
尺寸信息确定子第一模块，配置为根据所述待替换对象的类别信息，确定所述待替换对象的尺寸信息；
距离确定子模块，配置为根据所述尺寸信息以及所述待处理图像中所述待替换对象的尺寸，确定采集所述原始图像的装置与所述待替换对象之间的距离。
在上述装置中，在生成所述目标预设图像之后，所述装置还包括：
图像库更新模块，配置为将所述目标预设图像存储在所述预设图像库中。
在上述装置中，所述预设对象的工作参数为照明灯的光照强度；
所述工作参数确定子模块，包括：光照强度确定第一单元，配置为确定所述照明灯处于启动状态时的光照强度；光照强度确定第二单元，配置为根据所述距离以及所述照明灯处于启动状态时的光照强度，确定所述照明灯的与所述距离相匹配的光照强度；
所述目标预设图像生成子模块，还配置为：根据所述与距离相匹配的光照强度，生成包括具有与距离相匹配的光照强度的所述预设对象的目标预设图像。
在上述装置中，所述目标图像生成模块605，包括：
尺寸信息确定子第二模块，配置为确定所述替换图像的尺寸信息；
面积确定子模块，配置为确定所述待替换对象在所述待处理图像中所占据的区域的面积；
替换图像调整子模块，配置为根据所述面积，对所述替换图像的尺寸信息进行调整，得到已调整图像；
候选图像生成子模块，配置为采用所述已调整图像替换所述待替换对象所占据的区域的图像，生成候选图像；
目标图像生成子模块，配置为对所述候选图像进行平滑处理，生成所述目标图像。
在上述装置中，所述目标对象包括行驶设备、路灯。
需要说明的是，以上装置实施例的描述，与上述方法实施例的描述是类似的，具有同方法实施例相似的有益效果。对于本公开装置实施例中未披露的技术细节，请参照本公开方法实施例的描述而理解。
需要说明的是，本公开实施例中，如果以软件功能模块的形式实现上述的图像生成方法，并作为独立的产品销售或使用时，也可以存储在一个计算机可读取存储介质中。基于这样的理解，本公开实施例的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是终端、服务器等)执行本公开各个实施例所述方法的全部或部分。而前述的存储介质包括：U盘、运动硬盘、只读存储器(Read Only Memory，ROM)、磁碟或者光盘等各种可以存储程序代码的介质。这样，本公开实施例不限制于任何特定的硬件和软件结合。
对应地，本公开实施例再提供一种计算机程序产品，所述计算机程序产品包括计算机可执行指令，该计算机可执行指令被执行后，能够实现本公开实施例提供的图像生成方法中的步骤。相应的，本公开实施例再提供一种计算机存储介质，所述计算机存储介质上存储有计算机可执行指令，所述该计算机可执行指令被处理器执行时实现上述实施例提供的图像生成方法的步骤。相应的，本公开实施例提供一种计算机设备，图7为本公开实施例计算机设备的组成结构示意图，如图7所示，所述计算机设备700包括：一个处理器701、至少一个通信总线、通信接口702、至少一个外部通信接口和存储器703。其中，通信接口702配置为实现这些组件之间的连接通信。其中，通信接口702可以包括显示屏，外部通信接口可以包括标准的有线接口和无线接口。其中所述处理器701，配置为执行存储器中图像处理程序，以实现上述实施例提供的图像生成方法。
以上图像生成装置、计算机设备和存储介质实施例的描述，与上述方法实施例的描述是类似的，具有同相应方法实施例相似的技术描述和有益效果，限于篇幅，可案件上述方法实施例的记载，故在此不再赘述。对于本公开图像生成装置、计算机设备和存储介质实施例中未披露的技术细节，请参照本公开方法实施例的描述而理解。应理解，说明书通篇中提到的“一个实施例”或“一实施例”意味着与实施例有关的特定特征、结构或特性包括在本公开的至少一个实施例中。因此，在整个说明书各处出现的“在一个实施例中”或“在一实施例中”未必一定指相同的实施例。此外，这些特定的特征、结构或特性可以任意适合的方式结合在一个或多个实施例中。应理解，在本公开的各种实施例中，上述各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本公开实施例的实施过程构成任何限定。上述本公开实施例序号仅仅为了描述，不代表实施例的优劣。
需要说明的是，在本文中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者装置不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者装置所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的过程、方法、物品或者装置中还存在另外的相同要素。
在本公开所提供的几个实施例中，应该理解到，所揭露的设备和方法，可以通过其它的方式实现。以上所描述的设备实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，如：多个单元或组件可以结合，或可以集成到另一个系统，或一些特征可以忽略，或不执行。另外，所显示或讨论的 各组成部分相互之间的耦合、或直接耦合、或通信连接可以是通过一些接口，设备或单元的间接耦合或通信连接，可以是电性的、机械的或其它形式的。
上述作为分离部件说明的单元可以是、或也可以不是物理上分开的，作为单元显示的部件可以是、或也可以不是物理单元；既可以位于一个地方，也可以分布到多个网络单元上；可以根据实际的需要选择其中的部分或全部单元来实现本实施例方案的目的。另外，在本公开各实施例中的各功能单元可以全部集成在一个处理单元中，也可以是各单元分别单独作为一个单元，也可以两个或两个以上单元集成在一个单元中；上述集成的单元既可以采用硬件的形式实现，也可以采用硬件加软件功能单元的形式实现。本领域普通技术人员可以理解：实现上述方法实施例的全部或部分步骤可以通过程序指令相关的硬件来完成，前述的程序可以存储于计算机可读取存储介质中，该程序在执行时，执行包括上述方法实施例的步骤；而前述的存储介质包括：移动存储设备、只读存储器(Read Only Memory，ROM)、磁碟或者光盘等各种可以存储程序代码的介质。
或者，本公开上述集成的单元如果以软件功能模块的形式实现并作为独立的产品销售或使用时，也可以存储在一个计算机可读取存储介质中。基于这样的理解，本公开实施例的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机、服务器、或者网络设备等)执行本公开各个实施例所述方法的全部或部分。而前述的存储介质包括：移动存储设备、ROM、磁碟或者光盘等各种可以存储程序代码的介质。以上所述，仅为本公开的具体实施方式，但本公开的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本公开揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本公开的保护范围之内。因此，本公开的保护范围应以所述权利要求的保护范围为准。
工业实用性
本公开实施例提供一种图像生成方法、装置、设备及存储介质，其中，检测原始图像中的目标对象上的照明灯；将所述原始图像转换为特定场景下的待处理图像；在所述待处理图像中，确定工作状态与所述特定场景的场景信息不匹配的照明灯为待替换对象；根据所述场景信息以及所述待替换对象，确定包括工作状态与所述场景信息相匹配的待替换对象的替换图像；采用所述替换图像替换所述待处理图像中所述待替换对象所占据的区域的图像，生成目标图像。

Claims (13)

  1. 一种图像生成方法，所述方法由电子设备执行，所述方法包括：
    检测原始图像中的目标对象上的照明灯；
    将所述原始图像转换为特定场景下的待处理图像；
    在所述待处理图像中，确定工作状态与所述特定场景的场景信息不匹配的照明灯为待替换对象；
    根据所述场景信息以及所述待替换对象，确定包括工作状态与所述场景信息相匹配的待替换对象的替换图像；
    采用所述替换图像替换所述待处理图像中所述待替换对象所占据的区域的图像，生成目标图像。
  2. 根据权利要求1所述的方法，其中，所述根据所述场景信息以及所述待替换对象，确定包括工作状态与所述场景信息相匹配的待替换对象的替换图像，包括：
    确定所述待替换对象的属性信息；
    在预设图像库中，查找包括属性与所述待替换对象的属性信息相匹配的预设对象的目标预设图像；
    将查找到的目标预设图像，确定为所述替换图像；其中，所述预设对象在所述替换图像中的工作状态与所述场景信息相匹配。
  3. 根据权利要求2所述的方法，其中，所述方法还包括：
    确定所述待替换对象与采集所述原始图像的装置之间的距离；
    所述在预设图像库中，查找包括属性与所述待替换对象的属性信息相匹配的预设对象的目标预设图像，包括：
    根据所述待替换对象的类别信息以及确定的距离，在所述预设图像库中，查找包括属性与所述待替换对象的属性信息相匹配的预设对象的目标预设图像。
  4. 根据权利要求2或3所述的方法，其中，响应于在所述预设图像库中查找不到所述目标预设图像，所述方法还包括：
    生成包括属性与所述待替换对象的属性信息相匹配的预设对象的目标预设图像；
    将生成的目标预设图像，确定为所述替换图像。
  5. 根据权利要求4所述的方法，其中，所述生成包括属性与所述待替换对象的属性信息相匹配的预设对象的目标预设图像，包括：
    根据所述待替换对象与采集所述原始图像的装置之间的距离，确定所述预设对象的工作参数；
    根据确定的工作参数，生成包括所述预设对象的目标预设图像。
  6. 根据权利要求3至5任一项所述的方法，其中，所述确定所述待替换对象与采集所述原始图像的装置之间的距离，包括：
    根据所述待替换对象的类别信息，确定所述待替换对象的尺寸信息；
    根据所述尺寸信息以及所述待处理图像中所述待替换对象的尺寸，确定采集所述原始图像的装置与所述待替换对象之间的距离。
  7. 根据权利要求4或5所述的方法，其中，在生成所述目标预设图像之后，所述方法还包括：
    将所述目标预设图像存储在所述预设图像库中。
  8. 根据权利要求5所述的方法，其中，所述预设对象的工作参数为照明灯的光照强度；
    所述根据所述待替换对象与采集所述原始图像的装置之间的距离，确定所述预设对象的工作参数，包括：确定所述照明灯处于启动状态时的光照强度；根据所述距离以及所述照明灯处于启动状态时的光照强度，确定所述照明灯的与所述距离相匹配的光照强度；
    所述根据确定的工作参数，生成包括所述预设对象的目标预设图像，包括：根据所述与距离相匹配的光照强度，生成包括具有与距离相匹配的光照强度的所述预设对象的目标预设图像。
  9. 根据权利要求1至8任一项所述的方法，其中，所述采用所述替换图像替换所述待处理图像中所述待替换对象所占据的区域的图像，生成目标图像，包括：
    确定所述替换图像的尺寸信息；
    确定所述待替换对象在所述待处理图像中所占据的区域的面积；
    根据所述面积，对所述替换图像的尺寸信息进行调整，得到已调整图像；
    采用所述已调整图像替换所述待替换对象所占据的区域的图像，生成候选图像；
    对所述候选图像进行平滑处理，生成所述目标图像。
  10. 根据权利要求1所述的方法，其中，所述目标对象包括行驶设备、路灯。
  11. 一种图像生成装置，其中，所述装置包括：
    照明灯检测模块，配置为检测原始图像中的目标对象上的照明灯；
    图像转换模块，配置为将所述原始图像转换为特定场景下的待处理图像；
    对象确定模块，配置为在所述待处理图像中，确定工作状态与所述特定场景的场景信息不匹配的照明灯为待替换对象；
    替换图像确定模块，配置为根据所述场景信息以及所述待替换对象，确定包括工作状态与所述场景信息相匹配的待替换对象的替换图像；
    图像生成模块，配置为采用所述替换图像替换所述待处理图像中所述待替换对象所占据的区域的图像，生成目标图像。
  12. 一种计算机存储介质，其中，所述计算机存储介质上存储有计算机可执行指令，该计算机可执行指令被执行后，能够实现权利要求1至10任一项所述的图像生成方法。
  13. 一种电子设备，其中，所述电子设备包括存储器和处理器，所述存储器上存储有计算机可执行指令，所述处理器运行所述存储器上的计算机可执行指令时可实现权利要求1至10任一项所述的图像生成方法。
PCT/CN2022/074085 2021-03-31 2022-01-26 一种图像生成方法、装置、设备及存储介质 WO2022206158A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2023559731A JP2024511635A (ja) 2021-03-31 2022-01-26 画像生成方法、装置、機器及び記憶媒体
US18/472,702 US20240013348A1 (en) 2021-03-31 2023-09-22 Image generation method and apparatus, device, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110351897.2 2021-03-31
CN202110351897.2A CN113096000A (zh) 2021-03-31 2021-03-31 一种图像生成方法、装置、设备及存储介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/472,702 Continuation US20240013348A1 (en) 2021-03-31 2023-09-22 Image generation method and apparatus, device, and storage medium

Publications (1)

Publication Number Publication Date
WO2022206158A1 true WO2022206158A1 (zh) 2022-10-06

Family

ID=76672449

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/074085 WO2022206158A1 (zh) 2021-03-31 2022-01-26 一种图像生成方法、装置、设备及存储介质

Country Status (4)

Country Link
US (1) US20240013348A1 (zh)
JP (1) JP2024511635A (zh)
CN (1) CN113096000A (zh)
WO (1) WO2022206158A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113096000A (zh) * 2021-03-31 2021-07-09 商汤集团有限公司 一种图像生成方法、装置、设备及存储介质
CN113709370B (zh) * 2021-08-26 2023-05-09 维沃移动通信有限公司 图像生成方法、装置、电子设备及可读存储介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102360490A (zh) * 2011-09-30 2012-02-22 北京航空航天大学 基于颜色转换和编辑传播的图像季节特征增强方法
US20170244908A1 (en) * 2016-02-22 2017-08-24 GenMe Inc. Video background replacement system
CN108122276A (zh) * 2017-09-30 2018-06-05 苏州美房云客软件科技股份有限公司 存储设备、处理装置和场景模型的昼夜场景转换方法
CN110418095A (zh) * 2019-06-28 2019-11-05 广东虚拟现实科技有限公司 虚拟场景的处理方法、装置、电子设备及存储介质
CN111951157A (zh) * 2020-09-02 2020-11-17 深圳传音控股股份有限公司 图像处理方法、设备及存储介质
CN112348737A (zh) * 2020-10-28 2021-02-09 达闼机器人有限公司 模拟图像生成的方法、电子设备及存储介质
CN113096000A (zh) * 2021-03-31 2021-07-09 商汤集团有限公司 一种图像生成方法、装置、设备及存储介质

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111738968A (zh) * 2020-06-09 2020-10-02 北京三快在线科技有限公司 图像生成模型的训练方法、装置及图像生成方法、装置

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102360490A (zh) * 2011-09-30 2012-02-22 北京航空航天大学 基于颜色转换和编辑传播的图像季节特征增强方法
US20170244908A1 (en) * 2016-02-22 2017-08-24 GenMe Inc. Video background replacement system
CN108122276A (zh) * 2017-09-30 2018-06-05 苏州美房云客软件科技股份有限公司 存储设备、处理装置和场景模型的昼夜场景转换方法
CN110418095A (zh) * 2019-06-28 2019-11-05 广东虚拟现实科技有限公司 虚拟场景的处理方法、装置、电子设备及存储介质
CN111951157A (zh) * 2020-09-02 2020-11-17 深圳传音控股股份有限公司 图像处理方法、设备及存储介质
CN112348737A (zh) * 2020-10-28 2021-02-09 达闼机器人有限公司 模拟图像生成的方法、电子设备及存储介质
CN113096000A (zh) * 2021-03-31 2021-07-09 商汤集团有限公司 一种图像生成方法、装置、设备及存储介质

Also Published As

Publication number Publication date
US20240013348A1 (en) 2024-01-11
CN113096000A (zh) 2021-07-09
JP2024511635A (ja) 2024-03-14

Similar Documents

Publication Publication Date Title
WO2022206158A1 (zh) 一种图像生成方法、装置、设备及存储介质
US10504214B2 (en) System and method for image presentation by a vehicle driver assist module
O'Malley et al. Rear-lamp vehicle detection and tracking in low-exposure color video for night conditions
CN108460395B (zh) 目标检测方法和装置以及模糊处理方法和装置
CN103810469B (zh) 车外环境识别装置
CN108875458B (zh) 车辆远光灯开启的检测方法、装置、电子设备及摄像机
US20060215882A1 (en) Image processing apparatus and method, recording medium, and program
CN101142814A (zh) 图像处理装置及方法、程序、以及记录介质
JP6236039B2 (ja) 車外環境認識装置
JP3621065B2 (ja) 画像検出装置、プログラムおよび記録媒体
US20240013453A1 (en) Image generation method and apparatus, and storage medium
CN103810695B (zh) 一种光源定位方法及装置
US20230342894A1 (en) Converting input image data from a plurality of vehicle cameras of a surround-view system into optimised output image data
CN105740835A (zh) 夜视环境下基于车载相机的前方车辆检测方法
CN111402409A (zh) 一种展馆设计光照条件模型系统
JP5898535B2 (ja) 撮像ユニットの露光制御装置
CN111923824A (zh) 一种车辆远光灯控制方法、装置、车辆及存储介质
CN115022553B (zh) 一种补光灯的动态控制方法和装置
KR101694824B1 (ko) 듀얼 스마트 카메라를 이용한 에너지 절감 전광판 및 에너지 절감 방법
CN114296556A (zh) 一种基于人体姿态的交互显示方法、装置和系统
CN112693392A (zh) 多光源远光灯的发光亮度控制方法、装置及车辆
CN105740834A (zh) 夜视环境下对前方车辆的高精度检测方法
Hartmann et al. Adaptive Light Function Controlled in the Control Loop with Cameras
US20230394844A1 (en) System for avoiding accidents caused by wild animals crossing at dusk and at night
WO2022239206A1 (ja) 画像表示装置および画像表示システム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22778346

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2023559731

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE