WO2023207137A1

WO2023207137A1 - Image processing method and device

Info

Publication number: WO2023207137A1
Application number: PCT/CN2022/139253
Authority: WO
Inventors: 罗达新; 马莎
Original assignee: 华为技术有限公司
Priority date: 2022-04-28
Filing date: 2022-12-15
Publication date: 2023-11-02
Also published as: CN117011153A

Abstract

The present application relates to the field of image processing, and discloses an image processing method and device, which can reduce the amount of calculation in an image processing process. The method comprises: acquiring an image processing model, a first original image and first environment information; using the first environment information as an input parameter of the image processing model to obtain a first image processing parameter; and processing the first original image by means of the first image processing parameter to obtain a first processed image. The first environment information is information of an environment when the first original image is acquired.

Description

Image processing method and device

"This application claims priority to the patent application filed with the State Intellectual Property Office on April 28, 2022, with application number 202210470192.7 and the invention name "Image Processing Method and Device", the entire content of which is incorporated into this application by reference." .

Technical field

The present application relates to the field of image processing, and in particular to image processing methods and devices.

Background technique

In the field of image processing, the image processing device can process the acquired image so that the processed image meets certain requirements. For example, the image processing device can process an image with lower brightness so that the brightness of the processed image is greater than or equal to the threshold 1, or the image processing device can process an image with higher brightness and overexposure so that the processed image The brightness of the image is less than or equal to the threshold 2, so that the image recognition device can recognize the image.

Currently, image processing devices can use the adaptive Gamma correction with weighting distribution (AGCWD) algorithm to process original images to obtain processed images that meet certain requirements. Specifically, it can be shown in Figure 1. In Figure 1, the image processing device can obtain the original image, generate a grayscale histogram based on the original image, redistribute the grayscale histogram, obtain gamma (gamma) parameters, and enhance the original image through the gamma parameters. Get the processed image. The above process is relatively complex and computationally intensive.

Contents of the invention

Embodiments of the present application provide image processing methods and devices, which can reduce the amount of calculation in the image processing process.

In order to achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

In a first aspect, an image processing method is provided. The device for executing the method may be an image processing device; it may also be a module used in the image processing device, such as a chip or a chip system. The following description takes the execution subject as an image processing device as an example. The method includes: acquiring an image processing model, a first original image and first environment information, where the first environment information is information about the environment when acquiring the first original image; and using the first environment information as an input to the image processing model. parameters to obtain the first image processing parameters; use the first image processing parameters to process the first original image to obtain the first processed image.

Based on the method provided in the first aspect, the image processing device can obtain the first image processing parameters based on the environmental information and the image processing model when acquiring the first original image, and process the first original image according to the first image processing parameters, Get the processed image. In the above method, it does not involve the generation and redistribution of grayscale histograms and other computationally intensive processes. Instead, an image processing model trained in advance and related to environmental information is used to obtain the first image processing parameters. According to The first image processing parameter processes the first original image. The calculation process is relatively simple and the calculation amount is small. It can be applied to equipment with low computing power, so that the equipment can also perform image processing, or be applied to real-time processing. Highly demanding scenarios, such as smart driving scenarios, enable smart driving vehicles to identify targets (such as vehicles, pedestrians, lane lines or obstacles, etc.) as quickly as possible based on the first processed image. Moreover, the above method obtains the first image processing parameters based on the environmental information and the image processing model when acquiring the first original image. That is to say, the first image processing parameters are related to the environmental information and have nothing to do with the original image. Therefore, , there will be no problem that the processed image is too bright and overexposed due to dark objects in the original image. In addition, since the first image processing parameters are parameters related to environmental information, if the environmental information corresponding to the two original images is the same (or has little difference), then the first image processing parameters corresponding to the two original images are also the same. (Or not much different). Therefore, if this method is used to process video streams, the problem of video flickering will not occur.

In a possible implementation, the first environment information includes at least one of the following: time information for obtaining the first original image, information on the illumination intensity when the first original image is obtained, and information on the illumination direction when the first original image is obtained. information, brightness information when the first original image is obtained, or position information when the first original image is obtained.

It can be understood that the above information is related to the gamma correction function. Therefore, the first image processing parameters obtained according to the first environment information can be used for gamma correction. In addition, when shooting a video, the above information is also smooth and excessive, that is, the first environment information corresponding to two adjacent frames of images in the video is not much different, so the first environment information corresponding to the two adjacent frames of images obtained based on the first environment information The processing parameters are not much different, so there will be no problem of video flickering.

A possible implementation manner, obtaining an image processing model includes: obtaining at least one second environment information and at least one second original image, the at least one second original image including each second environment in the at least one second environment information A second original image corresponding to the information; acquiring at least one target image, the at least one target image including a target image corresponding to each second environment information in the at least one second environment information, the second environment information is to obtain the second environment When the second original image and the third original image corresponding to the information are environmental information, the third original image is the original image of the target image corresponding to the second environmental information, and the type of information included in the second environmental information is the same as that of the third original image. A piece of environmental information includes the same type of information; the image processing model is obtained according to the at least one second environmental information, the at least one second original image and the at least one target image.

Based on the above possible implementation manner, the image processing device can obtain the image processing model.

A possible implementation manner, obtaining the image processing model according to the at least one second environment information, the at least one second original image and the at least one target image includes: using at least one second image processing parameter to respectively A second original image is processed to obtain at least one second processed image, and the absolute value of the difference between the correlation coefficient of the at least one second processed image and the correlation coefficient of the at least one target image is less than or equal to the first threshold; according to the at least one A second environment information and the at least one second image processing parameter are used to obtain the image processing model.

Based on the above possible implementation manner, the image processing device can train at least one second image processing parameter and at least one second environment information to obtain an image processing model. Wherein, at least one second image processing parameter can make at least one second processed image have a high similarity with at least one target image. Therefore, environmental information (such as first environmental information) is input into the image processing model obtained according to the above method. , the image processing parameters (such as the first image processing parameters) can be obtained, and the original image (such as the first original image) corresponding to the environmental information is processed by using the image processing parameters, so that the processed image (such as the first processed image) can be compared with at least A target image has higher similarity. In other words, the processed image can also meet the requirements of the target image.

In one possible implementation, before using at least one second image processing parameter to respectively process the at least one second original image to obtain at least one second processed image, the method further includes: obtaining a set of candidate image processing parameters, the The set of candidate image processing parameters includes a plurality of candidate image processing parameters, the plurality of candidate image processing parameters including the at least one second processing parameter.

Based on the above possible implementation, the image processing device can first obtain multiple candidate parameters, and then use the candidate parameters to process the second original image to obtain a processed image. If the correlation coefficient of the processed image is consistent with the target image If the absolute value of the difference between the correlation coefficients is less than or equal to the first threshold, the image processing device determines the candidate parameter corresponding to the processed image as the second processing parameter.

In a possible implementation, the method further includes: obtaining initial image processing parameters; performing a first operation on the initial image processing parameters to obtain the candidate image processing parameter set.

Based on the above possible implementation manner, the image processing apparatus can obtain the candidate image processing parameter set.

In a possible implementation, the first operation includes at least one of the following: determining candidate image processing parameters according to rules according to the initial image processing parameters, randomly determining candidate image processing parameters based on the initial image processing parameters, or determining candidate image processing parameters based on historically determined parameters. The candidate image processing parameters determine the set of candidate image processing parameters.

Based on the above possible implementation manner, the image processing device can obtain the candidate image processing parameter set through multiple methods, which improves the flexibility and diversity of obtaining the candidate image processing parameter set.

In a possible implementation manner, the image processing model includes functions or algorithms.

It can be understood that the calculation amount of using a function or algorithm to calculate the parameters in the first environment information is much less than the calculation amount of generating and redistributing the grayscale histogram. Therefore, the image processing speed of the image processing device can be greatly improved.

In a second aspect, an image processing device is provided for implementing the above method. The image processing device may be the image processing device in the above-described first aspect, or a device including the above-described image processing device. The image processing device includes corresponding modules, units, or means (means) for implementing the above method. The modules, units, or means can be implemented by hardware, software, or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above functions.

In conjunction with the above second aspect, in a possible implementation, the image processing device may include a processing module. This processing module can be used to implement the processing functions in any of the above aspects and any possible implementation manner thereof. The processing module may be, for example, a processor.

A third aspect provides an image processing device, including: a processor; the processor is configured to be coupled to a memory, and after reading instructions in the memory, execute the method described in the first aspect according to the instructions. The image processing device may be the image processing device in the above-described first aspect, or a device including the above-described image processing device.

In conjunction with the above third aspect, in a possible implementation, the image processing device further includes a memory, and the memory is used to store necessary program instructions and data.

In conjunction with the above third aspect, in a possible implementation manner, the image processing device is a chip or a chip system. Optionally, when the image processing device is a chip system, it may be composed of a chip, or may include a chip and other discrete devices.

In a fourth aspect, an image processing device is provided, including: a processor and an interface circuit; the interface circuit is used to receive a computer program or instructions and transmit them to the processor; the processor is used to execute the computer program or instructions, so that the The image processing device executes the method described in the above first aspect.

In conjunction with the fourth aspect, in a possible implementation, the image processing device is a chip or a chip system. Optionally, when the image processing device is a chip system, it may be composed of a chip, or may include a chip and other discrete devices.

In a fifth aspect, a computer-readable storage medium is provided. Instructions are stored in the computer-readable storage medium, and when run on a computer, the computer can execute the method described in the first aspect.

A sixth aspect provides a computer program product containing instructions that, when run on a computer, enable the computer to execute the method described in the first aspect.

A seventh aspect provides an intelligent driving vehicle, which includes an image processing device for executing the method described in the first aspect.

Among them, the technical effects brought by any possible implementation method in the second to seventh aspects can be referred to the technical effects brought by the above-mentioned first aspect or different possible implementation methods in the first aspect, which will not be discussed here. Repeat.

It is understandable that, on the premise that the solutions are not inconsistent, the solutions in each of the above aspects can be combined.

Description of drawings

Figure 1 is a schematic diagram of the image processing process;

Figure 2 is a schematic diagram before and after image processing;

Figure 3A is a schematic diagram of the image processing system architecture provided by an embodiment of the present application;

Figure 3B is a schematic diagram of an image processing device provided by an embodiment of the present application;

Figure 4 is a schematic diagram of the hardware structure of an image processing device provided by an embodiment of the present application;

Figure 5 is a schematic flowchart 1 of the image processing method provided by the embodiment of the present application;

Figure 6 is a schematic flowchart 2 of the image processing method provided by the embodiment of the present application;

Figure 7 is a schematic flow chart of the feature extraction method provided by the embodiment of the present application;

Figure 8 is a schematic diagram of an image processing model provided by an embodiment of the present application;

FIG. 9 is a schematic structural diagram of an image processing device provided by an embodiment of the present application.

Detailed ways

Before introducing the embodiments of the present application, relevant technical terms involved in the embodiments of the present application will be explained. It can be understood that these explanations are intended to make the embodiments of the present application easier to understand and should not be regarded as limiting the scope of protection required by the embodiments of the present application.

1. Image data type

It can be understood that the image sensing device (such as a camera) can project the ambient light signal into the visual sensor chip, and the visual sensor chip can convert the light signal into an electrical signal and maintain it as a raw data (RAW) image. Subsequently, the image processing device can convert the RAW image into a red green blue (RGB) image. That is, the image data type may include RAW images or RGB images.

In the embodiment of the present application, a RAW image may represent a native, unprocessed image. For example, RAW images include images sensed from image sensing devices such as digital cameras, mobile phones, tablets, or scanners. RAW images correspond to RAW files. RAW files can contain data information needed to create a viewable image, such as file header and pixel area information. The structure of RAW files can follow a common pattern. For example, the structure of a RAW file may include a RAW image format ISO12234-2 or TIFF/EP that complies with the International Organization for Standardization (ISO) standard. Sometimes, a RAW image can refer to an image in bayer format. The number of pixels is the same as the number of pixels in the image perceived by the image sensing device. It is a matrix of width (width) * height (height) * 1, that is, the information of the pixel area includes width A matrix of size *height*1. Among them, width and height indicate the width and height of the image respectively, and 1 indicates that the RAW image has one channel. Each pixel represents one of three colors: red (red, R), green (green, G), or blue (blue, B). A floating point number from 0.0 to 1.0 is usually used to represent the signal strength of each color.

In the embodiment of this application, RGB images can represent various eye-friendly images generated after RAW images are processed by an image signal processor (ISP), for example, common jpeg format images, BMP format images or PNG format images, etc. Compared with RAW files, files corresponding to RGB images do not include file headers, but include pixel area information. For example, the file corresponding to an RGB image includes a matrix of width*height*3. 3 means that the RGB image has 3 channels, each channel represents one of the three colors of R/G/B, and an integer from 0 to 255 is used to represent each pixel value of the image.

2.ISP image processing process

In this embodiment of the present application, the image processing device may include an ISP for processing images. The ISPs used by different image processing devices may be the same or different. Among them, the ISP can have at least one of the following functions: demosaicing, black level correction, lens correction, bad pixel correction, gamma correction, denoising, white balance or color mapping. For any of the above functions, the parameters used by the ISPs of different image processing devices may be the same or different. For example, for image 1, the ISP of image processing device 1 processes image 1 through gamma parameter 1 to obtain image 2, and the ISP of image processing device 2 processes image 1 through gamma parameter 2 to obtain image 3. Or, for image 1, both the ISP of image processing device 1 and the ISP of image processing device 2 process image 1 through gamma parameter 1 to obtain image 2.

It can be understood that the image processing method provided by the embodiment of the present application can be applied to one or more of the above functions. For example, parameters in one or more of the above functions can be obtained according to environmental information, and the image can be processed according to the obtained parameters. Process to get the processed image. The following embodiments of the present application are described by taking the image processing method applied to gamma correction (for example, obtaining gamma parameters according to environmental information, processing the image according to the gamma parameters, and obtaining a processed image) as an example. The embodiments of the present application provide The application of the image processing method to other functions is similar to the application to gamma correction. Please refer to the following embodiments of this application and will not be described again.

Please refer to Figure 2. Figure 2 shows images taken under different lighting conditions from sunrise to sunset. In Figure 2, the original images taken at 5:00am (5:00am) and 19:00pm (19:00pm) have lower brightness and the images are too dark, while the original images taken at 13:00pm (13:00pm) The brightness of the image is high, and the image is too bright and overexposed, which makes it difficult for the image recognition device to identify the content in the image, such as objects or people in the image. The original image in Figure 2 can be used to adjust the brightness of the image through the method shown in Figure 1 to obtain the processed image. Obviously, the brightness of the processed image is moderate, which can improve the recognition rate of the image recognition device and the accuracy of identifying objects. However, the method shown in Figure 1 is more complex and requires a large amount of calculation. Usually, when digital signal processing (DSP) chips use this algorithm, it takes tens of milliseconds. It is not suitable for some scenarios that require high real-time performance, such as intelligent driving scenarios. Moreover, in the method shown in Figure 1, if there are dark objects in the original image, the processed image will be brighter, and the image may be too bright and overexposed. In addition, in the method shown in Figure 1, the gamma parameters obtained according to different original images are different. Therefore, if the method shown in Figure 1 is used to process the video stream, the gamma parameters of two adjacent frames of images will be different. Since the gamma parameter is related to image brightness, the problem of video flickering may occur.

In order to solve the above problems, embodiments of the present application provide an image processing method. The image processing method can obtain the first image processing parameters based on the environmental information and the image processing model when acquiring the first original image. According to the first image processing The parameters are used to process the first original image to obtain the processed image. In the above method, it does not involve the generation and redistribution of grayscale histograms and other computationally intensive processes. Instead, an image processing model trained in advance and related to environmental information is used to obtain the first image processing parameters. According to The first image processing parameter processes the first original image, the calculation process is relatively simple, and the calculation amount is small. Moreover, the above method obtains the first image processing parameters based on the environmental information and the image processing model when acquiring the first original image. That is to say, the first image processing parameters are related to the environmental information and have nothing to do with the original image. Therefore, , there will be no problem that the processed image is too bright and overexposed due to dark objects in the original image. In addition, since the first image processing parameters are parameters related to environmental information, if the environmental information corresponding to the two original images is the same (or has little difference), then the first image processing parameters corresponding to the two original images are also the same. (Or not much different). Therefore, if this method is used to process video streams, the problem of video flickering will not occur. The specific process of this method will be introduced in the embodiment shown in Figure 5 below, and will not be described again here.

The implementation of the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

The method provided by the embodiments of the present application can be used in various image processing systems to process the original image and obtain a processed image, so that the processed image meets certain requirements. The following uses the image processing system 30 shown in FIG. 3A as an example to describe the method provided by the embodiment of the present application.

As shown in FIG. 3A , it is a schematic architectural diagram of an image processing system 30 provided by an embodiment of the present application. In FIG. 3A , the image processing system 30 may include one or more image processing devices 301 (only one is shown). Optionally, the image processing system 30 also includes an image sensing device 302 and/or an image recognition device 303 that can communicate with the image processing device 301 . FIG. 3A is only a schematic diagram and does not constitute a limitation on the applicable scenarios of the technical solution provided by this application.

The image processing device in the embodiment of the present application, for example, the image processing device 301 can be any device with computing functions. Optionally, the image processing device also has image perception capabilities and/or image recognition capabilities, for example, it can perceive the original image (or shoot the original image) and/or be able to identify the content in the image. As an example, the image processing device 301 can obtain the image processing model, the first original image and the first environment information, use the first environment information as an input parameter of the image processing model, obtain the first image processing parameters, and use the first image processing The parameters are used to process the first original image to obtain the first processed image. The first environment information is information about the environment when the first original image is acquired.

In the embodiment of this application, the image processing device includes but is not limited to: handheld device, vehicle-mounted device, computing device or intelligent driving vehicle. For example, the image processing device may be a mobile phone, a tablet, a computer, various devices with computing capabilities in a car, or an intelligent driving vehicle with an automatic driving function or an assisted driving function. Various devices with computing capabilities in the car can include: gateway, vehicle T-Box (telematics box), body control module (BCM), smart cockpit domain controller (cockpit domain controller, CDC), multi-domain Controller (multi domain controller, MDC), vehicle control unit (vehicle control unit, VCU), electronic control unit (electronic control unit, ECU), vehicle domain controller (vehicle domain controller, VDC) or vehicle integrated unit (vehicle integrated/integration unit, VIU) etc. Automatic driving means that the automatic driving device in the vehicle can operate the vehicle to drive safely without the participation of the driver during the driving process. Assisted driving refers to the auxiliary driving device in the vehicle that assists the driver in safe driving while the vehicle is driving. Autonomous driving or assisted driving can also be called intelligent driving.

It can be understood that the image processing device can also be a virtual reality (VR) device, an augmented reality (AR) device, a wearable device, a wireless terminal in industrial control, a wireless terminal in driverless driving, or a remote control device. Wireless terminals in medical treatment, wireless terminals in smart grids, wireless terminals in smart cities, or wireless terminals in smart homes, etc.

In this application, the image processing device can also be a terminal in the Internet of Things (IoT) system. IoT is an important part of the future development of information technology. Its main technical feature is to connect items to the network through communication technology. This enables an intelligent network of human-computer interconnection and physical-object interconnection. The image processing device of the present application may be a vehicle-mounted module, vehicle-mounted module, vehicle-mounted component, vehicle-mounted chip or vehicle-mounted unit built into the vehicle as one or more components or units. The vehicle uses the built-in vehicle-mounted module, vehicle-mounted module, Vehicle-mounted components, vehicle-mounted chips or vehicle-mounted units can implement the method of the present application. Therefore, the embodiments of the present application can be applied to the Internet of Vehicles, such as vehicle outreach (vehicle to everything, V2X), inter-vehicle communication long term evolution technology (long term evolution vehicle, LTE-V), vehicle to vehicle (vehicle to vehicle, V2V) wait.

The image sensing device in the embodiment of the present application, for example, the image sensing device 302, can be any device with image sensing capabilities. For example, the image sensing device may include one or more of a monocular camera, a binocular camera, a trinocular camera, a depth camera, or a scanner.

The image recognition device in the embodiment of the present application, for example, the image recognition device 303, can be any device with image recognition capabilities and can recognize the content in the image.

The image processing system 30 shown in FIG. 3A is only used as an example and is not used to limit the technical solution of the present application. Those skilled in the art should understand that during specific implementation, the image processing system 30 may also include other equipment, and the number of image processing devices, image sensing devices or image recognition devices may also be determined according to specific needs without limitation.

It can be understood that the functions of the image processing apparatus in the embodiments of the present application can be implemented by one device or module, or by multiple devices or modules. As an example, if the functions of the image processing device are implemented by multiple devices or modules, the image processing device may be as shown in Figure 3B.

In Figure 3B, the image processing device 301 may include multiple modules, namely a model acquisition module 3011, an information acquisition module 3012 and an image processing module 3013. Optionally, the image processing device 301 also includes an image acquisition module 3014 and/or an image recognition module 3015.

As an example, the model acquisition module 3011 can be used to acquire an image processing model and send the image processing model to the image processing module 3013. The information acquisition module 3012 may be used to acquire the first environment information and send the first environment information to the image processing module 3013. The image processing module 3013 can be used to receive the image processing model from the model acquisition module 3011, receive the first environment information from the information acquisition module 3012, acquire the first original image, and use the first environment information as an input parameter of the image processing model to obtain first image processing parameters, and use the first image processing parameters to process the first original image to obtain a first processed image. Optionally, the image processing module 3013 can also send the first processed image to the image recognition module 3015, so that the image recognition module 3015 can recognize the content in the first processed image, etc.

As another example, the model acquisition module 3011 may be used to acquire an image processing model and send the image processing model to the image processing module 3013. The information acquisition module 3012 may be used to acquire the first environment information and send the first environment information to the image processing module 3013. The image acquisition module 3014 may be used to acquire the first original image and send the first original image to the image processing module 3013. The image processing module 3013 may be configured to receive the image processing model from the model acquisition module 3011, receive the first environment information from the information acquisition module 3012, receive the first original image from the image acquisition module 3014, and process the first environment information as an image. The input parameters of the model are used to obtain the first image processing parameters, and the first original image is processed using the first image processing parameters to obtain the first processed image. Optionally, the image processing module 3013 can also send the first processed image to the image recognition module 3015, so that the image recognition module 3015 can recognize the content in the first processed image, etc.

In Figure 3B, "module" can be replaced by "device". For example, "image processing module" can be replaced by "image processing device".

The image processing device 301 shown in FIG. 3B is only used as an example and is not used to limit the technical solution of the present application. Those skilled in the art should understand that during specific implementation, the image processing device 301 may also include other modules or equipment, and may also determine a model acquisition module, an information acquisition module, an image processing module, an image acquisition module, or a module according to specific needs. The number of image recognition modules is not limited.

Optionally, each device or module (such as an image processing device, a model acquisition module, an information acquisition module, or an image processing module, etc.) in Figure 3A or Figure 3B in the embodiment of this application can be a general device or a special device, The embodiments of the present application do not specifically limit this.

Optionally, the relevant functions of each device or module (such as an image processing device, a model acquisition module, an information acquisition module, or an image processing module, etc.) in Figure 3A or Figure 3B in the embodiment of this application can be implemented by one device, or they can It can be jointly implemented by multiple devices, or can also be implemented by one or more functional modules in one device, which is not specifically limited in the embodiments of the present application. It can be understood that the above functions can be either network elements in hardware devices, software functions running on dedicated hardware, or a combination of hardware and software, or virtualization instantiated on a platform (for example, a cloud platform) Function.

During specific implementation, each device or module (such as image processing device, model acquisition module, information acquisition module, or image processing module, etc.) in Figure 3A or Figure 3B in the embodiment of the present application can adopt the composition structure shown in Figure 4 , or include the components shown in Figure 4. FIG. 4 shows a schematic diagram of the hardware structure of an image processing device applicable to embodiments of the present application. The image processing device 40 includes at least one processor 401 and at least one communication interface 404, which are used to implement the method provided by the embodiment of the present application. The image processing device 40 may also include a communication line 402 and a memory 403.

The processor 401 can be a general central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more processors used to control the execution of the program of the present application. integrated circuit.

Communication line 402 may include a path, such as a bus, that carries information between the above-mentioned components.

Communication interface 404 is used to communicate with other devices or communication networks. The communication interface 404 can be any device such as a transceiver, such as an Ethernet interface, a radio access network (RAN) interface, a wireless local area networks (WLAN) interface, a transceiver, and pins , bus, or transceiver circuit, etc.

Memory 403 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (random access memory (RAM)) or other type that can store information and instructions. A dynamic storage device can also be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disk storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), disk storage media or other magnetic storage devices, or can be used to carry or store desired program code in the form of instructions or data structures and can be used by a computer Any other medium for access, but not limited to this. The memory may exist independently and be coupled to the processor 401 through a communication line 402 . Memory 403 may also be integrated with processor 401. The memory provided by the embodiment of the present application may generally be non-volatile.

Among them, the memory 403 is used to store computer execution instructions involved in executing the solutions provided by the embodiments of this application, and the processor 401 controls the execution. The processor 401 is used to execute computer execution instructions stored in the memory 403, thereby implementing the method provided by the embodiment of the present application. Or, optionally, in this embodiment of the present application, the processor 401 may also perform processing-related functions in the methods provided in the following embodiments of the present application, and the communication interface 404 is responsible for communicating with other devices or communication networks. This application implements The example does not specifically limit this.

Optionally, the computer-executed instructions in the embodiments of the present application may also be called application codes, which are not specifically limited in the embodiments of the present application.

The coupling in the embodiment of this application is an indirect coupling or communication connection between devices, units or modules, which may be in electrical, mechanical or other forms, and is used for information interaction between devices, units or modules.

As an embodiment, the processor 401 may include one or more CPUs, such as CPU0 and CPU1 in FIG. 4 .

As an embodiment, the image processing device 40 may include multiple processors, such as the processor 401 and the processor 407 in FIG. 4 . Each of these processors may be a single-CPU processor or a multi-CPU processor. A processor here may refer to one or more devices, circuits, and/or processing cores for processing data (eg, computer program instructions).

As an embodiment, the image processing apparatus 40 may also include an output device 405 and/or an input device 406. Output device 405 is coupled to processor 401 and can display information in a variety of ways. For example, the output device 405 may be a liquid crystal display (LCD), a light emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector (projector), etc. Input device 406 is coupled to processor 401 and can receive user input in a variety of ways. For example, the input device 406 may be a mouse, a keyboard, a touch screen device, a sensing device, or the like.

Optionally, the image processing device 40 also includes an image perception module and/or an image recognition module (not shown in Figure 4). The image sensing module can have image sensing capabilities. For example, the image processing device 40 is configured with one or more of a monocular camera, a binocular camera, a trinocular camera, a depth camera or a scanning module. The image recognition module can have the ability to recognize the content in the image.

It can be understood that the composition structure shown in Figure 4 does not constitute a limitation on the image processing device. In addition to the components shown in Figure 4, the image processing device may include more or fewer components than shown in the figure, or Combining certain parts, or different arrangements of parts.

The method provided by the embodiment of the present application will be described below with reference to the accompanying drawings. Each device in the following embodiments may be equipped with the components shown in Figure 4, which will not be described again.

It can be understood that the methods provided by the embodiments of the present application can be applied to multiple fields, such as: unmanned driving field, automatic driving field, assisted driving field, intelligent driving field, connected driving field, intelligent connected driving field, automobile Shared areas, etc.

It can be understood that the "image" in the embodiments of this application can be replaced by "picture" or "photograph" or other names similar to "image" without limitation.

It can be understood that the information name or the name of each parameter in the information in the following embodiments of the present application is just an example, and other names may also be used in specific implementations, and the embodiments of the present application do not specifically limit this.

It can be understood that in the embodiment of the present application, "/" may indicate that the related objects are in an "or" relationship. For example, A/B may indicate A or B; "and/or" may be used to describe There are three relationships between associated objects. For example, A and/or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. A and B can be singular or plural. In addition, expressions similar to "at least one of A, B and C" or "at least one of A, B or C" are often used to mean any of the following: A alone; B alone; alone C exists; A and B exist simultaneously; A and C exist simultaneously; B and C exist simultaneously; A, B, and C exist simultaneously. The above is an example of three elements A, B and C to illustrate the optional items of this project. When there are more elements in the expression, the meaning of the expression can be obtained according to the aforementioned rules.

In order to facilitate the description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" may be used to distinguish technical features with the same or similar functions. The words "first", "second" and other words do not limit the quantity and execution order, and the words "first" and "second" do not limit the number and execution order. In the embodiments of this application, words such as "exemplary" or "for example" are used to express examples, illustrations or illustrations, and any embodiment or design solution described as "exemplary" or "for example" shall not be interpreted. To be more preferred or advantageous than other embodiments or designs. The use of words such as "exemplary" or "such as" is intended to present related concepts in a concrete manner that is easier to understand.

It will be understood that reference throughout this specification to "an embodiment" means that a particular feature, structure, or characteristic associated with the embodiment is included in at least one embodiment of the present application. Therefore, various embodiments are not necessarily referred to the same embodiment throughout this specification. Furthermore, the particular features, structures or characteristics may be combined in any suitable manner in one or more embodiments. It can be understood that in the various embodiments of the present application, the size of the sequence numbers of each process does not mean the order of execution. The execution order of each process should be determined by its functions and internal logic, and should not be determined by the execution order of the embodiments of the present application. The implementation process constitutes no limitation.

It can be understood that in this application, "if" can mean that corresponding processing will be carried out under certain objective circumstances. It does not limit the time, nor does it require that there must be a judgment action during implementation, nor does it mean that there are other limited.

“At the same time” in this application can be understood as at the same point in time, within a period of time, or within the same cycle.

It can be understood that some optional features in the embodiments of the present application, in certain scenarios, can be implemented independently without relying on other features, such as the solutions they are currently based on, to solve corresponding technical problems and achieve corresponding effects. , and can also be combined with other features according to needs in certain scenarios. Correspondingly, the devices provided in the embodiments of the present application can also implement these features or functions, which will not be described again here. For example, in the following embodiments, S5011-S5013 can be implemented independently without relying on S501-S503.

It can be understood that the same step or steps with the same function or technical features in the embodiments of the present application can be used as reference between different embodiments.

It can be understood that in the embodiments of the present application, the image processing device can perform some or all of the steps in the embodiments of the present application. These steps are only examples. The embodiments of the present application can also perform other steps or variations of various steps. In addition, various steps may be performed in a different order than those presented in the embodiments of the present application, and it may not be necessary to perform all the steps in the embodiments of the present application.

As shown in Figure 5, an image processing method is provided in an embodiment of the present application. The image processing method may include the following steps:

S501: The image processing device obtains the image processing model, the first original image and the first environment information.

The image processing device in S501 may be the image processing device 301 in FIG. 3A or FIG. 3B. The image processing device may acquire the image processing model, the first original image, and the first environment information at the same time, or may not acquire the image processing model, the first original image, and the first environment information at the same time. For example, the image processing device may first acquire the image processing model and the first environment information, and then acquire the first original image, or the image processing device may first acquire the image processing model, then acquire the first environment information, and finally acquire the first original image.

In this embodiment of the present application, the image processing model may include an input terminal and an output terminal. After the input parameters are input into the image processing model through the input terminal, the output terminal can output the output parameters. As an example, an image processing model includes a function or algorithm. For example, the image processing model is y=f(x). Among them, x is the input parameter and y is the output parameter. It can be understood that the image processing model can include any function or algorithm, such as a quadratic function, a cubic function, an exponential function, a custom function or a custom algorithm, etc. It can be understood that the calculation amount of using a function or algorithm to calculate the parameters in the first environment information is much less than the calculation amount of generating and redistributing the grayscale histogram. Therefore, the image processing speed of the image processing device can be greatly improved.

In this embodiment of the present application, the input parameters may include environmental information, such as environmental information when acquiring the original image. For example, the input parameters include first environment information.

In this embodiment of the present application, the original image (such as the first original image, and/or the second original image, and/or the third original image, etc.) may be an image of any type or format. For example, the original image is a RAW image or an RGB image. For another example, the original image is an image in bayer format, an image in jpeg format, an image in BMP format, or an image in PNG format, etc.

In this embodiment of the present application, the output parameters may include image processing parameters (such as first image processing parameters). The image processing parameters may include at least one of the following: parameters in the demosaic function, parameters in the black level correction function, parameters in the lens correction function, parameters in the gamma correction function (such as gamma parameters), parameters in the white balance function Parameters or parameters in a colormap function.

In this embodiment of the present application, the first environment information is information about the environment when the first original image is acquired. It can be understood that the first environment information is related to the function applied by the image processing method. Specifically, if the function is a gamma correction function, the first environment information is related to the gamma correction function. In this way, the first image processing parameters obtained according to the first environment information can be used for gamma correction.

As an example, the first environment information includes at least one of the following: time information when the first original image is obtained, information about the illumination intensity when the first original image is obtained, information about the illumination direction when the first original image is obtained, information about the illumination direction when the first original image is obtained, The brightness information when the image is taken or the position information when the first original image is obtained.

The time information for obtaining the first original image may include at least one of the following: a moment when the first original image is obtained, a month when the first original image is obtained, or a season when the first original image is obtained. The lighting direction can include east, south, west, or north; or the lighting direction can include east, south, west, north, northeast, southeast, northwest, or southwest; or the lighting direction can include angles, such as the lighting direction and each coordinate system. angle of the axis. The coordinate system can be any two-dimensional or three-dimensional coordinate system in which the first original image is located. Location information can include longitude and latitude. It can be understood that the above information is related to the gamma correction function. Therefore, the first image processing parameters obtained according to the first environment information can be used for gamma correction. In addition, when shooting a video, the above information is also smooth and excessive, that is, the first environment information corresponding to two adjacent frames of images in the video is not much different, so the corresponding two adjacent frames of images obtained based on the first environment information in S502 The first processing parameters are also not much different, so there will be no problem of video flickering.

In one possible implementation, the first environment information includes multiple bits, and different bits correspond to different information. Taking the first environment information as an example including the season in which the first original image was acquired and the light intensity when the first original image was acquired, the first environment information may include 6 bits, where the first two bits are related to the season in which the first original image was acquired. Correspondingly, the last 4 bits correspond to the information of the illumination intensity when acquiring the first original image. If the value of the first two bits is "00", it means that the season in which the first original image is obtained is spring. If the value of the first two bits is "01", it means that the season in which the first original image is obtained is summer. If the value of the two bits is "10", it means that the season in which the first original image is obtained is autumn. If the value of the first two bits is "11", it means that the season in which the first original image is obtained is winter. The value of the last 4 bits can represent the illumination intensity when the first original image was acquired.

It can be understood that the image processing device can obtain the image processing model, and/or the first original image, and/or the first environment information in various ways. For example, the image processing device can obtain the above information locally, or the image processing device can obtain the above information through other devices or equipment, or the image processing device can obtain the above information by itself. Detailed introduction is given below.

As an example, the image processing device locally stores the image processing model, and/or the first original image, and/or the first environment information. Therefore, the image processing device can obtain the image processing model, and/or the first original image, and/or the first environment information locally.

As another example, the model acquisition device can acquire the image processing model and send it to the image processing device, and accordingly, the image processing device receives the image processing model. The image sensing device can acquire the first original image and send it to the image processing device, and accordingly, the image processing device receives the first original image. The image sensing device may be image sensing device 302 in Figure 3A. The information acquisition device can acquire the first environment information and send it to the image processing device. Correspondingly, the image processing device receives the first environment information. It can be understood that the model acquisition device, the image sensing device and the information acquisition device are different devices from the image processing device. The model acquisition device, image sensing device and information acquisition device may be the same device or different devices without limitation.

As another example, the image processing device may acquire the image processing model by itself in the following manner: the image processing device acquires at least one second environment information, at least one second original image, and at least one target image, and determines the image processing model according to the at least one second environment information. , at least one second original image and at least one target image, and obtain an image processing model. This process will be introduced in the embodiment shown in Figure 6 below, and will not be described in detail here. It can be understood that the image processing device may have image sensing capabilities. For example, the image processing device is configured with a camera device, and the first original image may be acquired through the camera device. The camera device may include one or more of the following: a monocular camera, a binocular camera, a trinocular camera, a depth camera or a scanner. It can be understood that the image processing device can obtain the first environment information through one or more of the network, a sensor in the image processing device, or software installed in the image processing device. For example, the image processing device determines the time information for acquiring the first original image through the network or time software installed thereon. For another example, the image processing device determines, through a sensor configured on it, the information on the illumination intensity when acquiring the first original image, and/or the information on the illumination direction when acquiring the first original image, and/or the information on the illumination direction when acquiring the first original image. brightness information. For another example, the image processing device determines the location information when the first original image is acquired through the map software installed thereon.

S502: The image processing device uses the first environment information as an input parameter of the image processing model to obtain the first image processing parameters.

In one possible implementation, the image processing device inputs the first environment information into the image processing model, that is, the first image processing parameters can be obtained. That is to say, the first image processing parameters can be obtained by calculating the first environment information using the function or algorithm in the image processing model.

In the embodiment of the present application, the first image processing parameters may include at least one of the following: parameters in the demosaic function, parameters in the black level correction function, parameters in the lens correction function, and parameters in the gamma correction function (such as gamma parameters), parameters in the white balance function, or parameters in the color mapping function.

S503: The image processing device uses the first image processing parameters to process the first original image to obtain the first processed image.

For example, taking the first image processing parameter including the gamma parameter as an example, the image processing device can use the gamma parameter to perform gamma correction on the first original image to obtain the first processed image.

Optionally, after S503, the image processing device may identify the content in the first processed image. Alternatively, the image processing device may send the first processed image to the image recognition device, so that the image recognition device recognizes the content in the first image. The image recognition device may be the image recognition device 303 in FIG. 3A.

Among them, the actions of the image processing device in the above S501-S503 can be executed by the processor 401 in the image processing device 40 shown in Figure 4 calling the application code stored in the memory 403. This embodiment of the present application does not do anything in this regard. limit.

Based on the method shown in Figure 5, the image processing device can obtain the first image processing parameters based on the environmental information and the image processing model when acquiring the first original image, and process the first original image according to the first image processing parameters to obtain Processed image. In the method shown in Figure 5, it does not involve the generation and redistribution of grayscale histograms and other computationally intensive processes. Instead, it uses an image processing model trained in advance and related to environmental information to obtain the first image. Processing parameters: process the first original image according to the first image processing parameters. The calculation process is relatively simple and the calculation amount is small. It can be applied to devices with low computing power, so that the device can also perform image processing, or, application For scenarios with high real-time requirements, such as smart driving scenarios, the smart driving vehicle can identify targets (such as vehicles, pedestrians, lane lines or obstacles, etc.) as quickly as possible based on the first processed image. Moreover, the method shown in Figure 5 above obtains the first image processing parameters based on the environment information and the image processing model when acquiring the first original image. That is to say, the first image processing parameters are related to the environment information and are related to the environment information. The original image has nothing to do, so there will be no problem of the processed image being too bright and overexposed due to dark objects in the original image. In addition, since the first image processing parameters are parameters related to environmental information, if the environmental information corresponding to the two original images is the same (or has little difference), then the first image processing parameters corresponding to the two original images are also the same. (Or not much different). Therefore, if this method is used to process video streams, the problem of video flickering will not occur.

As mentioned above, the image processing device can obtain the image processing model by itself. Specifically, as shown in Figure 6, the image processing device can obtain the image processing model through the following steps:

S5011: The image processing device obtains at least one second environment information and at least one second original image.

In this embodiment of the present application, at least one second original image includes a second original image corresponding to each of the at least one second environment information. In other words, one piece of second environment information may correspond to at least one second original image. For example, the image processing device acquires 2 pieces of second environment information and 50 pieces of second original images. Among them, the first second environment information corresponds to the first 20 second original images, and the second second environment information corresponds to the last 30 second original images.

It can be understood that the image processing device can obtain at least one second environment information and/or at least one second original image in various ways. For example, the image processing device can obtain the above information locally, or the image processing device can obtain the above information through other devices or equipment, or the image processing device can obtain the above information by itself. Specifically, reference may be made to the corresponding description of the image processing device acquiring the first original image and/or the first environment information in S501, which will not be described again.

S5012: The image processing device acquires at least one target image.

In this embodiment of the present application, at least one target image includes a target image corresponding to each second environment information in at least one second environment information. In other words, one piece of second environment information can correspond to at least one target image. The target image may be an image obtained by processing the third original image, and the target image meets certain requirements.

It can be understood that the image processing device can obtain at least one target image in various ways. Exemplarily, the image processing device may acquire at least one target image locally, that is, at least one target image is stored locally in advance. Alternatively, the image processing device can obtain at least one target image through other devices or equipment. For example, other devices or equipment use the method shown in Figure 1 to process at least one third original image to obtain the at least one third original image corresponding to at least one target image, and sending the at least one target image to the image processing device. Alternatively, the image processing device may acquire at least one target image by itself. For example, the image processing device uses the method shown in FIG. 1 to process at least one third original image to obtain at least one target image corresponding to at least one third original image. It should be understood that in addition to the method shown in Figure 1, other methods can also be used to obtain the target image without limitation.

In this embodiment of the present application, the second environment information is information about the environment when acquiring the second original image and the third original image corresponding to the second environment information. Here, the third original image is the original image of the target image corresponding to the second environment information. The second environment information includes at least one of the following: acquiring the time information of the second original image and the third original image corresponding to the second environment information, and acquiring the second original image and the third original image corresponding to the second environment information. Information on the illumination intensity, information on the illumination direction when obtaining the second original image and the third original image corresponding to the second environmental information, and obtaining the second original image and the third original image corresponding to the second environmental information. Brightness information, or position information when acquiring the second original image and the third original image corresponding to the second environment information.

For example, assuming that the second environment information includes the time at which the second original image and the third original image corresponding to the second environment information are acquired, and the time is 12:00, if the second original image corresponding to the second environment information is The images are image 1 and image 2, and the third original image corresponding to the second environment information is image 3. Then the time at which image 1, image 2 and image 3 are acquired is all 12:00. After processing image 3 using the method shown in Figure 1, the target image corresponding to the second environment information can be obtained.

It can be understood that the type of information included in the second environment information is the same as the type of information included in the first environment information. That is to say, the parameters included in the first environment information will be determined by the parameters included in the second environment information. However, the values of parameters included in the second environment information may not be the same as the values of parameters included in the first environment information. Same, maybe different. For example, if the first environment information includes information about the illumination intensity when acquiring the first original image, then the second environment information includes information about the illumination intensity when acquiring the second original image and the third original image corresponding to the second environment information, and The illumination intensity information included in the first environment information and the illumination intensity information included in the second environment information may be the same or different. For another example, if the first environment information includes information on the illumination direction when acquiring the first original image and brightness information when acquiring the first original image, then the second environment information includes acquiring the second original image corresponding to the second environment information and The illumination direction information of the third original image, and the brightness information of the second original image and the third original image corresponding to the second environment information, and the first environment information includes the illumination direction information and the second environment information The included illumination direction information may be the same or different, and the brightness information included in the first environment information and the brightness information included in the second environment information may be the same or different.

It can be understood that other introduction of the second environment information can be the corresponding description of the first environment information in parameter S501, which will not be described again.

S5013: The image processing device acquires an image processing model based on at least one second environment information, at least one second original image, and at least one target image.

It can be understood that the image processing device can train at least one second environment information, at least one second original image and at least one target image to obtain an image processing model. The greater the number of at least one second environment information, and/or the number of at least one second original image, and/or the number of at least one target image, the better the robustness and accuracy of the obtained image processing model. high. The image processing parameters (such as the first image processing parameters) obtained through the image processing model are also more accurate.

In one possible implementation, the image processing device uses at least one second image processing parameter to respectively process at least one second original image to obtain at least one second processed image, and performs processing according to at least one second environment information and at least one second Image processing parameters to obtain the image processing model.

In the embodiment of the present application, at least one second image processing parameter corresponds to at least one second original image one-to-one, and the second image processing parameters corresponding to different second original images may be the same or different. The absolute value of the difference between the correlation coefficient of the at least one second processed image and the correlation coefficient of the at least one target image is less than or equal to the first threshold. The correlation coefficient can also be called entropy.

In one possible design, the image processing device can extract features of at least one second processed image, and obtain a correlation coefficient of at least one second processed image based on the extracted features. Specifically, as shown in FIG. 7 , the image processing device can obtain the grayscale image of each second processed image in at least one second processed image, and obtain at least one grayscale histogram based on the at least one grayscale image. A gray histogram is averaged to obtain the first average gray histogram. Subsequently, the image processing device may determine the correlation coefficient of at least one second processed image based on the first average grayscale histogram.

As an example, the correlation coefficient of at least one second processed image can satisfy the formula:

Among them, H ₁ is the correlation coefficient of at least one second processed image, _xi is the pixel value, specifically it can be an integer greater than or equal to 0 and less than or equal to 255, p(xi ₎ is the point with pixel value _xi in the first The probability corresponding to the average gray-scale histogram. This probability can represent the probability that a point with pixel value _xi appears in at least one gray-scale histogram. N is 255.

It can be understood that the method by which the image processing apparatus obtains the correlation coefficient of at least one target image is similar to the method of obtaining the correlation coefficient of at least one second processed image. For example, the image processing device may extract features of at least one target image, and obtain a correlation coefficient of at least one target image based on the extracted features. Specifically, the image processing device can acquire a grayscale image of each target image in at least one target image, obtain at least one grayscale histogram based on the at least one grayscale image, average the at least one grayscale histogram, and obtain a third Two average grayscale histograms. Subsequently, the image processing device may determine the correlation coefficient of at least one target image according to the second average grayscale histogram.

As an example, the correlation coefficient of at least one target image can satisfy the formula:

Among them, H ₂ is the correlation coefficient of at least one target image, _xi is the pixel value, specifically it can be an integer greater than or equal to 0 and less than or equal to 255, q(xi ₎ is the second average gray value of the point with pixel value _xi The probability corresponding to the degree histogram. This probability can represent the probability that a point with a pixel value of x _i appears in at least one gray level histogram corresponding to at least one target image. N is 255.

It can be understood that the absolute value of the difference between the correlation coefficient of at least one second processing image and the correlation coefficient of at least one target image can be expressed as |H ₁ -H ₂ |. This absolute value may represent the similarity between at least one second processed image and at least one target image. Specifically, the larger the absolute value is, the less similar the at least one second processed image is to the at least one target image. The smaller the absolute value is, the more similar the at least one second processed image is to the at least one target image. Therefore, if the absolute value is less than or equal to the first threshold, the similarity between the at least one second processed image and the at least one target image can be high. The first threshold can be set as needed and is not limited.

In addition, the difference between the correlation coefficient of at least one second processed image and the correlation coefficient of at least one target image can also be characterized by KL divergence (Kullback–Leibler divergence) or relative entropy, or in other words, at least one second The processing image is similar to at least one target image. For example, the KL divergence of at least one second processing image and at least one target image can satisfy the formula:

Among them, D _KL (p||q) is the KL divergence of at least one second processing image and at least one target image, _xi is a pixel value, specifically it can be an integer greater than or equal to 0 and less than or equal to 255, p( _xi ) is the probability that the point with pixel value _xi corresponds to the first average grayscale histogram, and q(xi ₎ is the probability that the point with pixel value _xi corresponds to the second average grayscale histogram. Generally, a larger value of KL divergence indicates that the at least one second processing image and at least one target image are less similar, and a smaller value of KL divergence indicates that the at least one second processing image and at least one target image are more similar. Therefore, if the value of the KL divergence is less than or equal to the first threshold, the similarity between the at least one second processed image and the at least one target image can be made high. The first threshold can be set as needed and is not limited.

It can be seen from the above description that by using at least one second image processing parameter to process at least one second original image respectively, at least one second processed image that is highly similar to at least one target image can be obtained. Therefore, subsequently, the image processing device can obtain an image processing model based on at least one second environment information and at least one second image processing parameter. In this way, after the first image processing parameters obtained according to the first environment information and the image processing model are used to process the first original image, a first processed image with a high similarity to at least one target image can be obtained. That is to say, by inputting the environmental information into the image processing model obtained according to the above method, the image processing parameters can be obtained. Using the image processing parameters to process the original image corresponding to the environmental information can make the processed image similar to at least one target image. The degree is higher. Therefore, the processed image can also meet the requirements met by the target image.

Taking the image processing model as a quadratic function as an example, the following describes the process in which the image processing device obtains the image processing model based on at least one second environment information and at least one second image processing parameter.

As an example, as shown in Figure 8, the image processing model is y=ax ² +bx+c, where y is the second image processing parameter and x is the second environment information, then the values of a, b and c are obtained , that is, the image processing model can be determined. It can be understood that the values of a, b and c can be determined through at least three second image processing parameters and at least three second environmental information respectively corresponding to the at least three second image processing parameters, that is, the image processing can be obtained Model.

It can be understood that the image processing device can obtain at least one second image processing parameter in various ways.

In one possible implementation, at least one second image processing parameter can be preconfigured in the image processing device based on experience, and the image processing device obtains at least one second image processing parameter locally.

In another possible implementation, the image processing device obtains a set of candidate image processing parameters. The set of candidate image processing parameters includes a plurality of candidate image processing parameters, the plurality of candidate image processing parameters including at least one second processing parameter. That is to say, the image processing device can first obtain multiple candidate parameters, and then use the candidate parameters to process the second original image to obtain a processed image. If the correlation coefficient of the processed image and the correlation coefficient of the target image If the absolute value of the difference is less than or equal to the first threshold, the image processing device determines the candidate parameter corresponding to the processed image as the second processing parameter.

In one possible design, the image processing device obtains initial image processing parameters and performs a first operation on the initial image processing parameters to obtain a set of candidate image processing parameters. Among them, the initial image processing parameters can be set as needed. The first operation includes at least one of the following: determining candidate image processing parameters according to rules based on initial image processing parameters, randomly determining candidate image processing parameters based on initial image processing parameters, or determining a set of candidate image processing parameters based on historically determined candidate image processing parameters. .

As an example, if the initial image processing parameter is 2.0, the image processing device can determine 20 values in the interval from 1.90 to 2.10 with an interval of 0.01 as multiple candidate image processing parameters. Alternatively, if the initial image processing parameter is 2.0, the image processing device can randomly add or subtract a number within 0.3 to 2.0 to obtain multiple candidate image processing parameters. Alternatively, the image processing apparatus may determine all or part of the candidate image processing parameters determined historically as a plurality of candidate image processing parameters. Alternatively, if the initial image processing parameter is 2.0, the image processing device can determine 20 values in the interval from 1.90 to 2.10 at intervals of 0.01, and then select among the 20 values that have not been historically determined as candidate image processing parameters. Values are determined as multiple candidate image processing parameters. Alternatively, if the initial image processing parameter is 2.0, the image processing device can randomly add or subtract a number within 0.3 to 2.0 to obtain 30 values, and then select among the 30 values, values that have not been historically determined as candidate image processing parameters. , determined as multiple candidate image processing parameters.

It can be understood that in the above example, the image processing device first obtains a set of candidate image processing parameters, and then processes the second original image according to the parameters in the set to obtain at least one second image processing parameter. In addition, in specific applications, the image processing device can also obtain a candidate image processing parameter, and process the second original image according to the candidate image processing parameter. If the correlation coefficient of the processed image and the target image If the absolute value of the difference in correlation coefficients is less than or equal to the first threshold, then the candidate image processing parameter is the second image processing parameter. After that, the image processing device obtains a candidate image processing parameter again, and processes the second original image according to the candidate image processing parameter. If the absolute value of the difference between the correlation coefficient of the processed image and the correlation coefficient of the target image is less than or equal to the first a threshold, then the candidate image processing parameter is the second image processing parameter. By analogy, until the image processing device obtains sufficient second image processing parameters.

Based on the method shown in Figure 6, the image processing device can train at least one second image processing parameter and at least one second environment information to obtain an image processing model. Wherein, at least one second image processing parameter can make at least one second processed image have a high similarity with at least one target image. Therefore, environmental information (such as first environmental information) is input into the image processing model obtained according to the above method. , the image processing parameters (such as the first image processing parameters) can be obtained, and the original image (such as the first original image) corresponding to the environmental information is processed by using the image processing parameters, so that the processed image (such as the first processed image) can be compared with at least A target image has higher similarity. In other words, the processed image can also meet the requirements of the target image.

Among them, the actions of the image processing device in the above-mentioned S5011-S5013 can be executed by the processor 401 in the image processing device 40 shown in Figure 4 calling the application code stored in the memory 403. This embodiment of the present application does not do anything in this regard. limit.

The various embodiments mentioned above in this application can be combined without any limitation, as long as the solutions are not inconsistent.

It can be understood that in each of the above embodiments, the methods and/or steps implemented by the image processing device can also be implemented by components (such as chips or circuits) that can be used in the image processing device.

The above has introduced the solutions provided by the embodiments of the present application. Correspondingly, embodiments of the present application also provide an image processing device, which may be the image processing device in the above method embodiment, or a device including the above image processing device, or a component that can be used in the image processing device. It can be understood that, in order to implement the above functions, the above-mentioned image processing device includes hardware structures and/or software modules corresponding to each function. Persons skilled in the art should easily realize that, with the units and algorithm operations of each example described in conjunction with the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is performed by hardware or computer software driving the hardware depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each specific application, but such implementations should not be considered beyond the scope of this application.

Embodiments of the present application can divide the image processing device into functional modules according to the above method examples. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above integrated modules can be implemented in the form of hardware or software function modules. It can be understood that the division of modules in the embodiment of the present application is schematic and is only a logical function division. In actual implementation, there may be other division methods.

For example, when each functional module is divided into integrated modules, FIG. 9 shows a schematic structural diagram of an image processing device 90 . The image processing device 90 includes a processing module 901 . The processing module 901, which may also be called a processing unit, is used to perform operations other than sending and receiving operations, and may be, for example, a processing circuit or a processor.

In some embodiments, the image processing device 90 may also include a storage module (not shown in FIG. 9) for storing program instructions and data.

Illustratively, the image processing device 90 is used to implement the functions of the above image processing device. The image processing device 90 is, for example, the image processing device described in the embodiment shown in FIG. 5 or the embodiment shown in FIG. 6 .

Among them, the processing module 901 is used to obtain the image processing model, the first original image and the first environment information. The first environment information is information about the environment when the first original image is acquired. For example, the processing module 901 may be used to perform S501.

The processing module 901 is also used to use the first environment information as an input parameter of the image processing model to obtain the first image processing parameters. For example, the processing module 901 can also be used to perform S502.

The processing module 901 is also used to process the first original image using the first image processing parameters to obtain the first processed image. For example, the processing module 901 can also be used to perform S503.

In a possible implementation, the first environment information includes at least one of the following: time information for obtaining the first original image, information on the illumination intensity when obtaining the first original image, information on the illumination direction when obtaining the first original image, information on obtaining the first original image. The brightness information of an original image or the position information of the first original image.

In one possible implementation, the processing module 901 is specifically configured to obtain at least one second environment information and at least one second original image. The at least one second original image includes at least one second environment information corresponding to each second environment information. The second original image; the processing module 901 is also specifically configured to obtain at least one target image. The at least one target image includes a target image corresponding to each second environmental information in at least one second environmental information. The second environmental information is to obtain the first The second original image and the third original image corresponding to the second environmental information are environmental information, the third original image is the original image of the target image corresponding to the second environmental information, and the type of information included in the second environmental information is the same as that of the first environment. The information includes the same type of information; the processing module 901 is also specifically configured to obtain an image processing model based on at least one second environment information, at least one second original image, and at least one target image.

In one possible implementation, the processing module 901 is also specifically configured to use at least one second image processing parameter to process at least one second original image respectively to obtain at least one second processed image and the correlation of at least one second processed image. The absolute value of the difference between the coefficient and the correlation coefficient of at least one target image is less than or equal to the first threshold; the processing module 901 is also specifically configured to obtain an image processing model based on at least one second environment information and at least one second image processing parameter.

In one possible implementation, the processing module 901 is also used to obtain a candidate image processing parameter set, where the candidate image processing parameter set includes a plurality of candidate image processing parameters, and the plurality of candidate image processing parameters include at least one second processing parameter.

In one possible implementation, the processing module 901 is also used to obtain initial image processing parameters; the processing module 901 is also used to perform a first operation on the initial image processing parameters to obtain a set of candidate image processing parameters.

In a possible implementation, the first operation includes at least one of the following: determining candidate image processing parameters according to rules according to initial image processing parameters, randomly determining candidate image processing parameters based on initial image processing parameters, or determining candidate image processing parameters based on history. Parameters determine a set of candidate image processing parameters.

In one possible implementation, the image processing model includes functions or algorithms.

When used to implement the functions of the image processing device, for other functions that the image processing device 90 can implement, reference may be made to the embodiment shown in FIG. 5 or the relevant introduction of the embodiment shown in FIG. 6 , and will not be described again.

In a simple embodiment, those skilled in the art can imagine that the image processing device 90 may take the form shown in FIG. 4 . For example, the processor 401 in Figure 4 can cause the image processing device 90 to execute the method described in the above method embodiment by calling the computer execution instructions stored in the memory 403.

For example, the function/implementation process of the processing module 901 in Figure 9 can be implemented by the processor 401 in Figure 4 calling the computer execution instructions stored in the memory 403.

It can be understood that one or more of the above modules or units can be implemented in software, hardware, or a combination of both. When any of the above modules or units is implemented in software, the software exists in the form of computer program instructions and is stored in the memory. The processor can be used to execute the program instructions and implement the above method flow. The processor can be built into an SoC (System on a Chip) or ASIC, or it can be an independent semiconductor chip. In addition to the core used to execute software instructions for calculation or processing, the processor can further include necessary hardware accelerators, such as field programmable gate array (FPGA), PLD (programmable logic device) , or a logic circuit that implements dedicated logic operations.

When the above modules or units are implemented in hardware, the hardware can be a CPU, a microprocessor, a digital signal processing (DSP) chip, a microcontroller unit (MCU), an artificial intelligence processor, an ASIC, Any one or any combination of SoC, FPGA, PLD, dedicated digital circuits, hardware accelerators or non-integrated discrete devices, which can run the necessary software or not rely on software to perform the above method flow.

Optionally, embodiments of the present application also provide a chip system, including: at least one processor and an interface. The at least one processor is coupled to the memory through the interface. When the at least one processor executes the computer program or instructions in the memory When, the method in any of the above method embodiments is executed. In a possible implementation, the chip system further includes a memory. Optionally, the chip system may be composed of chips, or may include chips and other discrete devices, which is not specifically limited in the embodiments of the present application.

Optionally, embodiments of the present application also provide a computer-readable storage medium. All or part of the processes in the above method embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in the above computer-readable storage medium. When executed, the program can include the processes of the above method embodiments. . The computer-readable storage medium may be an internal storage unit of the image processing device of any of the aforementioned embodiments, such as a hard disk or memory of the image processing device. The computer-readable storage medium may also be an external storage device of the image processing device, such as a plug-in hard drive, a smart media card (SMC), or a secure digital (SD) equipped on the image processing device. card, flash card, etc. Furthermore, the above computer-readable storage medium may also include both the internal storage unit of the above image processing apparatus and an external storage device. The above-mentioned computer-readable storage medium is used to store the above-mentioned computer program and other programs and data required by the above-mentioned image processing apparatus. The above-mentioned computer-readable storage media can also be used to temporarily store data that has been output or is to be output.

Optionally, the embodiment of the present application also provides a computer program product. All or part of the processes in the above method embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in the above computer program product. When executed, the program can include the processes of the above method embodiments.

Optionally, the embodiment of the present application also provides a computer instruction. All or part of the processes in the above method embodiments can be completed by computer instructions to instruct related hardware (such as computers, processors, access network equipment, mobility management network elements or session management network elements, etc.). The program may be stored in the above-mentioned computer-readable storage medium or in the above-mentioned computer program product.

Optionally, embodiments of the present application also provide an intelligent driving vehicle, including the image processing device in the above embodiment.

Through the above description of the embodiments, those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above functional modules is used as an example. In actual applications, the above functions can be allocated as needed. It is completed by different functional modules, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods, for example, multiple units or components may be The combination can either be integrated into another device, or some features can be omitted, or not implemented. On the other hand, the coupling or direct coupling or communication connection between each other shown or discussed may be through some interfaces, and the indirect coupling or communication connection of the devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated. The components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place, or they may be distributed to multiple different places. . Some or all of the units can be selected according to actual needs to achieve the purpose of this embodiment.

In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated units can be implemented in the form of hardware or software functional units.

The above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be covered by the protection scope of the present application. . Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

An image processing method, characterized in that the method includes:

Obtain the image processing model, the first original image and the first environment information, where the first environment information is the information of the environment when the first original image is acquired;

Use the first environment information as an input parameter of the image processing model to obtain first image processing parameters;

The first original image is processed using the first image processing parameter to obtain a first processed image.
The method according to claim 1, characterized in that the first environment information includes at least one of the following: obtaining time information of the first original image, information on illumination intensity when obtaining the first original image, obtaining The first original image is the information of the illumination direction, the brightness information when the first original image is obtained, or the position information when the first original image is obtained.
The method according to claim 1 or 2, characterized in that said obtaining the image processing model includes:

Obtaining at least one second environmental information and at least one second original image, the at least one second original image including a second original image corresponding to each second environmental information in the at least one second environmental information;

Acquire at least one target image, the at least one target image includes a target image corresponding to each second environment information in the at least one second environment information, and the second environment information is to obtain the third environment information corresponding to the second environment information. When the second original image and the third original image are environmental information, the third original image is the original image of the target image corresponding to the second environmental information, and the type of information included in the second environmental information is the same as that of the third original image. The environmental information includes the same type of information;

The image processing model is obtained according to the at least one second environment information, the at least one second original image and the at least one target image.
The method of claim 3, wherein obtaining the image processing model based on the at least one second environment information, the at least one second original image and the at least one target image includes:

Using at least one second image processing parameter to process the at least one second original image respectively to obtain at least one second processed image, the correlation coefficient of the at least one second processed image and the correlation coefficient of the at least one target image The absolute value of the difference is less than or equal to the first threshold;

The image processing model is obtained according to the at least one second environment information and the at least one second image processing parameter.
The method according to claim 4, characterized in that, before using at least one second image processing parameter to respectively process the at least one second original image to obtain at least one second processed image, the method further includes:

A set of candidate image processing parameters is obtained, the set of candidate image processing parameters includes a plurality of candidate image processing parameters, and the plurality of candidate image processing parameters include the at least one second processing parameter.
The method of claim 5, further comprising:

Get initial image processing parameters;

Perform a first operation on the initial image processing parameters to obtain the set of candidate image processing parameters.
The method of claim 6, wherein the first operation includes at least one of the following: determining candidate image processing parameters according to rules according to the initial image processing parameters, and randomly determining candidate images according to the initial image processing parameters. processing parameters, or determining the set of candidate image processing parameters based on historically determined candidate image processing parameters.
The method according to any one of claims 1-7, characterized in that the image processing model includes a function or algorithm.
An image processing device, characterized in that the image processing device includes: a processing module;

The processing module is used to obtain the image processing model, the first original image and the first environmental information. The first environmental information is the information of the environment when the first original image is obtained;

The processing module is also configured to use the first environment information as an input parameter of the image processing model to obtain the first image processing parameter;

The processing module is also used to process the first original image using the first image processing parameter to obtain a first processed image.
The image processing device according to claim 9, characterized in that the first environment information includes at least one of the following: time information for obtaining the first original image, information on illumination intensity when obtaining the first original image. , the illumination direction information when acquiring the first original image, the brightness information when acquiring the first original image, or the position information when acquiring the first original image.
The image processing device according to claim 9 or 10, characterized in that:

The processing module is specifically configured to obtain at least one second environment information and at least one second original image. The at least one second original image includes a third image corresponding to each second environment information in the at least one second environment information. 2 original images;

The processing module is further specifically configured to acquire at least one target image, which includes a target image corresponding to each second environment information in the at least one second environment information, and the second environment information is obtained by The second original image and the third original image corresponding to the second environmental information are environmental information. The third original image is the original image of the target image corresponding to the second environmental information. The second environmental information The type of information included is the same as the type of information included in the first environment information;

The processing module is further specifically configured to obtain the image processing model based on the at least one second environment information, the at least one second original image, and the at least one target image.
The image processing device according to claim 11, characterized in that:

The processing module is further specifically configured to use at least one second image processing parameter to respectively process the at least one second original image to obtain at least one second processed image, and the correlation coefficient of the at least one second processed image is The absolute value of the difference between the correlation coefficients of the at least one target image is less than or equal to the first threshold;

The processing module is further specifically configured to obtain the image processing model according to the at least one second environment information and the at least one second image processing parameter.
The image processing device according to claim 12, characterized in that:

The processing module is also configured to obtain a candidate image processing parameter set, the candidate image processing parameter set includes a plurality of candidate image processing parameters, and the plurality of candidate image processing parameters include the at least one second processing parameter.
The image processing device according to claim 13, characterized in that:

The processing module is also used to obtain initial image processing parameters;

The processing module is also configured to perform a first operation on the initial image processing parameters to obtain the candidate image processing parameter set.
The image processing device according to claim 14, characterized in that the first operation includes at least one of the following: determining candidate image processing parameters according to rules according to the initial image processing parameters, randomly determining according to the initial image processing parameters Candidate image processing parameters, or the set of candidate image processing parameters is determined based on historically determined candidate image processing parameters.
The image processing device according to any one of claims 9-15, characterized in that the image processing model includes a function or algorithm.
An image processing device, characterized in that it includes: a processor, the processor is coupled to a memory, the memory is used to store programs or instructions, and when the program or instructions are executed by the processor, the The device performs the method according to any one of claims 1 to 8.
A computer-readable storage medium on which a computer program or instructions are stored, characterized in that, when executed, the computer program or instructions cause the computer to perform the method according to any one of claims 1 to 8.
A chip, characterized in that it includes: a processor, the processor is coupled to a memory, the memory is used to store programs or instructions, and when the programs or instructions are executed by the processor, the chip executes The method of any one of claims 1 to 8.
An intelligent driving vehicle, characterized by comprising: the image processing device according to any one of claims 9-16.