WO2022094784A1

WO2022094784A1 - Image processing method and apparatus, and movable platform

Info

Publication number: WO2022094784A1
Application number: PCT/CN2020/126421
Authority: WO
Inventors: 吴一凡; 刘怀宇; 钟文辉
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2020-11-04
Filing date: 2020-11-04
Publication date: 2022-05-12

Abstract

An image processing method and apparatus, and a movable platform. The method comprises: an image acquisition device cyclically acquiring images according to N different amounts of exposure; for a frame of first image currently acquired by the image acquisition device, fusing the first image and an (N-1) frames of second images acquired before the first image, so as to obtain a first high-dynamic-range (HDR) target image, the (N-1) frames of second images having different amounts of exposure from one another and having a different amount of exposure from the first image; and fusing the first image and (N-1) frames of third images acquired after the first image, so as to obtain a second HDR target image, the (N-1) frames of third images having different amounts of exposure from one another and having a different amount of exposure from the first image. By means of the repetitive use of each frame of image acquired by the image acquisition device and the fusion of each frame of image and images acquired before and after same, more HDR images can be obtained by means of synthesis, thereby greatly increasing the frame rate of the acquired HDR images.

Description

Image processing method, device and movable platform

technical field

The present application relates to the technical field of image processing, and in particular, to an image processing method, device and movable platform.

Background technique

Compared with ordinary images, HDR (High Dynamic Range) images have a higher dynamic range. When the brightness of the shooting scene is large or small, the image will not be overexposed or too dark, and the details of the image can be well displayed. , Therefore, some image acquisition devices are currently equipped with HDR function. However, at present, the HDR function of the image acquisition device acquires multiple frames of images by exposing the shooting scene to different degrees, and then uses the multiple frames of images to synthesize a frame of HDR image. For example, the scene is exposed with high exposure and low exposure to obtain two frames of images, and then two frames of images are used to synthesize one frame of HDR image. Although the dynamic range of the collected image is improved in this way, only one frame of image can be obtained by collecting multiple frames of images, which greatly reduces the frame rate of the image.

SUMMARY OF THE INVENTION

In view of this, the present application provides an image processing method, device and movable platform.

According to a first aspect of the present application, an image processing method is provided, the method comprising:

During the process that the image acquisition device cyclically performs image acquisition according to N exposures, the first image currently acquired by the image acquisition device is acquired, where N is an integer greater than or equal to 2;

The first image is fused with N-1 frames of second images collected before the first image to obtain a first target image, and the exposures of the N-1 frames of second images are different and different. exposure to the first image;

After N-1 frames of third images collected after the first image are acquired, the first image and the third image are fused to obtain a second target image, the N-1 frames of the third image The exposures are different and different from the exposure of the first image.

According to a second aspect of the present application, an image processing apparatus is provided, the apparatus includes a processor, a memory, and a computer program stored in the memory for execution by the processor, when the processor executes the computer program , implement the following steps:

According to a third aspect of the present application, a movable platform is provided, the movable platform includes an image acquisition device and an image processing chip, and the image processing chip includes a processor and a memory, and storage in the memory is available for the processing. A computer program executed by a processor, when the processor executes the computer program, the following steps are implemented:

During the process that the image acquisition device cyclically performs image acquisition according to N exposures, a first image currently acquired by the image acquisition device is acquired, where N is an integer greater than or equal to 2;

According to a fourth aspect of the present application, there is provided a computer-readable storage medium on which computer program instructions are stored. When the instructions are executed by a processor, the image processing method mentioned in the first aspect can be implemented.

By applying the solution provided by the present application, the image acquisition device can be made to perform image acquisition cyclically according to N different exposures, and for a frame of the first image currently collected by the image acquisition device, the first image can be compared with the first image acquired before the first image. The N-1 frame exposures of the N-1 frames are different and different from the exposure of the first image, and the second image is fused to obtain the first target image with high dynamic range. The exposure amount of the N-1 frames is different, and the third image is fused with the exposure amount different from the first image to obtain the second target image with high dynamic range. By reusing each frame of image collected by the image acquisition device and fusing it with images of different exposures collected before and after, more HDR images can be synthesized and the frame rate of HDR images can be greatly improved.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present application more clearly, the following briefly introduces the drawings that are used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative labor.

FIG. 1 is a schematic diagram of synthesizing an HDR image in the related art.

FIG. 2 is a flowchart of an image processing method according to an embodiment of the present application.

FIG. 3( a ) is a schematic diagram of synthesizing an HDR image according to an embodiment of the present application.

FIG. 3(b) is a schematic diagram of synthesizing an HDR image according to an embodiment of the present application.

FIG. 4( a ) is a schematic diagram of an application scenario of an embodiment of the present application.

Figure 4(b) is a schematic diagram of the internal structure of an unmanned aerial vehicle according to an embodiment of the present application.

FIG. 5 is a schematic diagram of a logical structure of an image processing apparatus according to an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

In a shooting scene with low light or bright light, the image collected by the image acquisition device may be partially overexposed or too dark, and the image details cannot be well displayed. In order to improve the dynamic range of the image, at present, many image capture devices are equipped with the HDR function. When the user turns on the HDR function, the image capture device will capture images of the shooting scene with different exposures to obtain multiple frames of images, and then use the HDR function. Multiple frames of images generate one frame of HDR image. Exposure is the amount of light reaching the photosensitive element when the image acquisition device captures an image. The brightness of the image captured by the image acquisition device is related to the exposure. The greater the exposure, the greater the brightness of the captured image. The exposure is related to the exposure parameters (exposure duration, aperture size, and ISO) of the image acquisition device and ambient brightness. By adjusting the exposure parameters, the exposure can be controlled, thereby adjusting the brightness of the image. When generating an HDR image, the image acquisition device can be controlled to acquire images with different exposures, for example, a high-exposure image and a low-exposure image are acquired, and then two frames of images are used to synthesize an HDR image.

Although this method can improve the dynamic range of the image collected by the image collection device at present, it greatly reduces the frame rate of the image because it needs to collect multiple frames of images to synthesize one frame of image. As shown in FIG. 1 , it is a schematic diagram of obtaining one frame of HDR image by using one frame of high-exposure image and one frame of low-exposure image. It can be seen from the figure that after 4 frames of images are collected, only 2 frames of HDR images can be synthesized. Using this method to generate HDR images, assuming that the original 1s image acquisition device can collect 60 frames of images, now only 30 frames of images can be obtained in 1s. The frame rate is reduced from 60 frames/s to 30 frames/s, and the frame rate of the image will be reduced by half.

Based on this, the present application provides an image processing method, which can allow an image acquisition device to cyclically perform image acquisition according to N different exposures, and for each frame of image collected by the image acquisition device, it can be reused, and the The HDR image is obtained by merging with the image collected before and with the image collected later, and the frame rate of the image can be greatly improved by reusing each frame of the image to synthesize the HDR image.

The image processing method of the present application can be performed by an image acquisition device, wherein the image acquisition device can be any device with an image acquisition function, such as a camera, a mobile phone, a tablet, a PTZ with a camera, and other devices. In some embodiments, if the image acquisition device is mounted on a movable platform such as an unmanned aerial vehicle, an unmanned vehicle, a gimbal, etc., the image processing method can also be executed by a movable platform, such as by a drone on a gimbal or a gimbal. Processing chip execution. In some embodiments, the image processing method may also be performed by other devices that are communicatively connected to the image capture device. For example, after the image capture device captures an image, it can be sent to a remote server for image processing, or, if the image capture device The device is mounted on the movable platform, and can also be sent to the control terminal of the movable platform for image processing, and then displayed to the user. Of course, for other application scenarios, the user can also set the execution body of the image processing method by himself, which is not limited in this application.

For ease of distinction, hereinafter, a frame of images currently captured by the image capturing device is referred to as a first image, an image captured before the first image is referred to as a second image, and an image captured after the first image is referred to as a third image. The image with larger dynamic range obtained by fusing the first image and the second image is called the first target image, and the image with larger dynamic range obtained by fusing the first image and the third image is called the second target image.

Specifically, the image processing method, as shown in Figure 2, includes the following steps:

S202. During the process of image acquisition by the image acquisition device cyclically according to N exposures, acquire a first image currently acquired by the image acquisition device, where N is an integer greater than or equal to 2;

S204. Fusion of the first image and N-1 frames of second images collected before the first image to obtain a first target image, where the exposures of the N-1 frames of second images are different, and different from the exposure of the first image;

S206. After acquiring the N-1 frames of the third image collected after the first image, fuse the first image and the third image to obtain a second target image, the N-1 frame of the third image. The exposure levels of the three images are different and different from the exposure levels of the first image.

In order to make each frame of image collected by the image capture device reusable, that is, it can be fused with the image captured before or after the image to obtain an image with a larger dynamic range, the image capture device can be cyclically arranged according to N kinds of images. Different exposures are used for image acquisition, N represents the type of exposure, and the exposure can be set to 2, 3, or even more. For example, images can be collected cyclically according to high exposure and low exposure, or images can be collected cyclically according to high exposure, normal exposure, and low exposure, where high exposure, normal exposure, and low exposure represent three For different exposure values, the relationship of exposure is: high exposure > normal exposure > low exposure. The exposure types can be set according to actual needs. For example, if the dynamic range of the fused image is more refined, the exposure types can be appropriately increased.

After the user turns on the HDR function, the image capture device can enter a mode of cyclically capturing images according to N exposures. For each frame of the first image captured by the image capture device at the current moment, the image capture device can The collected N-1 frames of the second images are fused to obtain a frame of the first target image with a larger dynamic range, wherein the exposures of the N-1 frames of the second images are different and different from the exposure of the first image. In this way, it can be ensured that each frame of the first target image is obtained by synthesizing N kinds of images with different exposures, thereby ensuring that each frame of the first target image has a larger dynamic range. Similarly, after N-1 frames of the third image collected after the first image are acquired, the first image and the third image can also be fused to obtain a second target image with a larger dynamic range, wherein, The exposures of the N-1 frames of the third image are also different and different from the exposure of the first image, so as to ensure that each frame of the first target image is synthesized from N images with different exposures, ensuring that each frame is The second target image has a larger dynamic range. For example, assuming that the image acquisition device cyclically collects images according to high exposure, normal exposure, and low exposure, and the currently collected image is a low exposure image, the second image can be the exposure collected before the first image. The third image can be two frames of images with high exposure and normal exposure, and the third image can be two frames of images with high exposure and normal exposure acquired after the first image.

Since the image acquisition device cyclically collects images according to different exposures, each frame of images includes images with different exposures before or after it. If low exposure is included, each frame of image can be reused and fused with images of different exposures collected before or after, to obtain images with a larger dynamic range. Since each frame of image is reused, the number of final composite HDR images will be greatly increased. Taking two frames of images with different exposures to obtain one frame of HDR image as an example, in the past, one frame of HDR image required the image acquisition device to collect two frames of images, and then synthesized into one frame of HDR image. It is assumed that the image acquisition device collects 60 frames per second. In the end, only 30 frames of HDR images can be output, and the frame rate is 30 frames/s. However, with the solution provided by the embodiment of the present application, each frame can be synthesized with the previous frame or the next frame to obtain one frame of HDR image, so that after 60 frames of images are collected, 59 frames of HDR images can be synthesized, that is, the frame The rate is 59 frames/s, which greatly improves the frame rate.

Since the image acquisition device cyclically collects images according to N exposures, for each frame of the first image currently collected, the first N-1 frames adjacent to it or the next N-1 frames adjacent to it The images all include exposure amounts other than the exposure amounts corresponding to the first image among the N exposure amounts. For example, assuming that the image acquisition device cyclically collects images according to three exposures: high exposure, normal exposure, and low exposure, and the current image is a low-exposure image, the first two adjacent frames of images are normal exposures respectively. And the image with high exposure, the next two adjacent frames of images are images of high exposure and normal exposure respectively, so, the first image is fused with the adjacent first 2 frames or the adjacent last 2 frames of images, HDR images can be obtained, and since each frame of HDR image contains three exposures: high exposure, normal exposure, and low exposure, the brightness of the different HDR images obtained by synthesis is relatively uniform, and the adjacent images are similar to the scene of the first image. The higher the value, the better the fusion effect can be obtained. Therefore, in some embodiments, N-1 frames of second images or N-1 frames of third images may be consecutive N-1 frames of images adjacent to the first image, and the first images are respectively adjacent to N- One frame of image is fused to obtain one frame of HDR image, as shown in Fig. 3(a), which is a schematic diagram of HDR image synthesis. Of course, in some embodiments, the N-1 frames of the second image or the N-1 frames of the third image may not be adjacent to the first image. For example, the image acquisition device cyclically collects images according to high exposure and low exposure, and the first image currently collected is a low-exposure image. Therefore, the third frame image (also a high-exposure image) before the first image can also be selected. ) and the first image are fused to obtain an HDR image, as shown in FIG. 3(b) , a schematic diagram of HDR image synthesis.

Since the exposure of the image capture device is related to the brightness of the environment and the exposure parameters of the image capture device (exposure duration, aperture size, ISO), the ISO of the image capture device is usually fixed, so when the ambient brightness is determined, it can be passed through Adjust the exposure duration and aperture size to adjust the exposure of the image capture device. Therefore, in some embodiments, N different exposures can be preset, and during the process of image acquisition by the image acquisition device, the exposure duration or aperture size of the image acquisition device can be adjusted in real time according to the brightness of the environment, or the exposure duration can be adjusted at the same time and the aperture size, so that the exposure of the image acquisition device reaches the preset N exposures.

For some scenarios, it is required that the HDR images captured by the image capture device be transmitted to other devices with the lowest possible delay. Take the scene where the drone sends the collected image to the control terminal on the ground as an example, the user can adjust the flight path of the drone according to the image sent by the drone to the control terminal. If the image transmission delay is large, it may be unfavorable. The user adjusts the flight path of the drone in time to cause an accident. In the related art, when generating an HDR image, after the image acquisition device has collected two frames of images with different exposures, two frames of images are used to synthesize one frame of HDR image, and then transmit. Considering that the delay of waiting for two frames of image acquisition, the delay of synthesizing HDR images, and the delay of image transmission are accumulated, the overall image transmission delay will be serious. Since each frame of image in the embodiments of the present application is fused with the previously collected image, in order to reduce the transmission delay of the HDR image, in some embodiments, it is not necessary to wait for the image data of the first image to be completely collected before starting In the synthesis of HDR images, after the image acquisition device collects part of the image data of the first image, it can acquire and store part of the collected image data, and store the part of the image data with the pixels corresponding to the part of the image data in the second image. The image data at the position is fused to obtain a part of the image data of the first target image. In this way, HDR image fusion can be performed while collecting image data, and there is no need to wait until the entire image is collected to perform fusion, which reduces the time for synthesizing HDR images, thereby reducing the delay of HDR images.

Of course, in some embodiments, the image capture device may capture and output image data of each frame of image line by line, so part of the image data may be several lines of image data of the first image. In some embodiments, the image capture device may The data of each frame of image is collected column by column and output, so part of the image data can be several columns of image data of the first image. Wherein, the data amount of the partial image can be determined based on the maximum storage bandwidth of the memory of the device executing the image processing method.

For example, taking the image acquisition device cyclically collecting images according to high exposure and low exposure as an example, when the image acquisition device collects the first 10 lines of image data of the first image, it will combine the first 10 lines of image data with the first image data. The first 10 lines of data of the previous frame image are fused to obtain the data of the first 10 lines of the HDR image, and then the next 10 lines of data are fused with the image data at the corresponding pixel positions of the previous frame image in turn, until the entire image is synthesized. HDR images.

In some embodiments, the image acquisition device can be mounted on a movable platform, and the movable platform can be various devices such as drones, unmanned vehicles, and PTZs. The movable platform may include an image processing chip, and the image processing method may be performed by an image acquisition device or an image processing chip of the movable platform. For example, the image acquisition device includes an image processing chip, and the image acquisition The images are composited into HDR images and output. Or the image acquisition device may only be responsible for acquiring images, and then output the acquired images to the image processing chip of the movable platform, so that the image processing chip can synthesize the HDR image.

In the scenario where the image processing method is performed by the image processing chip on the movable platform, the image processing chip on the movable platform may include a first memory, and the first memory may be a cache memory. When the image acquisition device collects part of the image data, The image processing chip can obtain this part of the image data and cache it in the first memory, and then use the cached part of the image data and the previously collected image to perform HDR image synthesis, wherein the data amount of the part of the image data is based on the maximum storage capacity of the first memory. Bandwidth is determined.

Of course, since each frame of image currently collected by the image collection device can be fused with the second image collected before, after the second image is collected, the second image can be stored so as to be combined with the second image collected later. An image is fused. In some embodiments, the movable platform may also include a second memory in which the second image may be stored.

In some embodiments, the image processing chip on the removable platform may be an ISP chip, and the second memory on the removable platform may be a DDR memory.

In some embodiments, if the image acquisition device is mounted on the movable platform, after the first target image or the second target image is synthesized, the first target image or the second target image can be sent to the control terminal of the movable platform, The control terminal may be various devices for controlling the movement of the movable platform, such as a remote controller, a mobile phone, flying glasses, etc., so that the control terminal displays the first target image or the second target image to the user. Since each frame of image is reused, the frame rate of the image is greatly improved, and the method of synthesizing HDR images while collecting is adopted, the delay in transmitting the synthesized HDR images to the control terminal is greatly reduced, and the user experience is improved.

In some embodiments, after the first target image or the second target image is synthesized, a video may also be generated by using the first target image or the second target image, and then the video may be stored or output to other devices. Since each frame of image is reused, the frame rate of the image is greatly improved, and the video picture is smoother.

In order to further explain the image processing method provided by the present application, a specific embodiment is explained below.

As shown in FIG. 4( a ), which is a schematic diagram of an application scenario of the embodiment of the present application, an image acquisition device 42 is mounted on the drone 41 , and the image acquisition device 42 collects images or videos during the flight of the drone 41 . Afterwards, it will be sent to the control terminal 43 on the ground, so that the user can view the image or video through the control terminal 43, so as to control the flight trajectory of the drone 41. During the flight of the drone 41 , there may be scenes where the image capture device 42 is backlit or facing the sun, and the light is very bright, so the captured image may be too bright or too dark, so that the user cannot see the details of the image clearly. In order to ensure the clarity of the image, an image acquisition device 42 with an HDR function may be used, and an HDR image is obtained by synthesizing the images collected by the image acquisition device 42, and then sent to the user to improve the dynamic range of the image.

The schematic diagram of the internal structure of the drone is shown in FIG. 4( b ). The drone includes an ISP chip 411 and a DDR memory 422 , and the ISP chip 411 includes a cache memory 4111 . After the image acquisition device 42 enables the HDR function, the image acquisition device 42 can cyclically perform image acquisition according to two preset exposures to obtain a high-exposure image and a low-exposure image, and then send the acquired image to the ISP of the drone 41 . chip 411, so that the ISP chip 411 can synthesize the HDR image. Each frame of image collected by the image collection device 42 can be stored in the DDR 412 of the unmanned aerial vehicle 41 so as to be fused with the next frame of image. After the image acquisition device 42 acquires the first frame of image, it can be directly sent to the control terminal 43, or of course, it can be directly stored in the DDR412 of the drone without transmitting. For the second frame image and subsequent images collected by the image capture device 42, the ISP chip 411 all processes according to the following processing logic:

The ISP chip 411 can cache the data of the first N lines of the current frame of the image output by the image acquisition device 42 in the cache memory 4111 of the ISP chip 411, and then obtain the data of the first N lines of the previous frame of the image from the DDR 412, and store the current image. The first N lines of data are fused with the first N lines of data of the previous frame of image to obtain the first N data of the HDR image, and the above process is repeated until the HDR synthesis of the entire image is completed. After synthesizing the HDR image, the ISP chip 411 can further perform noise reduction, lens shadow correction, white balance adjustment, sharpening, local enhancement and other processing on the HDR image, and then encode the processed HDR image and transmit it through wireless communication to the control terminal 43.

Since each frame of image is reused and fused with the previous frame image and the next frame image, the frame rate of the generated HDR image is greatly improved. And when synthesizing HDR images, after the image acquisition device only collects a part of the image data, the HDR image synthesis operation starts, that is, synthesizing while collecting, rather than waiting for the entire image to be collected and then synthesizing the HDR image. Reduce the delay of image transmission.

Correspondingly, the present application also provides an image processing apparatus. As shown in FIG. 5 , the apparatus includes a processor 51 , a memory 52 , and a computer program stored in the memory 52 for execution by the processor 51 . When the processor 51 executes the computer program, the following steps are implemented:

In some embodiments, the N-1 frames of second images or the N-1 frames of third images are adjacent to the first image.

In some embodiments, the N exposure amounts are obtained by adjusting the exposure duration and/or aperture size of the image capture device.

In some embodiments, when the processor is configured to fuse the first image with N-1 frames of second images collected before the first image to obtain the first target image, the processor is specifically configured to:

After the image acquisition device acquires part of the image data of the first image, acquire and store the part of the image data;

The stored part of the image data is fused with the image data at the pixel positions corresponding to the part of the image data in the second image to obtain image data of the target image.

In some embodiments, the partial image data includes several lines of image data of the first image; or

Several columns of image data for the first image.

In some embodiments, the image acquisition device is mounted on a movable platform, the movable platform includes an image processing chip, and the image processing device is the image acquisition device; or

The image processing device is the image processing chip.

In some embodiments, the image processing device is the image processing chip, the image processing chip includes a first memory, and the data amount of the partial image data is determined based on a maximum storage bandwidth of the first memory.

In some embodiments, the movable platform includes a second memory in which the second image is stored.

In some embodiments, the image processing chip includes an ISP chip, and the second memory includes DDR.

In some embodiments, after obtaining the first target image or the second target image, the apparatus is further configured to:

sending the first target image or the second target image to the control terminal of the movable platform, so that the control terminal displays the first target image or the second target image to the user, wherein the control The terminal is used to control the movement of the movable platform.

A video is generated and output using the first target image or the second target image.

The specific implementation details when the image processing apparatus performs image processing may refer to the description in the above-mentioned image processing method, which will not be repeated here.

Further, the present application also provides a movable platform, the movable platform includes an image acquisition device and an image processing chip, and the image processing chip includes a processor and a memory, and the memory is stored in the memory for the processor. Executed computer program, when the processor executes the computer program, the following steps are implemented:

In some embodiments, the image processing chip includes a first memory, and the processor is configured to acquire and store the partial image data in the first image after the image acquisition device collects the partial image data of the first image. the first memory, which fuses the part of the image data stored in the first memory with the image data at the pixel positions corresponding to the part of the image data in the second image to obtain image data of the target image.

In some embodiments, the data amount of the partial image data is determined based on a maximum storage bandwidth of the first memory.

In some embodiments, the movable platform further includes a second memory in which the second image is stored.

In some embodiments, the processor is further configured to send the first target image or the second target image to a control terminal of the movable platform, so that the control terminal sends the first target image or the second target image to the control terminal of the movable platform. The second target image is displayed to the user, wherein the control terminal is used to control the movement of the movable platform.

The specific implementation details when the image processing chip performs image processing can refer to the description in the above image processing method, which will not be repeated here.

Correspondingly, an embodiment of the present specification further provides a computer storage medium, where a program is stored in the storage medium, and when the program is executed by a processor, the image processing method in any of the foregoing embodiments is implemented.

Embodiments of the present specification may take the form of a computer program product embodied on one or more storage media having program code embodied therein, including but not limited to disk storage, CD-ROM, optical storage, and the like. Computer-usable storage media includes permanent and non-permanent, removable and non-removable media, and storage of information can be accomplished by any method or technology. Information may be computer readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Flash Memory or other memory technology, Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

For the apparatus embodiments, since they basically correspond to the method embodiments, reference may be made to the partial descriptions of the method embodiments for related parts. The device embodiments described above are only illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in One place, or it can be distributed over multiple network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution in this embodiment. Those of ordinary skill in the art can understand and implement it without creative effort.

It should be noted that, in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any relationship between these entities or operations. any such actual relationship or sequence exists. The terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion such that a process, method, article or device comprising a series of elements includes not only those elements, but also other not expressly listed elements, or also include elements inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in a process, method, article or apparatus that includes the element.

The methods and devices provided by the embodiments of the present invention have been described in detail above. The principles and implementations of the present invention are described with specific examples in this paper. The descriptions of the above embodiments are only used to help understand the methods of the present invention and its implementation. At the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation and application scope. To sum up, the content of this description should not be construed as a limitation to the present invention. .

Claims

An image processing method, characterized in that the method comprises:

During the process that the image acquisition device cyclically performs image acquisition according to N exposures, the first image currently acquired by the image acquisition device is acquired, where N is an integer greater than or equal to 2;

The first image is fused with N-1 frames of second images collected before the first image to obtain a first target image, and the exposures of the N-1 frames of second images are different and different. exposure to the first image;

After N-1 frames of third images collected after the first image are acquired, the first image and the third image are fused to obtain a second target image, the N-1 frames of the third image The exposures are different and different from the exposure of the first image.
The method according to claim 1, wherein the N-1 frames of the second image or the N-1 frames of the third image are adjacent to the first image.
The method according to claim 1 or 2, wherein the N exposure amounts are obtained by adjusting the exposure duration and/or aperture size of the image acquisition device.
The method according to any one of claims 1-3, wherein the first image is fused with N-1 frames of second images collected before the first image to obtain a first target image, include:

After the image acquisition device acquires part of the image data of the first image, acquire and store the part of the image data;

The stored part of the image data is fused with the image data at the pixel positions corresponding to the part of the image data in the second image to obtain image data of the target image.
The method according to claim 4, wherein the partial image data comprises several lines of image data of the first image; or

Several columns of image data for the first image.
The method according to claim 4 or 5, wherein the image acquisition device is mounted on a movable platform, the movable platform includes an image processing chip, and the image processing method is executed by the image acquisition device; or

The image processing method is performed by the image processing chip.
The method according to claim 6, wherein the image processing method is performed by the image processing chip, the image processing chip includes a first memory, and the data amount of the partial image data is based on the first memory The maximum storage bandwidth is determined.
8. The method of claim 7, wherein the movable platform includes a second memory, and wherein the second image is stored in the second memory.
The method according to claim 8, wherein the image processing chip comprises an ISP chip, and the second memory comprises a DDR.
The method according to claim 6, wherein after obtaining the first target image or the second target image, the method further comprises:

sending the first target image or the second target image to the control terminal of the movable platform, so that the control terminal displays the first target image or the second target image to the user, wherein the control The terminal is used to control the movement of the movable platform.
The method according to any one of claims 1-10, wherein after obtaining the first target image or the second target image, the method further comprises:

A video is generated and output using the first target image or the second target image.
An image processing device, characterized in that the device includes a processor, a memory, and a computer program stored in the memory for execution by the processor, and when the processor executes the computer program, the following steps are implemented:

During the process that the image acquisition device cyclically performs image acquisition according to N exposures, the first image currently acquired by the image acquisition device is acquired, where N is an integer greater than or equal to 2;

The first image is fused with N-1 frames of second images collected before the first image to obtain a first target image, and the exposures of the N-1 frames of second images are different and different. exposure to the first image;

After N-1 frames of third images collected after the first image are acquired, the first image and the third image are fused to obtain a second target image, the N-1 frames of the third image The exposures are different and different from the exposure of the first image.
The apparatus according to claim 12, wherein the N-1 frames of the second image or the N-1 frames of the third image are adjacent to the first image.
The apparatus according to claim 12 or 13, wherein the N exposure amounts are obtained by adjusting the exposure duration and/or aperture size of the image acquisition device.
The device according to any one of claims 12-14, wherein the processor is configured to fuse the first image with N-1 frames of second images collected before the first image, to obtain When the first target image is used, it is specifically used for:

After the image acquisition device acquires part of the image data of the first image, acquire and store the part of the image data;

The stored part of the image data is fused with the image data at the pixel positions corresponding to the part of the image data in the second image to obtain image data of the target image.
The apparatus of claim 15, wherein the partial image data comprises several lines of image data of the first image; or

Several columns of image data for the first image.
The device according to claim 15 or 16, wherein the image acquisition device is mounted on a movable platform, the movable platform includes an image processing chip, and the image processing device is the image acquisition device; or

The image processing device is the image processing chip.
The device according to claim 17, wherein the image processing device is the image processing chip, the image processing chip includes a first memory, and the data amount of the partial image data is based on the data of the first memory. The maximum storage bandwidth is determined.
19. The apparatus of claim 18, wherein the movable platform includes a second memory and the second image is stored in the second memory.
The apparatus of claim 19, wherein the image processing chip comprises an ISP chip, and the second memory comprises a DDR.
The device according to claim 17, wherein after obtaining the first target image or the second target image, the device is further configured to:

sending the first target image or the second target image to the control terminal of the movable platform, so that the control terminal displays the first target image or the second target image to the user, wherein the control The terminal is used to control the movement of the movable platform.
The device according to any one of claims 12-21, wherein after obtaining the first target image or the second target image, the device is further configured to:

A video is generated and output using the first target image or the second target image.
A movable platform, characterized in that the movable platform includes an image acquisition device and an image processing chip, and the image processing chip includes a processor, a memory, and a computer program stored in the memory for execution by the processor , when the processor executes the computer program, the following steps are implemented:

During the process that the image acquisition device cyclically performs image acquisition according to N exposures, a first image currently acquired by the image acquisition device is acquired, where N is an integer greater than or equal to 2;

The first image is fused with N-1 frames of second images collected before the first image to obtain a first target image, and the exposures of the N-1 frames of second images are different and different. exposure to the first image;

After N-1 frames of third images collected after the first image are acquired, the first image and the third image are fused to obtain a second target image, the N-1 frames of the third image The exposures are different and different from the exposure of the first image.
The movable platform according to claim 23, wherein the image processing chip comprises a first memory, and the processor is configured to acquire, after the image acquisition device acquires part of the image data of the first image, The part of the image data is stored in the first memory, and the part of the image data stored in the first memory is fused with the image data at the pixel position corresponding to the part of the image data in the second image to obtain image data of the target image.
The movable platform according to claim 24, wherein the data amount of the partial image data is determined based on the maximum storage bandwidth of the first memory.
The movable platform according to any one of claims 23-25, wherein the movable platform further comprises a second memory, and the second image is stored in the second memory.
The movable platform according to any one of claims 23-26, wherein the processor is further configured to send the first target image or the second target image to a control terminal of the movable platform, so that the control terminal displays the first target image or the second target image to the user, wherein the control terminal is used to control the movement of the movable platform.