WO2023236162A1 - 相机模组及图像处理方法、装置、终端、电子设备、介质 - Google Patents

相机模组及图像处理方法、装置、终端、电子设备、介质 Download PDF

Info

Publication number
WO2023236162A1
WO2023236162A1 PCT/CN2022/097972 CN2022097972W WO2023236162A1 WO 2023236162 A1 WO2023236162 A1 WO 2023236162A1 CN 2022097972 W CN2022097972 W CN 2022097972W WO 2023236162 A1 WO2023236162 A1 WO 2023236162A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
pixel
lens
area
pixels
Prior art date
Application number
PCT/CN2022/097972
Other languages
English (en)
French (fr)
Inventor
曹军
Original Assignee
北京小米移动软件有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京小米移动软件有限公司 filed Critical 北京小米移动软件有限公司
Priority to PCT/CN2022/097972 priority Critical patent/WO2023236162A1/zh
Priority to CN202280004460.4A priority patent/CN117643047A/zh
Publication of WO2023236162A1 publication Critical patent/WO2023236162A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects

Definitions

  • the present disclosure relates to the technical field of electronic equipment, and in particular, to a camera module and an image processing method, device, terminal, electronic equipment, and medium.
  • a camera module is configured in an electronic device, a lens is arranged on the image sensor in the camera module to perform imaging, and then an image signal processor (Image Signal Processor, ISP) is used to image the lens Enhance images to improve image quality.
  • ISP Image Signal Processor
  • the size of the image sensor changes and develops, its image surface may also increase accordingly.
  • the height of the lens is usually increased to accommodate the change in the size of the image sensor.
  • the height of the arranged lens is easily affected by the size of the image sensor.
  • the layout of the image sensor lens is not flexible enough to take into account the image processing effect, and cannot achieve an effective balance between the lens layout and the image processing effect.
  • Embodiments of the present disclosure propose a camera module, an image processing method, a device, a terminal, an electronic device, and a medium, which can be applied in the field of electronic equipment technology.
  • image processing is performed to generate the to-be-used images.
  • it can effectively retain image details, ensure the quality of image generation, and improve the image processing effect.
  • an embodiment of the present disclosure provides a camera module including an image sensor and at least two lenses;
  • Each lens corresponds to a different imaging area of the image sensor, and the central pixels of different imaging areas are not exactly the same;
  • the center pixel is the pixel in the imaging area that corresponds to the center of the field of view of the lens corresponding to the imaging area;
  • the types of central pixels of different imaging areas are equal to the types of pixels in the image sensor.
  • the first central pixel corresponding to the imaging area of the first lens is different from the corresponding first lens imaging area.
  • the arrangement of the second central pixel of the area or the adjacent pixels adjacent to the first central pixel is different from the arrangement of adjacent pixels adjacent to the second central pixel.
  • the field of view center of the third lens corresponding to the first imaging area and the fourth lens corresponding to the second imaging area The distance between the centers of the field of view of the lenses is less than or equal to the sum of the half length of the third lens and the half length of the fourth lens;
  • the half-side length of the third lens is half the distance of the farthest point in the orthographic projection formed by the third lens
  • the half-side length of the fourth lens is half the distance of the farthest point in the orthographic projection formed by the fourth lens.
  • embodiments of the present disclosure provide an image processing method, including:
  • each regional image corresponds to a different imaging area of the image sensor.
  • the central pixels of different imaging areas are not exactly the same.
  • the central pixel is the center of the field of view of the lens between the pixels of the imaging area. corresponding pixel;
  • generating a target image to be output based on multiple area images includes:
  • the target image is obtained based on the aligned image.
  • multiple area images are aligned to obtain an aligned image, including:
  • the plurality of area images are cropped, and for the third area image and the fourth area image that are adjacent in any second direction and share a side, the first central pixel in the cropping area of the third area image is relative to the fourth area image.
  • the second center pixel within the cropped area of the area image has a pixel offset;
  • the pixel offset is that, in the cropping area of the third area image, the pixel corresponding to the second center pixel position is located at a position that moves from the first center pixel along the second direction to a pixel of the same type as the second center pixel.
  • alignment processing is performed on the cropped images to obtain aligned images, including:
  • the central pixel of the regional image is located at the center of the cropped area of the regional image, or the central pixel of the regional image is closer to the region than the non-central pixel of the regional image.
  • the non-central pixels of the regional image are the pixels in the regional image except the central pixel.
  • obtaining the target image based on the aligned image includes:
  • Fusion of multiple frames of images to be fused to obtain the target image Fusion of multiple frames of images to be fused to obtain the target image.
  • multiple pixels are combined according to pixel positions to obtain an image to be fused, including:
  • the combined image is upsampled according to the semantic features of the image to obtain the image to be fused.
  • an image processing device including:
  • the acquisition module is used to acquire the original image, where the original image includes multiple regional images.
  • Each regional image corresponds to a different imaging area of the image sensor.
  • the central pixels of different imaging areas are not exactly the same.
  • the central pixel is the same as the pixel in the imaging area.
  • the pixel corresponding to the center of the lens' field of view;
  • a generation module is used to generate a target image to be output based on multiple area images.
  • the generation module includes:
  • the first processing submodule is used to perform alignment processing on multiple regional images to obtain aligned images; wherein, the alignment processing is used to make the pixel types corresponding to the image positions in each region the same;
  • the second processing submodule is used to obtain the target image based on the aligned image.
  • the first processing sub-module is specifically used for:
  • the plurality of area images are cropped, and for the third area image and the fourth area image that are adjacent in any second direction and share a side, the first central pixel in the cropping area of the third area image is relative to the fourth area image.
  • the second center pixel within the cropped area of the area image has a pixel offset;
  • the pixel offset is that, in the cropping area of the third area image, the pixel corresponding to the second center pixel position is located at a position that moves from the first center pixel along the second direction to a pixel of the same type as the second center pixel.
  • the first processing sub-module is specifically used for:
  • the central pixel of the regional image is located at the center of the cropped area of the regional image, or the central pixel of the regional image is closer to the region than the non-central pixel of the regional image.
  • the non-central pixels of the regional image are the pixels in the regional image except the central pixel.
  • the second processing sub-module is specifically used for:
  • Fusion of multiple frames of images to be fused to obtain the target image Fusion of multiple frames of images to be fused to obtain the target image.
  • the second processing sub-module is specifically used for:
  • the combined image is upsampled according to the semantic features of the image to obtain the image to be fused.
  • an embodiment of the present disclosure provides a terminal, including: the camera module proposed in the embodiment of the first aspect.
  • an electronic device including:
  • a memory communicatively connected to at least one processor; wherein,
  • the memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor, so that at least one processor can execute the image processing method proposed in the foregoing embodiment of the second aspect.
  • an embodiment of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, the computer instructions being used to cause the computer to execute the image processing method proposed in the second embodiment of the present disclosure.
  • an embodiment of the present disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the image processing method proposed in the embodiment of the second aspect of the present disclosure.
  • the camera module and image processing method, device, terminal, electronic device, storage medium, computer program, and computer program product provided by the embodiments of the present disclosure can achieve the following technical effects:
  • each regional image corresponds to a different imaging area of the image sensor, the central pixels of different imaging areas are not exactly the same, and the central pixel is the field of view of the pixels in the imaging area and the lens
  • the pixel corresponding to the center processes the target image to be output based on multiple regional images.
  • image processing is performed on the regional images captured in different imaging areas of the image sensor to generate the target image to be output, the image details can be effectively retained and the image guaranteed. Generate quality and improve image processing effects.
  • Figure 1 is a schematic structural diagram of a camera module proposed by an embodiment of the present disclosure
  • Figure 2 is a schematic diagram of lens layout in an embodiment of the present disclosure
  • Figure 3 is a schematic diagram of a regional image in this disclosure.
  • Figure 4 is a schematic flowchart of an image processing method proposed by an embodiment of the present disclosure
  • Figure 5 is a schematic diagram of target image generation proposed by another embodiment of the present disclosure.
  • Figure 6 is a schematic flowchart of an image processing method proposed by another embodiment of the present disclosure.
  • Figure 7 is a schematic flowchart of an image processing method proposed by another embodiment of the present disclosure.
  • Figure 8 is a schematic diagram of pixel offset proposed by another embodiment of the present disclosure.
  • Figure 9 is a schematic flowchart of an image processing method proposed by another embodiment of the present disclosure.
  • Figure 10 is a schematic diagram of image alignment in an embodiment of the present disclosure.
  • Figure 11 is a schematic diagram of an image fusion method in an embodiment of the present disclosure.
  • Figure 12 is a comparison chart of pixel offset processing results proposed by another embodiment of the present disclosure.
  • Figure 13 is a schematic structural diagram of an image processing device according to an embodiment of the present disclosure.
  • Figure 14 is a schematic structural diagram of an image processing device according to another embodiment of the present disclosure.
  • Figure 15 is a schematic structural diagram of a terminal proposed by an embodiment of the present disclosure.
  • Figure 16 illustrates a block diagram of an exemplary electronic device suitable for implementing embodiments of the present disclosure.
  • first, second, third, etc. may be used to describe various information in the embodiments of the present disclosure, the information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other.
  • first information may also be called second information, and similarly, the second information may also be called first information.
  • the words "if” and “if” as used herein may be interpreted as “when” or “when” or “in response to determining.”
  • the imaging area is the area in the image sensor used to identify the visible light captured by the lens and perform imaging.
  • the center pixel is the pixel in the imaging area that corresponds to the center of the field of view of the lens corresponding to the imaging area.
  • FIG. 1 is a schematic structural diagram of a camera module according to an embodiment of the present disclosure.
  • the camera module 10 includes an image sensor 101 and at least two lenses 102. Each lens 102 corresponds to a different imaging area of the image sensor 101, and the central pixels of different imaging areas are not exactly the same.
  • the embodiment of the present disclosure supports the arrangement of multiple lenses 102 in one image sensor 101.
  • the number of lenses 102 is at least two, and there is no limit to this.
  • the corresponding arrangement of the multiple lenses 102 in the image sensor 101 can be preset, and the multiple lenses 102 can be arranged in the image sensor 101 according to the predetermined lens arrangement, so that each lens 102 corresponds to the image sensor. 101 different imaging areas.
  • the arrangement can be a special-shaped cutting arrangement, where special-shaped cutting is a way of cutting lenses.
  • special-shaped cutting is a way of cutting lenses.
  • the distance between some lenses in the lens array can be effectively shortened, so that The multiple lenses 102 can be arranged as close as possible.
  • they can also be arranged in a corresponding arrangement according to the size and shape of the image sensor 101, and there is no limit to this.
  • Figure 2 is a schematic diagram of the lens layout method in the embodiment of the present disclosure.
  • Several lenses 102 can be laid out for one image sensor 101.
  • four lenses 102 are laid out for the image sensor 101.
  • Lens 102 is used as an example without limitation.
  • the imaging area is an area in the image sensor 101 used to identify the ambient light information captured by the lens 102 and perform imaging.
  • the area of the image sensor 101 corresponding to the lens 102 can be used as an imaging area. area, there are no restrictions on this.
  • the information about the ambient light may specifically include, for example, the light intensity and wavelength information of the ambient light, which is not limited.
  • the central pixel is a pixel in the imaging area that corresponds to the center of the field of view of the lens 102 corresponding to the imaging area.
  • the center position of the lens 102 (the center position can be used as the field of view of the lens 102)
  • the pixel of the imaging area corresponding to the center of the field) is used as the center pixel.
  • the center pixel can also be flexibly defined according to actual shooting needs, and there is no restriction on this.
  • the field of view is the field of view (Field Of View, FOV) that the lens 102 in the camera module 10 can perceive the ambient light
  • the center of the field of view is the center point of the field of view that the lens 102 can perceive.
  • FOV Field Of View
  • the center point of the area image captured by a single lens 102 can be used as the center of the field of view corresponding to the lens, and the center of the field of view has a corresponding center pixel in the imaging area, and there is no limit to this.
  • the image formed by the lens 102 based on the ambient light information captured by the corresponding imaging pixels may be called a regional image, as shown in FIG. 3 , which is a schematic diagram of the regional image in this disclosure.
  • the regional image may be, for example, a RAW format image without any processing that is collected based on the corresponding imaging area through an image sensor of an electronic device, and this is not limited.
  • the RAW format image is an area image in which the image sensor converts the light source signal captured based on the corresponding imaging area into a digital signal.
  • RAW format images record the original information of the digital camera sensor, and also record some metadata generated by the camera, such as sensitivity settings, shutter speed, aperture value, white balance, etc.
  • the center of the field of view corresponding to the lens can be determined according to the midpoint position of the regional image, and the center pixel in the corresponding imaging area of the image sensor can be determined according to the center of the field of view.
  • the central pixels of different imaging areas are not exactly the same, and the types of central pixels of different imaging areas can be multiple. That is to say, the central pixels of different imaging areas have corresponding types.
  • the types of the central pixels can be Various, no restrictions on this.
  • the types of central pixels of different imaging areas are equal to the types of pixels in the image sensor.
  • the type of pixels can be specifically, for example, the corresponding types in the red, green, blue (RGB) color system, that is, the pixels can be divided into red (Red, R) pixels, green (Green, G) pixels, Blue (Blue, B) pixels, etc., or the pixels can also be classified according to brightness, chrominance and density into brightness (Luminance, Y) pixels, chrominance (Chrominance, U) pixels and density (Chroma, V) pixels, there is no limit on this.
  • the pixels in the image sensor 101 can be arranged according to certain arrangement rules, such as in a Bayer array. As shown in FIG. 3 , four lenses 102 are configured for the image sensor 101 as an example.
  • the image sensor 101 The arrangement of the medium pixels is Red Green Green Blue (RGGB).
  • RGGB Red Green Green Blue
  • the four lenses 102 capture regional images and provide them to the image sensor 101, the centers of the fields of view of different lenses are respectively aligned with different types of pixels in the imaging area.
  • the center of the field of view of lens 1 is aligned with the red pixel
  • the center of the field of view of lens 2 is aligned with the green pixel
  • the center of the field of view of lens 3 is aligned with the green pixel that is different from lens 2
  • the center of the field of view of lens 4 is aligned with the green pixel.
  • the center of the field is aligned with the blue pixel. Therefore, the types of central pixels in different imaging areas corresponding to each lens are equal to the types of pixels in the image sensor, and the central pixels in different imaging areas are not exactly the same to ensure that each type of pixel in the image sensor has a corresponding type. center pixel, thereby effectively improving the imaging effect of the camera module.
  • the first central pixel corresponding to the imaging area of the first lens is different from the corresponding first lens pixel of the second lens.
  • the arrangement of the second central pixel of the imaging area or the adjacent pixels adjacent to the first central pixel is different from the arrangement of adjacent pixels adjacent to the second central pixel.
  • the first direction is any one of horizontal, vertical, or diagonal directions, and the first lens and the second lens are two adjacent lenses in any first direction.
  • the first lens is lens 1
  • the second lens is lens 2
  • the first lens and the second lens are two horizontally adjacent lenses
  • the first lens is lens 1
  • the second lens is lens 3
  • the first lens and the second lens are vertically adjacent lenses.
  • the first lens is lens 1 and the second lens is lens 4
  • the first lens and the second lens are in diagonal directions. Two adjacent shots.
  • the adjacent pixel arrangement is centered on a certain pixel, and the arrangement of other pixels adjacent to the pixel can be called the adjacent pixel arrangement.
  • the first lens has a corresponding first lens imaging area in the image sensor 101, the first lens imaging area has a first central pixel, and the second lens has a corresponding first lens in the image sensor 101.
  • the second lens imaging area has a second central pixel, and the first central pixel is different from the second central pixel.
  • the arrangement of adjacent pixels corresponding to the first central pixel is different from the arrangement of adjacent pixels corresponding to the second central pixel, which is not limited.
  • the pixels are arranged in red, green, green, blue (RGGB).
  • the first lens is lens 1
  • the second lens is lens 2
  • the pixel is a red pixel (center pixel of lens 1)
  • the second center pixel is a green pixel (center pixel of lens 2).
  • the first center pixel is different from the second center pixel.
  • the first lens is lens 2
  • the second lens is lens 2. 3.
  • the first center pixel is one of the green pixels (center pixel of lens 2)
  • the second center pixel is another green pixel (center pixel of lens 3).
  • the arrangement of adjacent pixels of the first center pixel is the same as that of the second center pixel. Neighboring pixels of pixels are arranged differently.
  • the field of view center of the third lens corresponding to the first imaging area and the fourth lens corresponding to the second imaging area is less than or equal to the sum of the half length of the third lens and the half length of the fourth lens.
  • the half-side length of the third lens is half the distance of the farthest point in the orthographic projection figure formed by the third lens
  • the half-side length of the fourth lens is half the distance of the farthest point in the orthographic projection figure formed by the fourth lens, as shown in Figure 2
  • the half length of the third lens and the half length of the fourth lens are as follows As shown in Figure 2.
  • the center of the field of view of the third lens is the position corresponding to the center pixel of the third lens
  • the center of the field of view of the fourth lens is the position corresponding to the center pixel of the fourth lens
  • Figure 3 shows that when the first imaging area and the second imaging area are adjacent and share a side, the pixels corresponding to the center of the field of view of the third lens in the first imaging area and the pixels corresponding to the center of the field of view of the fourth lens in the second imaging area are Marked with triangle symbols respectively, combining Figure 2 and Figure 3, the distance between the center of the field of view of the third lens in the first imaging area and the center of the field of view of the fourth lens in the second imaging area is less than or equal to half the length of the third lens and the half length of the fourth lens, that is to say, in this embodiment, by setting the distance between the center of the field of view of the third lens in the first imaging area and the center of the field of view of the fourth lens in the second imaging area to be less than or
  • each lens corresponds to a different imaging area of the image sensor.
  • the central pixels of different imaging areas are not exactly the same.
  • the types of central pixels of different imaging areas are equal to the types of pixels in the image sensor.
  • the first central pixel corresponding to the imaging area of the first lens is different from the second central pixel corresponding to the imaging area of the second lens, or is different from the first central pixel corresponding to the imaging area of the second lens.
  • the arrangement of adjacent pixels adjacent to the central pixel is different from the arrangement of adjacent pixels adjacent to the second central pixel.
  • the corresponding first The distance between the field of view center of the third lens in the imaging area and the field of view center of the fourth lens corresponding to the second imaging area is less than or equal to the sum of the half side length of the third lens and the half side length of the fourth lens, thereby enabling multiple The lenses are properly arranged in the image sensor, taking into account both the image output effect and the image sensor utilization, so that image details can be effectively retained and the quality of image generation can be guaranteed.
  • FIG. 4 is a schematic flowchart of an image processing method proposed by an embodiment of the present disclosure.
  • the execution subject of the image processing method in this embodiment is an image processing device.
  • the device can be implemented by software and/or hardware.
  • the device can be configured in an electronic device.
  • the electronic device can be a mobile phone. , tablet computers, personal digital assistants, wearable devices and other hardware devices with various operating systems and imaging devices, there are no restrictions on this.
  • the signals and data related to image processing in the embodiments of the present disclosure are all obtained after authorization from relevant users.
  • the acquisition process complies with relevant laws and regulations and does not violate public order and good customs.
  • the image processing method includes:
  • S401 Obtain the original image, where the original image includes multiple regional images. Each regional image corresponds to a different imaging area of the image sensor. The central pixels of different imaging areas are not exactly the same. The central pixel is the pixel in the imaging area that is the same as the lens's visual angle. The pixel corresponding to the center of the field.
  • the image sensor can obtain the light intensity and wavelength information captured by each imaging pixel in the imaging area, and provide an image of the area that can be processed by the image signal processor ISP.
  • the original image is obtained, where the original image is composed of regional images captured by each lens.
  • One lens corresponds to an imaging area of the image sensor, and the imaging pixels in the imaging area capture information about the ambient light transmitted by the corresponding lens, so as to
  • the regional image can be provided to the image signal processor ISP, and the image signal processor ISP triggers subsequent steps.
  • S402 Generate a target image to be output based on multiple area images.
  • regional images captured by different lenses can be processed to generate a target image to be output.
  • the image obtained by performing corresponding processing on multiple area images can be called a target image. Therefore, when processing with reference to the center pixels of different imaging areas Multiple regional images can make the target image carry personalized camera information of the regional images captured by each lens, so that the target image has higher resolution, picture quality, and image details, thereby realizing the optimization of lens layout and image generation effect. Effective balance.
  • a deep learning image processing model can be set to use the image processing model to generate a target image to be output based on multiple area images, or a custom image processing algorithm can also be used to generate a target image based on multiple area images.
  • the target image to be output there is no restriction on this.
  • Figure 5 is a schematic diagram of target image generation proposed by another embodiment of the present disclosure.
  • the four lenses respectively acquire regional images of the same scene and are enhanced by algorithms. Processed with super-resolution algorithm to obtain high-quality target images.
  • the original image is obtained, where the original image includes multiple regional images.
  • Each regional image corresponds to a different imaging area of the image sensor.
  • the central pixels of different imaging areas are not exactly the same.
  • the central pixel is one of the pixels in the imaging area.
  • the pixels corresponding to the center of the field of view of the lens process the target image to be output based on multiple regional images.
  • FIG. 6 is a schematic flowchart of an image processing method proposed by another embodiment of the present disclosure.
  • the image processing method includes:
  • S601 Obtain the original image, where the original image includes multiple regional images. Each regional image corresponds to a different imaging area of the image sensor. The central pixels of different imaging areas are not exactly the same. The central pixel is the pixel in the imaging area that is the same as the lens's visual angle. The pixel corresponding to the center of the field.
  • S602 Perform alignment processing on multiple regional images to obtain aligned images; wherein, the alignment processing is used to make the pixel types corresponding to the positions of each regional image the same.
  • the aligned image is an aligned image obtained by aligning regional images captured by different lenses in the same scene according to the position of the central pixel.
  • the regional images captured by the four lenses have different positions of the center pixels.
  • the arrangement of the pixels in the image sensor can also be determined, based on The arrangement of pixels is used to process the regional image captured by each lens, and the regional image is aligned and processed into an image consistent with the arrangement of pixels. As an aligned image, there is no limit to this.
  • the image processing method based on artificial intelligence can be used to process the regional image to perform alignment processing on the regional image to obtain the aligned image.
  • the regional image can also be cropped by referring to the pixel type combined with image cropping method to obtain the aligned image. , or you can also use any other possible implementation method to align the regional images to obtain the aligned images, and there is no limit to this.
  • the pixels corresponding to each aligned image position are of the same type, and the adjacent pixels around the pixels corresponding to each aligned image position are arranged in the same manner, so as to facilitate subsequent processing of the aligned image.
  • multiple frames of aligned images can be fused to obtain a target image, or image information extraction technology can be used to extract information from the image to obtain the target image.
  • image information extraction technology can be used to extract information from the image to obtain the target image.
  • Generate a target image or you can also use image rendering, image enhancement and other technologies to process the aligned images to obtain the target image, without any restrictions.
  • a corresponding image processing algorithm such as a super-resolution algorithm
  • the image processing network of deep learning reconstructs a high-quality target image by extracting the features of multi-frame aligned images, or it can also use any other possible implementation method to process multi-frame aligned images to obtain the target image. There is no limit to this. .
  • the regional images captured in different imaging areas of the image sensor are processed to generate the target image to be output, the image details can be effectively retained, the image processing quality is ensured, and the image processing effect is improved.
  • Aligning multiple regional images to obtain an aligned image, and obtaining a target image based on the aligned image can effectively improve the processing efficiency of regional images, achieve better image processing effects, and effectively improve the resolution and picture quality of the target image.
  • FIG. 7 is a schematic flowchart of an image processing method proposed by another embodiment of the present disclosure.
  • the image processing method includes:
  • S701 Obtain the original image, where the original image includes multiple regional images. Each regional image corresponds to a different imaging area of the image sensor. The central pixels of different imaging areas are not exactly the same. The central pixel is the pixel in the imaging area that is the same as the lens. The pixel corresponding to the center of the field.
  • S702 Crop multiple area images, and for the third area image and the fourth area image that are adjacent in any second direction and share a side, the first center pixel in the cropping area of the third area image is relative to The second center pixel within the cropped area of the fourth area image has a pixel offset.
  • the pixel offset is that, in the cropping area of the third area image, the pixel corresponding to the second center pixel position is located at a position that moves from the first center pixel along the second direction to a pixel of the same type as the second center pixel.
  • the pixel offset distance can be set to the length of one pixel, or other pixel offset distances of any other length (such as half a pixel in length, two pixels in length) can also be set according to actual needs. etc.), there is no restriction on this.
  • the second direction is either horizontal or vertical, that is, the second direction can be horizontal or vertical, and there is no limit to this.
  • the central pixel of the regional image is located at the center of the cropping area of the regional image, or the central pixel of the regional image is closer to the non-central pixel of the regional image.
  • the center position of the area, where the non-center pixels of the area image are pixels other than the central pixel in the area image. Since the position of the center pixel is set, the center pixel corresponding to the cropping area can be accurately characterized, making the cropping process as easy as possible.
  • the central pixel is located at the center of the cropping area, effectively ensuring the efficiency and effect of determining the center pixel in the cropping area, and effectively enhancing the effect of image processing.
  • the central pixel of the regional image can be set to be located at the center of the cropped area of the regional image, or the central pixel of the regional image of the cropped area can be set to be located closer to the center of the region relative to the non-central pixels of the regional image.
  • the pixel at the third row and third column in the cropping area can be used as the center pixel, and there is no restriction on this.
  • Figure 8 is a schematic diagram of pixel offset proposed by another embodiment of the present disclosure.
  • the area in the dotted circle is the cropping area.
  • a central pixel is the pixel corresponding to the center of the first area image field of view
  • the second center pixel is the pixel corresponding to the center of the second area image field of view. It can be intuitively seen that in the cropping area of the third area image and the third area image, The pixel corresponding to the second center pixel is located one pixel laterally moved to the left from the pixel corresponding to the first center pixel, that is, the pixel offset.
  • the first center pixel also has a pixel offset relative to the second center pixel in the cropping area of the fourth area image. Therefore, the center pixel is determined, and the pixel offset is determined based on the center pixel to perform processing on multiple area images. Accurate cropping ensures that the image obtained after cropping has a pixel offset effect.
  • S703 Perform alignment processing on the cropped images to obtain aligned images.
  • the images obtained after cropping can be aligned according to pixels corresponding to the same position to obtain an aligned image.
  • image processing algorithms can be used to perform cropping of regional images and alignment of images obtained after cropping, or an image processing big data model can also be set to perform cropping based on big data model technology. Align the images to obtain an aligned image, or you can also use feature extraction technology, image recognition technology, and other technologies to align the cropped images to obtain an aligned image, and there is no limit to this.
  • the regional images captured in different imaging areas of the image sensor are processed to generate the target image to be output, the image details can be effectively retained, the image processing quality is ensured, and the image processing effect is improved.
  • Aligning multiple regional images to obtain an aligned image, and obtaining a target image based on the aligned image can effectively improve the processing efficiency of regional images, achieve better image processing effects, and effectively improve the resolution and picture quality of the target image.
  • the image can have better picture quality in the color dimension, effectively improving the color performance of the target image, thereby effectively improving the resolution and picture quality of the target image.
  • FIG. 9 is a schematic flowchart of an image processing method proposed by another embodiment of the present disclosure.
  • the image processing method includes:
  • S901 Obtain the original image, where the original image includes multiple regional images. Each regional image corresponds to a different imaging area of the image sensor. The central pixels of different imaging areas are not exactly the same. The central pixel is the pixel in the imaging area that is the same as the lens's visual angle. The pixel corresponding to the center of the field.
  • S902 Crop multiple area images, and for the third area image and the fourth area image that are adjacent in any second direction and share a side, the first central pixel in the cropping area of the third area image is relative to The second center pixel within the cropped area of the fourth area image has a pixel offset.
  • S903 Generate an optical flow feature map corresponding to the cropped image.
  • the image obtained after cropping can be processed to generate an optical flow feature map.
  • images obtained after different cropping can have the same or different optical flow characteristics, and corresponding optical flow feature maps can be generated based on the optical flow characteristics corresponding to the images obtained after cropping.
  • Embodiments of the present disclosure can build an optical flow network, process the cropped image through the optical flow network, and obtain an optical flow feature map corresponding to the cropped image.
  • the optical flow network of deep learning can be used to process the cropped image to generate an optical flow feature map corresponding to the cropped image.
  • the cropped image can be The image obtained after the cropping is processed by downsampling. If the pixels of the cropped image are arranged in Red Green Green Blue (RGGB), you can discard a green (G) pixel to turn it into a downsampled red pixel. An image with Red Green Blue (RGB) pixels arranged to facilitate the generation of optical flow feature maps. Of course, you can also use any other possible method to process the cropped image to generate an optical flow feature map. This is not done. limit.
  • RGGB Red Green Green Blue
  • S904 Perform upsampling processing on the optical flow feature map to obtain an upsampled image.
  • the image obtained by upsampling the optical flow feature map can be called an upsampled image.
  • the upsampling image can be generated using bilinear interpolation upsampling, or it can also be processed using methods such as nearest neighbor value upsampling and bicubic interpolation upsampling.
  • Optical flow feature maps are used to generate upsampled images without any restrictions.
  • S905 Perform alignment processing on the upsampled image to obtain an aligned image.
  • multiple frames of upsampled images obtained by upsampling may be aligned to obtain aligned images.
  • a region image is selected as a reference region image, other region images are cropped based on the reference region image, and the cropped image is Perform downsampling processing, and input the processed image into the optical flow network to generate an optical flow feature map.
  • the optical flow feature map undergoes bilinear interpolation upsampling processing to generate an upsampled image, and the upsampled images are aligned to Get aligned images.
  • S906 Extract multiple pixels of the same kind from the aligned image, where the pixels have corresponding pixel positions in the aligned image.
  • a method of pixel recognition can be used to extract multiple pixels of the same type from the aligned image and record the corresponding pixel positions.
  • the pixel positions corresponding to the multiple pixels of the same type can also be determined in advance, according to The pixel position extracts multiple pixels of the same kind from the aligned image, or any other possible implementation method may be used to extract multiple pixels of the same kind from the aligned image, without limitation.
  • the pixel arrangement is RGGB
  • S907 Combine multiple pixels according to their positions to obtain the image to be fused.
  • the image obtained by combining multiple pixels can be called an image to be fused, and the image to be fused can be used to fuse into a target image.
  • multiple pixels can be combined according to the pixel position to obtain a combined image, the image semantic features corresponding to the combined image are determined, and the combined image is upsampled according to the image semantic features to obtain the image to be fused.
  • the image to be fused is obtained by combining images based on image semantic features, it can effectively reduce the impact of picture noise on the image to be fused, thereby improving the image quality of the image to be fused and enhancing the image processing effect.
  • the features of the image semantic dimension of the combined image can be called image semantic features.
  • the image semantic features can be the texture, color and other features of the combined image, or they can also be the depth features corresponding to the combined image. This is not the case. Make restrictions.
  • the corresponding image feature extraction network can be used to determine the depth features as the image semantic features corresponding to the combined image, or the image recognition method can be used to perform semantic recognition on the combined image to determine the image semantic features.
  • the image recognition method can be used to perform semantic recognition on the combined image to determine the image semantic features.
  • any other possible implementation method may be used to determine the image semantic features corresponding to the combined image, and there is no limit to this.
  • the outline features representing item A and item B can be extracted through image processing or other methods, and the outline features can be used as the image of the combined image.
  • Semantic features, or the deep features corresponding to the combined image can be extracted as image semantic features through a deep learning network (such as a "U"-structured semantic segmentation algorithm network, etc.).
  • the fusion network of deep learning can be used to extract the image semantic features corresponding to the combined image, and the pixel shuffle technology can be used to combine the image semantic features to separately upsample the combined image to determine the image to be fused.
  • an image fusion processing model can also be used, based on which the combined image is upsampled to obtain the image to be fused, or any other possible implementation method can be used to upsample the combined image to obtain the image to be fused. , there is no restriction on this.
  • a feature extraction system can also be built, and corresponding algorithm models can be used to extract image semantic features corresponding to the combined image.
  • corresponding algorithm models can be used to extract image semantic features corresponding to the combined image.
  • any other possible implementation method can also be used to extract image semantic features corresponding to the combined image. , there is no restriction on this.
  • S908 Fusion of multiple frames of images to be fused to obtain the target image.
  • image processing methods can be used to directly fuse the multiple frames of images to be fused, or a deep learning network can be used to fuse multiple frames of images to be fused, or other methods can be used. Fuse multiple frames of images to be fused in any possible way to obtain the target image, and there is no limit to this.
  • Figure 11 is a schematic diagram of the image fusion method in the embodiment of the present disclosure.
  • the images can be combined according to the same type of pixel composition, and then, Using a fusion network based on deep learning, the image semantic features corresponding to the combined image are extracted through the network model (such as semantic segmentation algorithm network, deep learning network, etc.), and then the image semantic features are output using pixel shuffle technology and upsampling processing. , and use a convolutional layer to reconstruct high-quality and high-resolution target images.
  • the network model such as semantic segmentation algorithm network, deep learning network, etc.
  • the image dimension of the combined image obtained after processing is (H, W, 16).
  • the image semantics is extracted through the network model Features, the generated dimensions are (H, W, 2s ⁇ 2s ⁇ 512), where “s” is the number of extracted image semantic features.
  • the image semantic features are reorganized into pixels, and the dimensions of the reorganized image to be fused are obtained. is (2sH, 2sW, 512), and then uses a convolution layer to convolve the reorganized image to reconstruct a high-quality and high-resolution target image.
  • Figure 12 is a comparison chart of pixel offset processing results proposed by another embodiment of the present disclosure. From the comparison of Figure 12, it can be concluded that the target image quality and resolution obtained using the pixel offset technology Higher, the image processing effect is better.
  • the regional images captured in different imaging areas of the image sensor are processed to generate the target image to be output, the image details can be effectively retained, the image processing quality is ensured, and the image processing effect is improved.
  • Aligning multiple regional images to obtain an aligned image, and obtaining a target image based on the aligned image can effectively improve the processing efficiency of regional images, achieve better image processing effects, and effectively improve the resolution and picture quality of the target image.
  • the image can have better picture quality in the color dimension, effectively improving the color performance of the target image, thereby effectively improving the resolution and picture quality of the target image. Since the cropped image is subjected to optical flow feature extraction and generation, as well as upsampling processing, and then image alignment is performed, the image details can be effectively retained during the process of aligning the cropped image, and avoids the need for alignment processing. The image details are lost in the process, and it can effectively retain the image details while reducing the processing resource consumption required for image alignment processing, achieving a balance between the alignment processing effect and the alignment processing efficiency. Since the image to be fused is obtained by processing and combining images based on the semantic features of the image, it can effectively reduce the impact of picture noise on the image to be fused, thereby improving the image quality of the image to be fused and enhancing the image processing
  • FIG. 13 is a schematic structural diagram of an image processing device according to an embodiment of the present disclosure.
  • the image processing device 130 includes:
  • the acquisition module 1301 is used to acquire an original image, where the original image includes multiple regional images. Each regional image corresponds to a different imaging area of the image sensor. The central pixels of different imaging areas are not exactly the same. The central pixel is among the pixels in the imaging area. The pixel corresponding to the center of the lens' field of view;
  • Generating module 1302 configured to generate a target image to be output based on multiple area images.
  • the generation module 1302 includes:
  • the first processing sub-module 13021 is used to perform alignment processing on multiple regional images to obtain an aligned image; wherein the alignment processing is used to make the pixel types corresponding to the positions of each regional image the same;
  • the second processing sub-module 13022 is used to obtain the target image based on the aligned image.
  • the first processing sub-module 13021 is specifically used to:
  • the plurality of area images are cropped, and for the third area image and the fourth area image that are adjacent in any second direction and share a side, the first central pixel in the cropping area of the third area image is relative to the fourth area image.
  • the second center pixel within the cropped area of the area image has a pixel offset;
  • the pixel offset is that, in the cropping area of the third area image, the pixel corresponding to the second center pixel position is located at a position that moves from the first center pixel along the second direction to a pixel of the same type as the second center pixel.
  • the first processing sub-module 13021 is specifically used to:
  • the central pixel of the regional image is located at the center of the cropping area of the regional image, or the central pixel of the regional image is relative to Non-center pixels are closer to the center of the area;
  • the non-central pixels of the regional image are the pixels in the regional image except the central pixel.
  • the second processing sub-module 13022 is specifically used to:
  • Fusion of multiple frames of images to be fused to obtain the target image Fusion of multiple frames of images to be fused to obtain the target image.
  • the second processing sub-module 13022 is specifically used to:
  • the combined image is upsampled according to the semantic features of the image to obtain the image to be fused.
  • the present disclosure also provides an image processing device. Since the image processing device provided by the embodiments of the present disclosure is different from the image processing method provided by the above embodiments of FIGS. 4 to 12 The method corresponds to the method, so the implementation of the image processing method is also applicable to the image processing device provided in the embodiment of the present disclosure, and will not be described in detail in the embodiment of the present disclosure.
  • the original image is obtained, where the original image includes multiple regional images.
  • Each regional image corresponds to a different imaging area of the image sensor.
  • the central pixels of different imaging areas are not exactly the same.
  • the central pixel is one of the pixels in the imaging area.
  • the pixels corresponding to the center of the field of view of the lens generate the target image to be output based on multiple regional images.
  • Figure 15 is a schematic structural diagram of a terminal proposed by an embodiment of the present disclosure.
  • the terminal 150 includes a camera module 10 .
  • FIG. 16 illustrates a block diagram of an exemplary electronic device suitable for implementing embodiments of the present disclosure.
  • the electronic device 12 shown in FIG. 16 is only an example and should not bring any limitations to the functions and scope of use of the embodiments of the present disclosure.
  • electronic device 12 is embodied in the form of a general computing device.
  • the components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 17, a system memory 28, and a bus 18 connecting different system components (including the system memory 28 and the processing unit 17).
  • Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics accelerated port, a processor, or a local bus using any of a variety of bus structures.
  • these architectures include but are not limited to Industry Standard Architecture (hereinafter referred to as: ISA) bus, Micro Channel Architecture (Micro Channel Architecture; hereafter referred to as: MAC) bus, enhanced ISA bus, video electronics Standards Association (Video Electronics Standards Association; hereinafter referred to as: VESA) local bus and Peripheral Component Interconnection (hereinafter referred to as: PCI) bus.
  • ISA Industry Standard Architecture
  • MAC Micro Channel Architecture
  • VESA Video Electronics Standards Association
  • PCI Peripheral Component Interconnection
  • Electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 12, including volatile and nonvolatile media, removable and non-removable media.
  • the memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (Random Access Memory; hereinafter referred to as: RAM) 30 and/or cache memory 32.
  • Electronic device 12 may further include other removable/non-removable, volatile/non-volatile computer system storage media.
  • storage system 34 may be used to read and write to non-removable, non-volatile magnetic media (not shown in Figure 16, commonly referred to as a "hard drive").
  • a disk drive for reading and writing a removable non-volatile disk (e.g., a "floppy disk") and a removable non-volatile optical disk (e.g., a compact disk read-only memory) may be provided.
  • Disc Read Only Memory hereinafter referred to as: CD-ROM
  • DVD-ROM Digital Video Disc Read Only Memory
  • each drive may be connected to bus 18 through one or more data media interfaces.
  • Memory 28 may include at least one program product having a set (eg, at least one) of program modules configured to perform the functions of embodiments of the present disclosure.
  • a program/utility 40 having a set of (at least one) program modules 42 may be stored, for example, in memory 28 , each of these examples or some combination may include the implementation of a network environment.
  • Program modules 42 generally perform functions and/or methods in the embodiments described in this disclosure.
  • Electronic device 12 may also communicate with one or more external devices 15 (e.g., keyboard, pointing device, display 24, etc.), may also communicate with one or more devices that enable a user to interact with electronic device 12, and/or with Any device (eg, network card, modem, etc.) that enables the electronic device 12 to communicate with one or more other computing devices. This communication may occur through input/output (I/O) interface 22.
  • the electronic device 12 can also communicate with one or more networks (such as a local area network (Local Area Network; hereinafter referred to as: LAN), a wide area network (Wide Area Network; hereinafter referred to as: WAN)) and/or a public network through the network adapter 20, such as Internet) communications.
  • networks such as a local area network (Local Area Network; hereinafter referred to as: LAN), a wide area network (Wide Area Network; hereinafter referred to as: WAN)
  • a public network such as Internet
  • network adapter 20 communicates with other modules of electronic device 12 via bus 18 .
  • other hardware and/or software modules may be used in conjunction with electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives And data backup storage system, etc.
  • the processing unit 17 executes programs stored in the system memory 28 to perform various functional applications and data processing, such as implementing the image processing method mentioned in the previous embodiment.
  • the present disclosure also provides an electronic device, including: a camera module, a memory, a processor, and a computer program stored in the memory and executable on the processor.
  • the camera module is electrically connected to the processor.
  • the processor executes the computer program, the image processing method of the foregoing embodiments of the present disclosure is implemented.
  • the present disclosure also proposes a non-transitory computer-readable storage medium on which a computer program is stored.
  • the program is executed by a processor, the image processing method as proposed in the previous embodiments of the present disclosure is implemented.
  • the present disclosure also provides a computer program product, which includes a computer program.
  • the computer program When executed by a processor, the computer program implements the image processing method of the foregoing embodiments of the present disclosure.
  • the above embodiments it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof.
  • software it may be implemented in whole or in part in the form of a computer program product.
  • the computer program product includes one or more computer programs.
  • the computer program When the computer program is loaded and executed on a computer, the processes or functions described in accordance with the embodiments of the present disclosure are generated in whole or in part.
  • the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.
  • the computer program may be stored in or transferred from one computer-readable storage medium to another, for example, the computer program may be transferred from a website, computer, server, or data center Transmission to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) means.
  • the computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains one or more available media integrated.
  • the usable media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., high-density digital video discs (DVD)), or semiconductor media (e.g., solid state disks, SSD)) etc.
  • magnetic media e.g., floppy disks, hard disks, magnetic tapes
  • optical media e.g., high-density digital video discs (DVD)
  • DVD digital video discs
  • semiconductor media e.g., solid state disks, SSD
  • At least one in the present disclosure can also be described as one or more, and the plurality can be two, three, four or more, and the present disclosure is not limited.
  • the technical feature is distinguished by “first”, “second”, “third”, “A”, “B”, “C” and “D” etc.
  • the technical features described in “first”, “second”, “third”, “A”, “B”, “C” and “D” are in no particular order or order.
  • each table in this disclosure can be configured or predefined.
  • the values of the information in each table are only examples and can be configured as other values, which is not limited by this disclosure.
  • it is not necessarily required to configure all the correspondences shown in each table.
  • the corresponding relationships shown in some rows may not be configured.
  • appropriate deformation adjustments can be made based on the above table, such as splitting, merging, etc.
  • the names of the parameters shown in the titles of the above tables may also be other names understandable by the communication device, and the values or expressions of the parameters may also be other values or expressions understandable by the communication device.
  • other data structures can also be used, such as arrays, queues, containers, stacks, linear lists, pointers, linked lists, trees, graphs, structures, classes, heaps, hash tables or hash tables. wait.
  • Predefinition in this disclosure may be understood as definition, pre-definition, storage, pre-storage, pre-negotiation, pre-configuration, solidification, or pre-burning.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Studio Devices (AREA)
  • Image Processing (AREA)

Abstract

本公开提出一种相机模组及图像处理方法、装置、终端、电子设备、介质，该相机模组包括图像传感器和至少两个镜头，该方法包括：获取原始图像，其中，原始图像包括多个区域图像，每一区域图像对应图像传感器的不同成像区域，不同成像区域的中心像素不完全相同，中心像素为成像区域的像素中与镜头的视场中心对应的像素，基于多个区域图像生成待输出的目标图像。当针对图像传感器的不同成像区域捕获的区域图像，进行图像处理生成待输出的目标图像时，能够有效地保留图像细节，保障图像处理质量，提升图像处理效果。

Description

相机模组及图像处理方法、装置、终端、电子设备、介质 技术领域
本公开涉及电子设备技术领域，尤其涉及一种相机模组及图像处理方法、装置、终端、电子设备、介质。
背景技术
随着摄像技术的发展，对电子设备拍摄图像的清晰度的需求越来越高。
相关技术中，在电子设备中配置相机模组，在该相机模组中的图像传感器上布局一个镜头，以进行成像，而后，采用图像信号处理器(Image Signal Processor，ISP)对该镜头的成像图像进行增强，以提升图像画质。随着图像传感器尺寸的变化发展，其像面也可能会相应增大，而为了补偿感光像素的损失，通常是增高镜头的高度，以适应图像传感器的尺寸变化。
这种方式下，所布局镜头的高度易于受到图像传感器尺寸的影响，图像传感器的镜头的布局方式不够灵活，不能够兼顾图像处理效果，不能够实现镜头布局方式和图像处理效果的有效平衡。
发明内容
本公开实施例提出一种相机模组及图像处理方法、装置、终端、电子设备、介质，可以应用于电子设备技术领域，当针对图像传感器的不同成像区域捕获的区域图像，进行图像处理生成待输出的目标图像时，能够有效地保留图像细节，保障图像生成质量，提升图像处理效果。
第一方面，本公开实施例提供一种相机模组，包括图像传感器和至少两个镜头；
每一镜头对应图像传感器的不同成像区域，不同成像区域的中心像素不完全相同；
其中，中心像素为成像区域的像素中，与成像区域所对应镜头的视场中心相对应的像素；
在本公开的一些实施例中，不同成像区域的中心像素的种类等于图像传感器中像素的种类。
在本公开的一些实施例中，对至少两个镜头中在任一第一方向上相邻的第一镜头和第二镜头，对应第一镜头成像区域的第一中心像素不同于对应第二镜头成像区域的第二中心像素，或者与第一中心像素相邻的相邻像素排布方式不同于与第二中心像素相邻的相邻像素排布方式。
在本公开的一些实施例中，对任意相邻且共用侧边的第一成像区域和第二成像区域，对应第一成像区域的第三镜头的视场中心与对应第二成像区域的第四镜头的视场中心之间的距离，小于或等于第三镜头半边长与第四镜头半边长的和；
其中，第三镜头半边长为第三镜头形成的正投影图形中最远点距离的一半，第四镜头半边长为第四镜头形成的正投影图形中最远点距离的一半。
第二方面，本公开实施例提供一种图像处理方法，包括：
获取原始图像，其中，原始图像包括多个区域图像，每一区域图像对应图像传感器的不同成像区域，不同成像区域的中心像素不完全相同，中心像素为成像区域的像素中与镜头的视场中心对应的像素；
基于多个区域图像生成待输出的目标图像。
在本公开的一些实施例中，基于多个区域图像生成待输出的目标图像，包括：
对多个区域图像进行对齐处理，得到对齐图像；其中，对齐处理用于使得每一区域图像位置对应处的像素种类相同；
基于对齐图像得到目标图像。
在本公开的一些实施例中，对多个区域图像进行对齐处理，得到对齐图像，包括：
对多个区域图像进行裁剪，且对于在任一第二方向上相邻且共用侧边的第三区域图像和第四区域图像，第三区域图像的裁剪区域内的第一中心像素相对于第四区域图像的裁剪区域内的第二中心像素具有像素偏移；
对裁剪后得到的图像进行对齐处理，得到对齐图像；
其中，像素偏移为，在第三区域图像的裁剪区域内，与第二中心像素位置对应的像素位于从第一中心像素沿着第二方向移动到与第二中心像素同种类像素的位置。
在本公开的一些实施例中，对裁剪后得到的图像进行对齐处理，得到对齐图像，包括：
生成与裁剪后得到的图像对应的光流特征图；
对光流特征图进行上采样处理，以得到上采样图像；
对上采样图像进行对齐处理，以得到对齐图像。
在本公开的一些实施例中，每一区域图像的裁剪区域内，区域图像的中心像素位于区域图像的裁剪区域的中心位置，或者区域图像的中心像素相对于区域图像的非中心像素更接近区域中心位置；
其中，区域图像的非中心像素为区域图像中除了中心像素之外的像素。
在本公开的一些实施例中，基于对齐图像得到目标图像，包括：
从对齐图像中提取相同种类的多个像素，其中，像素在对齐图像中具有对应的像素位置；
根据像素位置组合多个像素，得到待融合图像；
融合多帧待融合图像，以得到目标图像。
在本公开的一些实施例中，根据像素位置组合多个像素，得到待融合图像，包括：
根据像素位置组合多个像素，以得到组合图像；
确定与组合图像对应的图像语义特征；
根据图像语义特征对组合图像进行上采样处理，以得到待融合图像。
第三方面，本公开实施例提供一种图像处理装置，包括：
获取模块，用于获取原始图像，其中，原始图像包括多个区域图像，每一区域图像对应图像传感器的不同成像区域，不同成像区域的中心像素不完全相同，中心像素为成像区域的像素中与镜头的视场中心对应的像素；
生成模块，用于基于多个区域图像生成待输出的目标图像。
在本公开的一些实施例中，生成模块，包括：
第一处理子模块，用于对多个区域图像进行对齐处理，得到对齐图像；其中，对齐处理用于使得每一区域图像位置对应处的像素种类相同；
第二处理子模块，用于基于对齐图像得到目标图像。
在本公开的一些实施例中，第一处理子模块，具体用于：
对多个区域图像进行裁剪，且对于在任一第二方向上相邻且共用侧边的第三区域图像和第四区域图像，第三区域图像的裁剪区域内的第一中心像素相对于第四区域图像的裁剪区域内的第二中心像素具有像素偏移；
对裁剪后得到的图像进行对齐处理，得到对齐图像；
其中，像素偏移为，在第三区域图像的裁剪区域内，与第二中心像素位置对应的像素位于从第一中心像素沿着第二方向移动到与第二中心像素同种类像素的位置。
在本公开的一些实施例中，第一处理子模块，具体用于：
生成与裁剪后得到的图像对应的光流特征图；
对光流特征图进行上采样处理，以得到上采样图像；
对上采样图像进行对齐处理，以得到对齐图像。
在本公开的一些实施例中，每一区域图像的裁剪区域内，区域图像的中心像素位于区域图像的裁剪区域的中心位置，或者区域图像的中心像素相对于区域图像的非中心像素更接近区域中心位置；
其中，区域图像的非中心像素为区域图像中除了中心像素之外的像素。
在本公开的一些实施例中，第二处理子模块，具体用于：
从对齐图像中提取相同种类的多个像素，其中，像素在对齐图像中具有对应的像素位置；
根据像素位置组合多个像素，得到待融合图像；
融合多帧待融合图像，以得到目标图像。
在本公开的一些实施例中，第二处理子模块，具体用于：
根据像素位置组合多个像素，以得到组合图像；
确定与组合图像对应的图像语义特征；
根据图像语义特征对组合图像进行上采样处理，以得到待融合图像。
第四方面，本公开实施例提供一种终端，包括：第一方面实施例提出的相机模组。
第五方面，本公开实施例提供一种电子设备，包括：
相机模组；
至少一个处理器；以及
与至少一个处理器通信连接的存储器；其中，
存储器存储有可被至少一个处理器执行的指令，指令被至少一个处理器执行，以使至少一个处理器能够执行前述第二方面实施例提出的图像处理方法。
第六方面，本公开实施例提供一种存储有计算机指令的非瞬时计算机可读存储介质，所述计算机指令用于使所述计算机执行本公开第二方面实施例提出的图像处理方法。
第七方面，本公开实施例提供一种计算机程序产品，包括计算机程序，所述计算机程序在被处理器执行时实现本公开第二方面实施例提出的图像处理方法。
综上所述，在本公开实施例提供的相机模组及图像处理方法、装置、终端、电子设备、存储介质、计算机程序、计算机程序产品，可以实现以下技术效果：
通过获取原始图像，其中，原始图像包括多个区域图像，每一区域图像对应图像传感器的不同成像区域，不同成像区域的中心像素不完全相同，中心像素为成像区域的像素中与镜头的视场中心对应的像素，基于多个区域图像处理待输出的目标图像，当针对图像传感器的不同成像区域捕获的区域图像，进行图像处理生成待输出的目标图像时，能够有效地保留图像细节，保障图像生成质量，提升图像处理效果。
附图说明
为了更清楚地说明本公开实施例或背景技术中的技术方案，下面将对本公开实施例或背景技术中所需要使用的附图进行说明。
图1是本公开一实施例提出的相机模组的结构示意图；
图2是本公开实施例中镜头布局方式示意图；
图3是本公开中区域图像示意图；
图4是本公开一实施例提出的图像处理方法的流程示意图；
图5是本公开另一实施例提出的目标图像生成示意图；
图6是本公开另一实施例提出的图像处理方法的流程示意图；
图7是本公开另一实施例提出的图像处理方法的流程示意图；
图8是本公开另一实施例提出的像素偏移示意图；
图9是本公开另一实施例提出的图像处理方法的流程示意图；
图10是本公开实施例中图像对齐方式示意图；
图11是本公开实施例中图像融合方式示意图；
图12是本公开另一实施例提出的像素偏移处理结果对比图；
图13是本公开一实施例提出的图像处理装置的结构示意图；
图14是本公开另一实施例提出的图像处理装置的结构示意图；
图15是本公开一实施例提出的终端的结构示意图；
图16示出了适于用来实现本公开实施方式的示例性电子设备的框图。
具体实施方式
这里将详细地对示例性实施例进行说明，其示例表示在附图中。下面的描述涉及附图时，除非另有表示，不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本公开实施例相一致的所有实施方式。相反，它们仅是与如所附权利要求书中所详述的、本公开实施例的一些方面相一致的装置和方法的例子。
在本公开实施例使用的术语是仅仅出于描述特定实施例的目的，而非旨在限制本公开实施例。在本公开实施例和所附权利要求书中所使用的单数形式的“一种”和“该”也旨在包括多数形式，除非上下文清楚地表示其他含义。还应当理解，本文中使用的术语“和/或”是指并包含一个或多个相关联的列出项目的任何或所有可能组合。
应当理解，尽管在本公开实施例可能采用术语第一、第二、第三等来描述各种信息，但这些信息不应限于这些术语。这些术语仅用来将同一类型的信息彼此区分开。例如，在不脱离本公开实施例范围的情况下，第一信息也可以被称为第二信息，类似地，第二信息也可以被称为第一信息。取决于语境，如在此所使用的词语“如果”及“若”可以被解释成为“在……时”或“当……时”或“响应于确定”。
为了便于理解，首先介绍本公开涉及的术语。
1、成像区域
成像区域，是图像传感器中用于识别镜头捕获的可见光并进行成像的区域。
2、中心像素
中心像素，是成像区域的像素中，与成像区域所对应镜头的视场中心相对应的像素。
图1是本公开一实施例提出的相机模组的结构示意图。
其中，该相机模组10包括图像传感器101和至少两个镜头102，每一镜头102对应图像传感器101的不同成像区域，不同成像区域的中心像素不完全相同。
也即是说，本公开实施例支持在一个图像传感器101中布置多个镜头102，镜头102的数目至少为两个，对此不做限制。
本公开实施例中，可以预先设置多个镜头102在图像传感器101中对应的排列方式，多个镜头102可以按照预定的镜头排列方式排列在图像传感器101中，从而使得每一镜头102对应图像传感器101的不同成像区域。
举例而言，该排列方式，可以是异形切割排列，其中，异形切割，是对镜头进行切割的一种方式，通过对镜头进行异形切割，能够有效缩短镜头阵列中部分镜头之间的间距，使得多个镜头102能够尽可能近的排布，当然，也可以根据图像传感器101的大小与形状，配置对应的排列方式，对此不做限制。
本公开实施例中，如图2所示，图2是本公开实施例中镜头布局方式示意图，其中，针对一个图像传感器101可布局若干个镜头102，图2中以针对图像传感器101布局四个镜头102进行示例，对此不做限制。
其中，成像区域是图像传感器101中用于识别镜头102捕获的环境光的信息并进行成像的区域，如图2所示的镜头布局中，镜头102所对应的图像传感器101的区域，可以作为成像区域，对此不做限制。
其中，环境光的信息可以具体例如为环境光的光强度和波长信息等，对此不做限制。
其中，中心像素为成像区域的像素中，与成像区域所对应镜头102的视场中心相对应的像素，如图2所示，可以将镜头102的中心位置(该中心位置可以作为镜头102的视场中心)所对应的成像区域的像素作为中心像素，当然，也可以根据实际拍摄需求灵活定义中心像素，对此不做限制。
其中，视场，是相机模组10中镜头102所能够感知环境光的视野范围(Field Of View，FOV)，而视场中心，是镜头102所能够感知到的视野范围的中心点，本公开实施例的中可以将单个镜头102所捕获区域图像的中心点作为该镜头对应的视场中心，而该视场中心在成像区域中具有与之对应的中心像素，对此不做限制。
其中，由镜头102基于相应成像像素捕捉的环境光的信息所形成的图像，可以被称为区域图像，如图3所示，图3是本公开中区域图像示意图。
其中，区域图像，可以具体例如通过电子设备的图像传感器，基于相应的成像区域采集得到的未做任何处理的RAW格式图像，对此不作限制。
其中，RAW格式图像，即图像传感器将基于相应的成像区域捕捉到的光源信号转化为数字信号的区域图像。RAW格式图像记录了数码相机传感器的原始信息，同时记录了由相机拍摄所产生的一些元数据，如感光度的设置、快门速度、光圈值、白平衡等。
本公开实施例中，可以根据区域图像的中点位置确定镜头对应的视场中心，并根据该视场中心确定对应的图像传感器成像区域中的中心像素。
本公开实施例中，因镜头102的数目为多个，则不同镜头可以分别设置不同的中心像素，不同成像区域的中心像素不完全相同，对此不做限制。
其中，不同成像区域的中心像素不完全相同，不同成像区域的中心像素的种类可以为多种，也即是说，不同成像区域的中心像素具有与之对应的种类，该中心像素的种类可以为多种，对此不做限制。
在本公开的一些实施例中，不同成像区域的中心像素的种类等于图像传感器中像素的种类。
其中，像素的种类，可以具体例如为红绿蓝(Red Green Blue，RGB)颜色系统中对应的种类，也即可以将像素分为红色(Red，R)像素、绿色(Green，G)像素、蓝色(Blue，B)像素等，或者，也可以根据明亮度、色度与浓度将像素分类为明亮度(Luminance，Y)像素、色度(Chrominance，U)像素与浓度(Chroma，V)像素，对此不做限制。
本公开实施例中，图像传感器101中的像素可以按照一定的排列规则进行排列，例如在拜耳Bayer阵列中，如图3所示，以针对图像传感器101配置四个镜头102进行示例，图像传感器101中像素的排列为红绿绿蓝(Red Green Green Blue，RGGB)排列，在四个镜头102捕捉区域图像，并提供至图像传感器101时,不同镜头的视场中心分别对准成像区域中不同种类的像素上，例如，图3中镜头1的视场中心对准红色像素，镜头2的视场中心对准绿色像素，镜头3的视场中心对准与镜头2不同的绿色像素，镜头4的视场中心对准蓝色像素。由此，各个镜头对应的不同成像区域的中心像素的种类等于图像传感器中像素的种类，且不同成像区域的中心像素不完全相同，以保证图像传感器中每一种像素的种类均有与之对应的中心像素，从而有效提升该相机模组的成像效果。
在本公开的一些实施例中，对至少两个镜头101中在任一第一方向上相邻的第一镜头和第二镜头，对应第一镜头成像区域的第一中心像素不同于对应第二镜头成像区域的第二中心像素，或者与第一中心像素相邻的相邻像素排布方式不同于与第二中心像素相邻的相邻像素排布方式。
其中，第一方向，是横向、竖向、或者对角方向中的任一种，第一镜头和第二镜头为任一第一方向上相邻的两个镜头。
举例而言，如图2所示，若第一镜头为镜头1，第二镜头为镜头2，第一镜头和第二镜头为横向上相邻的两个镜头，若第一镜头为镜头1，第二镜头为镜头3，第一镜头和第二镜头为竖向上相邻的两个镜头，若第一镜头为镜头1，第二镜头为镜头4，第一镜头和第二镜头为对角方向上相邻的两个镜头。
其中，相邻像素排布方式，是以某一像素为中心，与该像素相邻的其他像素的排布方式，可以被称为相邻像素排布方式。
一些实施例中，第一镜头在图像传感器101中具有与之对应的第一镜头成像区域，该第一镜头成像区域具有第一中心像素，第二镜头在图像传感器101中具有与之对应的第二镜头成像区域，该第二镜头成像区域具有第二中心像素，第一中心像素与第二中心像素不同。
另一些实施例中，第一中心像素对应的相邻像素排布方式与第二中心像素对应的相邻像素排布方式不同，对此不做限制。
举例而言，如图3所示，在拜耳阵列中，像素的排列为红绿绿蓝(Red Green Green Blue，RGGB)排列，若第一镜头为镜头1，第二镜头为镜头2，第一中心像素为红色像素(镜头1中心像素)，第二中心像素为绿色像素(镜头2中心像素)，则第一中心像素与第二中心像素不同，若第一镜头为镜头2，第二镜头为镜头3，第一中心像素为其中一个绿色像素(镜头2中心像素)，第二中心像素为另一个绿色像素(镜头3中心像素)，则第一中心像素的相邻像素排布方式与第二中心像素的相邻像素排布方式不同。
在本公开的一些实施例中，对任意相邻且共用侧边的第一成像区域和第二成像区域，对应第一成像区域的第三镜头的视场中心与对应第二成像区域的第四镜头的视场中心之间的距离，小于或等于第三镜头半边长与第四镜头半边长的和。
其中，第三镜头半边长为第三镜头形成的正投影图形中最远点距离的一半，第四镜头半边长为第四镜头形成的正投影图形中最远点距离的一半，如图2所示，以镜头3作为第三镜头，镜头4作为第四镜头，则第三镜头与第四镜头所对应的成像区域相邻且共用侧边，则第三镜头半边长与第四镜头半边长如图2所示。
本公开实施例中，第三镜头的视场中心，为第三镜头的中心像素所对应的位置，第四镜头的视场中心，为第四镜头的中心像素所对应的位置，如图3所示，在第一成像区域与第二成像区域相邻且共用侧边时，将第一成像区域内第三镜头视场中心对应的像素与第二成像区域内第四镜头视场中心对应的像素分别用三角符号标注，则结合图2与图3，第一成像区域内第三镜头视场中心与第二成像区域内第四镜头视场中心之间的距离，小于或等于第三镜头半边长与第四镜头半边长的和，也即是说，本实施例中，通过设置第一成像区域内第三镜头视场中心与第二成像区域内第四镜头视场中心之间的距离小于或等于第三镜头半边长与第四镜头半边长的和，能够有效保证多个镜头102间的紧凑排列，从而能够利用尽可能多的图像传感器101的像面面积，减少由于镜头102间距导致的图像传感器101感光像素的损失，进而有效提升图像传感器的成像效果。
本实施例中提出的相机模组，通过每一镜头对应图像传感器的不同成像区域，不同成像区域的中心像素不完全相同，不同成像区域的中心像素的种类等于图像传感器中像素的种类，对至少两个镜头中在任一第一方向上相邻的第一镜头和第二镜头，对应第一镜头成像区域的第一中心像素不同于对应第二镜头成像区域的第二中心像素，或者与第一中心像素相邻的相邻像素排布方式不同于与第二中心像素相邻的相邻像素排布方式，对任意相邻且共用侧边的第一成像区域和第二成像区域，对应第一成像区域的第三镜头的视场中心与对应第二成像区域的第四镜头的视场中心之间的距离，小于或等于第三镜头半边长与第四镜头半边长的和，从而能够使得多个镜头在图像传感器中实现合适的排列，同时兼顾图像输出效果与图像传感器利用率，从而能够有效地保留图像细节，保障图像生成质量。
图4是本公开一实施例提出的图像处理方法的流程示意图。
其中，需要说明的是，本实施例的图像处理方法的执行主体为图像处理装置，该装置可以由软件和/或硬件的方式实现，该装置可以配置在电子设备中，该电子设备可以为手机、平板电脑、个人数字助理、穿戴式设备等具有各种操作系统、成像设备的硬件设备，对此不做限制。
需要说明的是，本公开实施例中的获取图像处理相关的信号与数据，均是在经过相关用户授权后获取的，其获取过程均符合相关法律、法规的规定，且不违背公序良俗。
如图4所示，该图像处理方法，包括：
S401：获取原始图像，其中，原始图像包括多个区域图像，每一区域图像对应图像传感器的不同成像区域，不同成像区域的中心像素不完全相同，中心像素为成像区域的像素中与镜头的视场中心对应的像素。
其中，图像传感器可获取成像区域中的每个成像像素捕捉的光强度和波长信息，并提供可由图像信号处理器ISP处理的区域图像。
本公开实施例中获取原始图像，其中，原始图像由各个镜头捕获的区域图像组成，一个镜头对应图像传感器的一块成像区域，成像区域中的成像像素捕捉相应镜头所透射的环境光的信息，以形成区域图像，可以将区域图像提供至图像信号处理器ISP，由图像信号处理器ISP触发执行后续步骤。
S402：基于多个区域图像生成待输出的目标图像。
本公开实施例中，可以将不同镜头所捕获的区域图像进行处理，以生成待输出的目标图像。
其中，对多个区域图像进行相应的处理(例如，使用相关算法、模型等，对此不做限制)得到的图像，可以被称为目标图像，由此，当参考不同成像区域的中心像素处理多个区域图像，能够使得目标图像携带了各个镜头捕获的区域图像的个性化摄像信息，使得目标图像具有更高的分辨率、画面质量，以及图像细节，从而实现镜头布局方式和图像生成效果的有效平衡。
本公开实施例中，可以设置深度学习的图像处理模型，以使用图像处理模型基于多个区域图像生成待输出的目标图像，或者，也可以使用自定义的图像处理算法，基于多个区域图像生成待输出的目标图像，对此不做限制。
以相机模组包括四个镜头为具体示例，如图5所示，图5是本公开另一实施例提出的目标图像生成示意图，四个镜头分别获取同一场景下的区域图像，并经过算法增强与超分算法处理，得到高品质的目标图像。
本实施例中，通过获取原始图像，其中，原始图像包括多个区域图像，每一区域图像对应图像传感器的不同成像区域，不同成像区域的中心像素不完全相同，中心像素为成像区域的像素中与镜头的视场中心对应的像素，基于多个区域图像处理待输出的目标图像，当针对图像传感器的不同成像区域捕获的区域图像，进行图像处理生成待输出的目标图像时，能够有效地保留图像细节，保障图像生成质量，提升图像处理效果。
图6是本公开另一实施例提出的图像处理方法的流程示意图。
如图6所示，该图像处理方法，包括：
S601：获取原始图像，其中，原始图像包括多个区域图像，每一区域图像对应图像传感器的不同成像区域，不同成像区域的中心像素不完全相同，中心像素为成像区域的像素中与镜头的视场中心对应的像素。
S601的描述说明可以具体参见上述实施例，在此不再赘述。
S602：对多个区域图像进行对齐处理，得到对齐图像；其中，对齐处理用于使得每一区域图像位置对应处的像素种类相同。
其中，对齐图像，是将同一场景下不同镜头拍摄得到的区域图像根据中心像素的位置进行对齐处理，得到的对齐后的图像。
举例而言，以针对图像传感器配置四个镜头进行示例，四个镜头分别拍摄得到的区域图像，其中心像素的位置不同，可以设置四个镜头中的一个镜头对应的中心像素的位置为参考，将其余三个镜头捕捉到的区域图像，联合相应镜头的中心像素的位置向作为参考的中心像素的位置进行对齐，以得到对齐图像，或者，也可以确定图像传感器中的像素的排列情况，基于像素的排列情况处理各个镜头所拍摄的区域图像，将该区域图像对齐处理为与该种像素的排列情况相一致图像，作为对齐图像，对此不做限制。
本公开实施例中，可以使用基于人工智能的图像处理方式处理区域图像，以对区域图像进行对齐处理，得到对齐图像，或者，也可以参考像素种类联合图像裁剪的方式裁剪区域图像，得到对齐图像，或者，还可以使用其他任意可能的实现方式，对区域图像进行对齐处理，以得到对齐图像，对此不做限制。
本公开实施例得到的对齐图像，每一对齐图像位置对应处的像素种类相同，与每一对齐图像位置对应处的像素周围相邻像素的排列方式也相同，以便于后续对齐图像的处理。
S603：基于对齐图像得到目标图像。
本公开实施例在对区域图像进行对齐处理，得到对齐图像之后，可以对多帧对齐图像进行融合处理，得到目标图像，或者，也可以使用图像信息提取技术，提取对其图像中的信息，以生成目标图像，或者，还可以使用图像渲染、图像增强等技术，处理对齐图像，得到目标图像，对此不做限制。
本公开实施例中，处理多帧对齐图像，可以使用相应的图像处理算法(例如超分算法)，将多帧对齐图像进行混叠融合，得到分辨率更高的目标图像，或者，也可以使用深度学习的图像处理网络，通过提取多帧对齐图像的特征，重建高质量的目标图像，或者，还可以是使用其他任意可能的实现方式处理多帧对齐图像，得到目标图像，对此不做限制。
本实施例中，由于是在针对图像传感器的不同成像区域捕获的区域图像，进行图像处理生成待输出的目标图像时，能够有效地保留图像细节，保障图像处理质量，提升图像处理效果，由于是对多个区域图像进行对齐处理，得到对齐图像，基于对齐图像得到目标图像，能够有效地提升区域图像的处理效率，达到更优的图像处理效果，有效提升目标图像的分辨率与画面质量。
图7是本公开另一实施例提出的图像处理方法的流程示意图。
如图7所示，该图像处理方法，包括：
S701：获取原始图像，其中，原始图像包括多个区域图像，每一区域图像对应图像传感器的不同成像区域，不同成像区域的中心像素不完全相同，中心像素为成像区域的像素中与镜头的视场中心对应的像素。
S701的描述说明可以具体参见上述实施例，在此不再赘述。
S702：对多个区域图像进行裁剪，且对于在任一第二方向上相邻且共用侧边的第三区域图像和第四区域图像，第三区域图像的裁剪区域内的第一中心像素相对于第四区域图像的裁剪区域内的第二中心像素具有像素偏移。
其中，像素偏移为，在第三区域图像的裁剪区域内，与第二中心像素位置对应的像素位于从第一中心像素沿着第二方向移动到与第二中心像素同种类像素的位置。
本公开实施例中，可以设置像素偏移的距离为一个像素的长度，或者，还可以根据实际需求，设置其他任意长度的像素偏移的距离(例如半个像素的长度、两个像素的长度等)，对此不做限制。
其中，第二方向，为横向或者竖向的一种，也即第二方向可以为横向，或者，也可以为竖向，对此不做限制。
可选地，本公开实施例中，每一区域图像的裁剪区域内，区域图像的中心像素位于区域图像的裁剪区域的中心位置，或者区域图像的中心像素相对于区域图像的非中心像素更接近区域中心位置，其中，区域图像的非中心像素为区域图像中除了中心像素之外的像素，由于是设置中心像素的位置，能够准确地表征出与裁剪区域对应的中心像素，尽可能使得裁剪时中心像素位于裁剪区域的中心位置，有效地保障裁剪区域内中心像素的确定效率和效果，有效增强图像处理的效果。
也即是说，可以设置区域图像的中心像素位于该区域图像裁剪区域的中心位置，或者，也可以设置裁剪区域的区域图像的中心像素位于相对于区域图像的非中心像素更接近区域中心位置。
举例而言，以裁剪区域的大小为五个像素乘以五个像素，则可以将裁剪区域内第三行第三列所处位置的像素作为中心像素，对此不做限制。
本公开实施例中，如图8所示，图8是本公开另一实施例提出的像素偏移示意图，以区域图像的数量为4张为例，经由虚线圈中的区域为裁剪区域，第一中心像素为第一区域图像视场中心所对应的像素， 第二中心像素为第二区域图像视场中心所对应的像素，能够直观地看出，在第三区域图像的裁剪区域内与第二中心像素所对应位置的像素，位于从与第一中心像素所对应位置的像素沿着横向向左移动一个像素点的位置，也即像素偏移，同理，第三区域图像的裁剪区域内的第一中心像素相对于第四区域图像的裁剪区域内的第二中心像素也具有像素偏移，由此，确定中心像素，并根据中心像素确定像素偏移情况，以对多个区域图像进行准确裁剪，保证裁剪后得到的图像具有像素偏移效果。
S703：对裁剪后得到的图像进行对齐处理，得到对齐图像。
本公开实施例中，对裁剪后得到的图像，可以根据相同位置对应的像素进行对齐，得到对齐图像。
在本公开的一些实施例中，可以使用图像处理算法，执行区域图像的裁剪与裁剪后得到的图像的对齐处理，或者，也可以设置图像处理大数据模型，基于大数据模型技术对裁剪后得到的图像进行对齐处理，得到对齐图像，或者，还可以使用特征提取技术、图像识别技术等多种技术对裁剪后得到的图像进行对齐处理，得到对齐图像，对此不做限制。
本实施例中，由于是在针对图像传感器的不同成像区域捕获的区域图像，进行图像处理生成待输出的目标图像时，能够有效地保留图像细节，保障图像处理质量，提升图像处理效果，由于是对多个区域图像进行对齐处理，得到对齐图像，基于对齐图像得到目标图像，能够有效地提升区域图像的处理效率，达到更优的图像处理效果，有效提升目标图像的分辨率与画面质量，由于对多个区域图像进行裁剪，以使得多个裁剪后得到的图像具有像素偏移效果，且将裁剪后得到的图像进行对齐处理，得到对齐图像，能够保障所生成目标图像的色彩细节，使得目标图像在色彩维度能够具有更优的画面质量，有效提升目标图像的色彩表现效果，从而有效提升目标图像的分辨率与画面质量。
图9是本公开另一实施例提出的图像处理方法的流程示意图。
如图9所示，该图像处理方法，包括：
S901：获取原始图像，其中，原始图像包括多个区域图像，每一区域图像对应图像传感器的不同成像区域，不同成像区域的中心像素不完全相同，中心像素为成像区域的像素中与镜头的视场中心对应的像素。
S902：对多个区域图像进行裁剪，且对于在任一第二方向上相邻且共用侧边的第三区域图像和第四区域图像，第三区域图像的裁剪区域内的第一中心像素相对于第四区域图像的裁剪区域内的第二中心像素具有像素偏移。
S901-S902的描述说明可以具体参见上述实施例，在此不再赘述。
S903：生成与裁剪后得到的图像对应的光流特征图。
本公开实施例中，可以对裁剪后得到的图像进行处理，以生成光流特征图。
其中，不同裁剪后得到的图像可以具有相同或者不相同的光流特征，可以基于裁剪后得到的图像对应的光流特征生成相应的光流特征图。
本公开实施例可以通过搭建光流网络，经过该光流网络对裁剪后得到的图像进行处理，得到与裁剪后得到的图像对应的光流特征图。
举例而言，可以使用深度学习的光流网络处理裁剪后得到的图像，以生成与裁剪后得到的图像对应的光流特征图，对此不做限制，本公开一些实施例中，可以对裁剪后得到的图像进行下采样处理，若裁剪后得到的图像的像素排列为红绿绿蓝(Red Green Green Blue，RGGB)排列，可以通过丢弃一个绿色(G)像素，使其变为下采样的红绿蓝(Red Green Blue，RGB)像素排列的图像，以便于光流特征图 的生成，当然，还可以使用其他任意可能的方式处理裁剪后得到的图像，生成光流特征图，对此不做限制。
S904：对光流特征图进行上采样处理，以得到上采样图像。
其中，对光流特征图进行上采样处理得到的图像，可以被称为上采样图像。
本公开实施例中，对于光流网络输出的光流特征图，可以使用双线性插值上采样方式生成上采样图像，或者，也可以使用最近邻值上采样、双三次插值上采样等方法处理光流特征图，生成上采样图像，对此不做限制。
S905：对上采样图像进行对齐处理，以得到对齐图像。
本公开实施例中，可以将上采样得到的多帧上采样图像进行对齐处理，以得到对齐图像。
举例而言，如图10所示，图10是本公开实施例中图像对齐方式示意图，选择一份区域图像作为参考区域图像，基于参考区域图像对其他区域图像进行裁剪，对裁剪后得到的图像进行下采样处理，并将处理得到的图像输入至光流网络中，生成光流特征图，光流特征图经过双线性插值上采样处理，生成上采样图像，将上采样图像进行对齐，以得到对齐图像。
S906：从对齐图像中提取相同种类的多个像素，其中，像素在对齐图像中具有对应的像素位置。
本公开实施例中，可以使用像素识别的方式，从对齐图像中提取相同种类的多个像素，并记录对应的像素位置，或者，也可以预先确定相同种类的多个像素对应的像素位置，根据像素位置从对齐图像中提取相同种类的多个像素，或者，还可以是使用其他任意可能的实现方式从对齐图像中提取相同种类的多个像素，对此不做限制。
举例而言，若像素排列为RGGB排列，可以提取一张对齐图像中红色种类的多个像素，并记录提取红色种类的像素对应的像素位置，同理也可以提取绿色种类的多个像素和蓝色种类的多个像素，对此不做限制。
S907：根据像素位置组合多个像素，得到待融合图像。
其中，经过组合多个像素所得到的图像，可以被称为待融合图像，待融合图像可以被用于融合为目标图像。
可选地，一些实施例中，可以根据像素位置组合多个像素，以得到组合图像，确定与组合图像对应的图像语义特征，根据图像语义特征对组合图像进行上采样处理，以得到待融合图像，由于是根据图像语义特征处理组合图像得到待融合图像，能够有效减少画面噪点对待融合图像的影响，进而提升待融合图像的图像质量，增强图像处理效果。
其中，组合图像所具有的图像语义维度的特征，可以被称为图像语义特征，图像语义特征可以是组合图像的纹理、色彩等特征，或者，也可以是组合图像对应的深度特征，对此不做限制。
本公开实施例中，可以使用对应的图像特征提取网络，确定深度特征作为组合图像对应的图像语义特征，或者，也可以是使用图像识别的方式对组合图像进行语义识别，以确定图像语义特征，或者，还可以是使用其他任意可能的实现方式确定与组合图像对应的图像语义特征，对此不做限制。
举例而言，若一张组合图像中展示物品A与物品B，则可以通过图像处理等方式提取用于表示物品A的轮廓特征与物品B的轮廓特征，该轮廓特征可以作为该组合图像的图像语义特征，或者，也可以通过深度学习网络(例如“U”型结构的语义分割算法网络等)提取组合图像对应的深度特征作为图像语义特征。
本公开实施例中，可以使用深度学习的融合网络提取组合图像对应的图像语义特征，并使用像素重组(pixel shuffle)技术联合图像语义特征分别对组合图像进行上采样处理，以确定待融合图像，或者，也可以使用图像融合处理模型，基于该模型对组合图像进行上采样处理，得到待融合图像，或者，还可以是使用其他任意可能的实现方式对组合图像进行上采样处理，得到待融合图像，对此不做限制。
可选地，另一些实施例中，也可以搭建特征提取系统，使用相应算法模型，提取组合图像对应的图像语义特征，当然，还可以使用其他任意可能的实现方式提取组合图像对应的图像语义特征，对此不做限制。
S908：融合多帧待融合图像，以得到目标图像。
本公开实施例中，对多帧待融合图像进行融合处理，可以使用图像处理方式直接融合多帧待融合图像，或者，也可以使用深度学习网络融合多帧待融合图像，或者，还可以采用其他任意可能的方式融合多帧待融合图像，以得到目标图像，对此不做限制。
本公开实施例中，对对齐图像的处理过程可以如图11所示，图11是本公开实施例中图像融合方式示意图，其中，对于对齐图像，可以按照相同种类的像素组成组合图像，而后，采用基于深度学习的融合网络，通过网络模型(例如语义分割算法网络、深度学习网络等)提取组合图像对应的图像语义特征，而后，使用像素重组(pixel shuffle)技术和上采样处理输出图像语义特征，并使用一个卷积层来重建高质量高分辨率的目标图像，若对齐图像为4张，则可以设置输入的对齐图像的维度为(2H，2W，4)，其中，“H”表示对齐图像的长，“W”表示对齐图像的宽，“4”表示对齐图像的张数，由此，经过处理得到的组合图像的图像维度为(H，W，16)，通过网络模型提取图像语义特征，生成的维度为(H，W，2s×2s×512)，其中，“s”为提取的图像语义特征的数量，对图像语义特征进行像素重组，得到的重组后的待融合图像的维度为(2sH，2sW，512)，而后使用一个卷积层对重组后的图像进行卷积处理，重建高质量高分辨率的目标图像。
举例而言，如图12所示，图12是本公开另一实施例提出的像素偏移处理结果对比图，由图12对比可以得出，使用像素偏移技术得到的目标图像质量与分辨率更高，图像处理效果更好。
本实施例中，由于是在针对图像传感器的不同成像区域捕获的区域图像，进行图像处理生成待输出的目标图像时，能够有效地保留图像细节，保障图像处理质量，提升图像处理效果，由于是对多个区域图像进行对齐处理，得到对齐图像，基于对齐图像得到目标图像，能够有效地提升区域图像的处理效率，达到更优的图像处理效果，有效提升目标图像的分辨率与画面质量，由于对多个区域图像进行裁剪，以使得多个裁剪后得到的图像具有像素偏移效果，且将裁剪后得到的图像进行对齐处理，得到对齐图像，能够保障所生成目标图像的色彩细节，使得目标图像在色彩维度能够具有更优的画面质量，有效提升目标图像的色彩表现效果，从而有效提升目标图像的分辨率与画面质量。由于是对裁剪后得到的图像进行光流特征提取和生成、以及上采样处理，而后进行图像对齐，能够在将裁剪后得到的图像进行对齐的过程中，有效地保留图像细节，避免在对齐处理的过程中对图像细节带入损失，并且能够在有效保留图像细节的同时，降低图像对齐处理所需的处理资源消耗，实现对齐处理效果和对齐处理效率的平衡。由于是根据图像语义特征处理组合图像得到待融合图像，能够有效减少画面噪点对待融合图像的影响，进而提升待融合图像的图像质量，增强图像处理效果。
图13是本公开一实施例提出的图像处理装置的结构示意图。
如图13所示，该图像处理装置130，包括：
获取模块1301，用于获取原始图像，其中，原始图像包括多个区域图像，每一区域图像对应图像传感器的不同成像区域，不同成像区域的中心像素不完全相同，中心像素为成像区域的像素中与镜头的视场中心对应的像素；
生成模块1302，用于基于多个区域图像生成待输出的目标图像。
在本公开的一些实施例中，如图14所示，图14是本公开另一实施例提出的图像处理装置的结构示意图，生成模块1302，包括：
第一处理子模块13021，用于对多个区域图像进行对齐处理，得到对齐图像；其中，对齐处理用于使得每一区域图像位置对应处的像素种类相同；
第二处理子模块13022，用于基于对齐图像得到目标图像。
在本公开的一些实施例中，如图14所示，第一处理子模块13021，具体用于：
对多个区域图像进行裁剪，且对于在任一第二方向上相邻且共用侧边的第三区域图像和第四区域图像，第三区域图像的裁剪区域内的第一中心像素相对于第四区域图像的裁剪区域内的第二中心像素具有像素偏移；
对裁剪后得到的图像进行对齐处理，得到对齐图像；
其中，像素偏移为，在第三区域图像的裁剪区域内，与第二中心像素位置对应的像素位于从第一中心像素沿着第二方向移动到与第二中心像素同种类像素的位置。
在本公开的一些实施例中，如图14所示，第一处理子模块13021，具体用于：
生成与裁剪后得到的图像对应的光流特征图；
对光流特征图进行上采样处理，以得到上采样图像；
对上采样图像进行对齐处理，以得到对齐图像。
在本公开的一些实施例中，如图14所示，每一区域图像的裁剪区域内，区域图像的中心像素位于区域图像的裁剪区域的中心位置，或者区域图像的中心像素相对于区域图像的非中心像素更接近区域中心位置；
其中，区域图像的非中心像素为区域图像中除了中心像素之外的像素。
在本公开的一些实施例中，如图14所示，第二处理子模块13022，具体用于：
从对齐图像中提取相同种类的多个像素，其中，像素在对齐图像中具有对应的像素位置；
根据像素位置组合多个像素，得到待融合图像；
融合多帧待融合图像，以得到目标图像。
在本公开的一些实施例中，如图14所示，第二处理子模块13022，具体用于：
根据像素位置组合多个像素，以得到组合图像；
确定与组合图像对应的图像语义特征；
根据图像语义特征对组合图像进行上采样处理，以得到待融合图像。
与上述图4至图12实施例提供的图像处理方法相对应，本公开还提供一种图像处理装置，由于本公开实施例提供的图像处理装置与上述图4至图12实施例提供的图像处理方法相对应，因此在图像处理方法的实施方式也适用于本公开实施例提供的图像处理装置，在本公开实施例中不再详细描述。
本实施例中，通过获取原始图像，其中，原始图像包括多个区域图像，每一区域图像对应图像传感器的不同成像区域，不同成像区域的中心像素不完全相同，中心像素为成像区域的像素中与镜头的视场 中心对应的像素，基于多个区域图像生成待输出的目标图像，当针对图像传感器的不同成像区域捕获的区域图像，进行图像处理生成待输出的目标图像时，能够有效地保留图像细节，保障图像处理质量，提升图像处理效果。
图15是本公开一实施例提出的终端的结构示意图。
如图15所示，该终端150，包括相机模组10。
图16示出了适于用来实现本公开实施方式的示例性电子设备的框图。图16显示的电子设备12仅仅是一个示例，不应对本公开实施例的功能和使用范围带来任何限制。
如图16所示，电子设备12以通用计算设备的形式表现。电子设备12的组件可以包括但不限于：一个或者多个处理器或者处理单元17，系统存储器28，连接不同系统组件(包括系统存储器28和处理单元17)的总线18。总线18表示几类总线结构中的一种或多种，包括存储器总线或者存储器控制器，外围总线，图形加速端口，处理器或者使用多种总线结构中的任意总线结构的局域总线。举例来说，这些体系结构包括但不限于工业标准体系结构(Industry Standard Architecture；以下简称：ISA)总线，微通道体系结构(Micro Channel Architecture；以下简称：MAC)总线，增强型ISA总线、视频电子标准协会(Video Electronics Standards Association；以下简称：VESA)局域总线以及外围组件互连(Peripheral Component Interconnection；以下简称：PCI)总线。
电子设备12典型地包括多种计算机系统可读介质。这些介质可以是任何能够被电子设备12访问的可用介质，包括易失性和非易失性介质，可移动的和不可移动的介质。
存储器28可以包括易失性存储器形式的计算机系统可读介质，例如随机存取存储器(Random Access Memory；以下简称：RAM)30和/或高速缓存存储器32。电子设备12可以进一步包括其他可移动/不可移动的、易失性/非易失性计算机系统存储介质。仅作为举例，存储系统34可以用于读写不可移动的、非易失性磁介质(图16未显示，通常称为“硬盘驱动器”)。
尽管图16中未示出，可以提供用于对可移动非易失性磁盘(例如“软盘”)读写的磁盘驱动器，以及对可移动非易失性光盘(例如：光盘只读存储器(Compact Disc Read Only Memory；以下简称：CD-ROM)、数字多功能只读光盘(Digital Video Disc Read Only Memory；以下简称：DVD-ROM)或者其他光介质)读写的光盘驱动器。在这些情况下，每个驱动器可以通过一个或者多个数据介质接口与总线18相连。存储器28可以包括至少一个程序产品，该程序产品具有一组(例如至少一个)程序模块，这些程序模块被配置以执行本公开各实施例的功能。
具有一组(至少一个)程序模块42的程序/实用工具40，可以存储在例如存储器28中，这样的程序模块42包括但不限于操作系统、一个或者多个应用程序、其他程序模块以及程序数据，这些示例中的每一个或某种组合中可能包括网络环境的实现。程序模块42通常执行本公开所描述的实施例中的功能和/或方法。
电子设备12也可以与一个或多个外部设备15(例如键盘、指向设备、显示器24等)通信，还可与一个或者多个使得用户能与该电子设备12交互的设备通信，和/或与使得该电子设备12能与一个或多个其他计算设备进行通信的任何设备(例如网卡，调制解调器等等)通信。这种通信可以通过输入/输出(I/O)接口22进行。并且，电子设备12还可以通过网络适配器20与一个或者多个网络(例如局域网(Local Area Network；以下简称：LAN)，广域网(Wide Area Net work；以下简称：WAN)和/或公共网络，例如因特网)通信。如图所示，网络适配器20通过总线18与电子设备12的其他模块通信。 应当明白，尽管图中未示出，可以结合电子设备12使用其他硬件和/或软件模块，包括但不限于：微代码、设备驱动器、冗余处理单元、外部磁盘驱动阵列、RAID系统、磁带驱动器以及数据备份存储系统等。
处理单元17通过运行存储在系统存储器28中的程序，从而执行各种功能应用以及数据处理，例如实现前述实施例中提及的图像处理方法。
为了实现上述实施例，本公开还提供了一种电子设备，包括：相机模组、存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，相机模组与处理器电连接，处理器执行计算机程序时，实现本公开前述实施例的图像处理方法。
为了实现上述实施例，本公开还提出一种非临时性计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时实现如本公开前述实施例提出的图像处理方法。
为了实现上述实施例，本公开还提供了一种计算机程序产品，包括计算机程序，计算机程序在被处理器执行时实现本公开前述实施例的图像处理方法。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机程序。在计算机上加载和执行所述计算机程序时，全部或部分地产生按照本公开实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机程序可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，所述计算机程序可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(digital subscriber line，DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质(例如，软盘、硬盘、磁带)、光介质(例如，高密度数字视频光盘(digital video disc，DVD))、或者半导体介质(例如，固态硬盘(solid state disk，SSD))等。
本领域普通技术人员可以理解：本公开中涉及的第一、第二等各种数字编号仅为描述方便进行的区分，并不用来限制本公开实施例的范围，也表示先后顺序。
本公开中的至少一个还可以描述为一个或多个，多个可以是两个、三个、四个或者更多个，本公开不做限制。在本公开实施例中，对于一种技术特征，通过“第一”、“第二”、“第三”、“A”、“B”、“C”和“D”等区分该种技术特征中的技术特征，该“第一”、“第二”、“第三”、“A”、“B”、“C”和“D”描述的技术特征间无先后顺序或者大小顺序。
本公开中各表所示的对应关系可以被配置，也可以是预定义的。各表中的信息的取值仅仅是举例，可以配置为其他值，本公开并不限定。在配置信息与各参数的对应关系时，并不一定要求必须配置各表中示意出的所有对应关系。例如，本公开中的表格中，某些行示出的对应关系也可以不配置。又例如，可以基于上述表格做适当的变形调整，例如，拆分，合并等等。上述各表中标题示出参数的名称也可以采用通信装置可理解的其他名称，其参数的取值或表示方式也可以通信装置可理解的其他取值或表示方式。上述各表在实现时，也可以采用其他的数据结构，例如可以采用数组、队列、容器、栈、线性表、指针、链表、树、图、结构体、类、堆、散列表或哈希表等。
本公开中的预定义可以理解为定义、预先定义、存储、预存储、预协商、预配置、固化、或预烧制。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本公开的范围。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统、装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
以上所述，仅为本公开的具体实施方式，但本公开的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本公开揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本公开的保护范围之内。因此，本公开的保护范围应以所述权利要求的保护范围为准。

Claims (15)

  1. 一种相机模组，其特征在于，包括图像传感器和至少两个镜头；
    每一镜头对应所述图像传感器的不同成像区域，不同成像区域的中心像素不完全相同；
    其中，中心像素为成像区域的像素中，与成像区域所对应镜头的视场中心相对应的像素。
  2. 根据权利要求1所述的相机模组，其特征在于，
    不同成像区域的中心像素的种类等于所述图像传感器中像素的种类。
  3. 根据权利要求1或2所述的相机模组，其特征在于，
    对所述至少两个镜头中在任一第一方向上相邻的第一镜头和第二镜头，对应所述第一镜头成像区域的第一中心像素不同于对应所述第二镜头成像区域的第二中心像素，或者与所述第一中心像素相邻的相邻像素排布方式不同于与所述第二中心像素相邻的相邻像素排布方式。
  4. 根据权利要求1-3任一项所述的相机模组，其特征在于，
    对任意相邻且共用侧边的第一成像区域和第二成像区域，对应所述第一成像区域的第三镜头的视场中心与对应所述第二成像区域的第四镜头的视场中心之间的距离，小于或等于所述第三镜头半边长与所述第四镜头半边长的和；
    其中，所述第三镜头半边长为第三镜头形成的正投影图形中最远点距离的一半，所述第四镜头半边长为第四镜头形成的正投影图形中最远点距离的一半。
  5. 一种图像处理方法，其特征在于，包括：
    获取原始图像，其中，所述原始图像包括多个区域图像，每一区域图像对应图像传感器的不同成像区域，不同成像区域的中心像素不完全相同，中心像素为成像区域的像素中与镜头的视场中心对应的像素；
    基于多个区域图像生成待输出的目标图像。
  6. 如权利要求5所述的图像处理方法，其特征在于，所述基于多个区域图像生成待输出的目标图像，包括：
    对所述多个区域图像进行对齐处理，得到对齐图像；其中，所述对齐处理用于使得每一区域图像位置对应处的像素种类相同；
    基于所述对齐图像得到所述目标图像。
  7. 根据权利要求6所述的图像处理方法，其特征在于，所述对所述多个区域图像进行对齐处理，得到对齐图像，包括：
    对所述多个区域图像进行裁剪，且对于在任一第二方向上相邻且共用侧边的第三区域图像和第四区域图像，所述第三区域图像的裁剪区域内的第一中心像素相对于所述第四区域图像的裁剪区域内的第二 中心像素具有像素偏移；
    对裁剪后得到的图像进行对齐处理，得到所述对齐图像；
    其中，所述像素偏移为，在所述第三区域图像的裁剪区域内，与所述第二中心像素位置对应的像素位于从所述第一中心像素沿着所述第二方向移动到与所述第二中心像素同种类像素的位置。
  8. 根据权利要求7所述的图像处理方法，其特征在于，所述对裁剪后得到的图像进行对齐处理，得到所述对齐图像，包括：
    生成与所述裁剪后得到的图像对应的光流特征图；
    对所述光流特征图进行上采样处理，以得到上采样图像；
    对所述上采样图像进行对齐处理，以得到所述对齐图像。
  9. 根据权利要求7所述的图像处理方法，其特征在于，
    每一区域图像的裁剪区域内，区域图像的中心像素位于区域图像的裁剪区域的中心位置，或者区域图像的中心像素相对于区域图像的非中心像素更接近区域所述中心位置；
    其中，区域图像的非中心像素为区域图像中除了中心像素之外的像素。
  10. 根据权利要求6所述的图像处理方法，其特征在于，所述基于所述对齐图像得到所述目标图像，包括：
    从所述对齐图像中提取相同种类的多个像素，其中，所述像素在所述对齐图像中具有对应的像素位置；
    根据所述像素位置组合所述多个像素，得到待融合图像；
    融合多帧所述待融合图像，以得到所述目标图像。
  11. 根据权利要求10所述的图像处理方法，其特征在于，所述根据所述像素位置组合所述多个像素，得到待融合图像，包括：
    根据所述像素位置组合所述多个像素，以得到组合图像；
    确定与所述组合图像对应的图像语义特征；
    根据所述图像语义特征对所述组合图像进行上采样处理，以得到所述待融合图像。
  12. 一种图像处理装置，其特征在于，包括：
    获取模块，用于获取原始图像，其中，所述原始图像包括多个区域图像，每一区域图像对应图像传感器的不同成像区域，不同成像区域的中心像素不完全相同，中心像素为成像区域的像素中与镜头的视场中心对应的像素；
    生成模块，用于基于多个区域图像生成待输出的目标图像。
  13. 一种终端，其特征在于，包括权利要求1-4任一项所述的相机模组。
  14. 一种电子设备，其特征在于，包括：
    相机模组；
    至少一个处理器；以及
    与所述至少一个处理器通信连接的存储器；其中，
    所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行权利要求5-11中任一项所述的图像处理方法。
  15. 一种存储有计算机指令的非瞬时计算机可读存储介质，其特征在于，其中，所述计算机指令用于使所述计算机执行权利要求5-11中任一项所述的图像处理方法。
PCT/CN2022/097972 2022-06-09 2022-06-09 相机模组及图像处理方法、装置、终端、电子设备、介质 WO2023236162A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2022/097972 WO2023236162A1 (zh) 2022-06-09 2022-06-09 相机模组及图像处理方法、装置、终端、电子设备、介质
CN202280004460.4A CN117643047A (zh) 2022-06-09 2022-06-09 相机模组及图像处理方法、装置、终端、电子设备、介质

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/097972 WO2023236162A1 (zh) 2022-06-09 2022-06-09 相机模组及图像处理方法、装置、终端、电子设备、介质

Publications (1)

Publication Number Publication Date
WO2023236162A1 true WO2023236162A1 (zh) 2023-12-14

Family

ID=89117410

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/097972 WO2023236162A1 (zh) 2022-06-09 2022-06-09 相机模组及图像处理方法、装置、终端、电子设备、介质

Country Status (2)

Country Link
CN (1) CN117643047A (zh)
WO (1) WO2023236162A1 (zh)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005176040A (ja) * 2003-12-12 2005-06-30 Canon Inc 撮像装置
CN102547080A (zh) * 2010-12-31 2012-07-04 联想(北京)有限公司 摄像模组以及包含该摄像模组的信息处理设备
CN102801929A (zh) * 2011-05-26 2012-11-28 佳能株式会社 图像传感器和摄像设备
JP2014127839A (ja) * 2012-12-26 2014-07-07 Mitsubishi Electric Corp 画像合成装置、および画像合成方法
CN109963082A (zh) * 2019-03-26 2019-07-02 Oppo广东移动通信有限公司 图像拍摄方法、装置、电子设备、计算机可读存储介质
CN112104796A (zh) * 2019-06-18 2020-12-18 Oppo广东移动通信有限公司 图像处理方法和装置、电子设备、计算机可读存储介质
CN112532839A (zh) * 2020-11-25 2021-03-19 深圳市锐尔觅移动通信有限公司 一种摄像头模组、成像方法、成像装置及移动设备
CN113156656A (zh) * 2021-03-31 2021-07-23 浙江罗比科技有限公司 一种变焦摄像机光轴校正方法

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005176040A (ja) * 2003-12-12 2005-06-30 Canon Inc 撮像装置
CN102547080A (zh) * 2010-12-31 2012-07-04 联想(北京)有限公司 摄像模组以及包含该摄像模组的信息处理设备
CN102801929A (zh) * 2011-05-26 2012-11-28 佳能株式会社 图像传感器和摄像设备
JP2014127839A (ja) * 2012-12-26 2014-07-07 Mitsubishi Electric Corp 画像合成装置、および画像合成方法
CN109963082A (zh) * 2019-03-26 2019-07-02 Oppo广东移动通信有限公司 图像拍摄方法、装置、电子设备、计算机可读存储介质
CN112104796A (zh) * 2019-06-18 2020-12-18 Oppo广东移动通信有限公司 图像处理方法和装置、电子设备、计算机可读存储介质
CN112532839A (zh) * 2020-11-25 2021-03-19 深圳市锐尔觅移动通信有限公司 一种摄像头模组、成像方法、成像装置及移动设备
CN113156656A (zh) * 2021-03-31 2021-07-23 浙江罗比科技有限公司 一种变焦摄像机光轴校正方法

Also Published As

Publication number Publication date
CN117643047A (zh) 2024-03-01

Similar Documents

Publication Publication Date Title
US10645368B1 (en) Method and apparatus for estimating depth of field information
US9754422B2 (en) Systems and method for performing depth based image editing
US9591237B2 (en) Automated generation of panning shots
US9373187B2 (en) Method and apparatus for producing a cinemagraph
CN111353948B (zh) 一种图像降噪方法、装置及设备
JP2020536327A (ja) 単一のカメラを使用した深度の推定
WO2018082185A1 (zh) 图像处理方法和装置
TW200903347A (en) Edge mapping using panchromatic pixels
CN104641625B (zh) 图像处理装置、摄像装置及图像处理方法
CN104641624B (zh) 图像处理装置、摄像装置及图像处理方法
WO2021115179A1 (zh) 图像处理方法、图像处理装置、存储介质与终端设备
EP3681144A1 (en) Video processing method and apparatus based on augmented reality, and electronic device
US20140362252A1 (en) Digital photographing apparatus and method of controlling the same
CN110930301A (zh) 图像处理方法、装置、存储介质及电子设备
US8995784B2 (en) Structure descriptors for image processing
WO2023226218A1 (zh) 轴对称光学成像并行仿真方法及装置
CN111353965B (zh) 图像修复方法、装置、终端及存储介质
AU2018271418B2 (en) Creating selective virtual long-exposure images
US9143754B2 (en) Systems and methods for modifying stereoscopic images
US20230033956A1 (en) Estimating depth based on iris size
WO2023236162A1 (zh) 相机模组及图像处理方法、装置、终端、电子设备、介质
US20130343656A1 (en) Image creating device, image creating method and recording medium
US20230098437A1 (en) Reference-Based Super-Resolution for Image and Video Enhancement
WO2023283855A1 (en) Super resolution based on saliency
CN107517367A (zh) 拜尔域图像插值方法、装置、图像处理芯片及存储装置

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 202280004460.4

Country of ref document: CN

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22945305

Country of ref document: EP

Kind code of ref document: A1