WO2020011112A1 - Image processing method and system, readable storage medium, and terminal - Google Patents

Abstract

Provided are an imaging processing method and system, a readable storage medium, and a terminal. The method is applied to the terminal on which a main camera and at least two auxiliary cameras are provided, wherein all of the cameras are connected through a synchronization signal line, and all the cameras have a common photographing area. The method comprises: acquiring images synchronously photographed by all the cameras; carrying out image synthesis on each auxiliarily photographed image and a mainly photographed image in sequence, and selecting one of the images as a reference image according to a first pre-set rule during each synthesis process, so as to synthesize a plurality of primary optimized images; and carrying out image synthesis on the plurality of primary optimized images according to a second pre-set rule, so as to synthesize a secondary optimized image. By means of the image processing method and system, the readable storage medium and the terminal in the present invention, a plurality of auxiliary cameras are arranged, and image synthesis is carried out repeatedly, so that a more accurate depth of field image can be output, erroneous blurring can be avoided, and a low-noise and high-definition image can also be output.

Description

Image processing method, system, readable storage medium and terminal [Technical Field]

The present invention relates to the technical field of multi-camera imaging, and in particular, to an image processing method, system, readable storage medium, and terminal.

【Background technique】

With the continuous improvement of people's living standards, terminals are constantly popularizing, such as mobile phones, tablets, computers, and cameras. At present, almost all smart devices have integrated camera lenses to achieve the camera's camera functions.

In recent years, with the continuous improvement of users' requirements for the photo quality of terminals, a number of terminals equipped with dual front and rear cameras have been born. Dual cameras can obtain the depth of field of images, achieve background blur, or support Optical zoom, color + black and white noise reduction and other effects. Current terminals are moving towards this multi-camera trend.

However, in the prior art, although the current dual cameras can obtain the depth of field of the image, they cannot be calibrated, resulting in a poor background blur effect. For example, objects such as faces and hair at the focus distance are incorrectly blurred. At the same time, the dual cameras cannot support both optical zoom and image noise reduction, resulting in low noise and high definition images.

[Summary of the Invention]

Based on this, the object of the present invention is to provide an image processing method, system, readable storage medium, and terminal to solve the technical problem that the existing dual cameras cannot obtain low noise and high definition images.

An image processing method according to an embodiment of the present invention is applied to a terminal. The terminal is provided with a main camera and at least two sub cameras. All cameras are connected through a synchronization signal line, and the main camera is connected to each of the cameras. The sub-cameras have a common shooting area. The method includes: acquiring a main-photograph image taken by the main camera, and separately acquiring a sub-photograph image synchronized by each of the sub-cameras; sequentially each of the sub-cameras An image is synthesized with one of the main images, and each of the images is selected as a reference image according to a first preset rule to synthesize multiple optimized images at a time. According to the second preset rule, Image synthesis is performed on a plurality of the primary optimized images to synthesize a secondary optimized image.

In addition, an image processing method according to the foregoing embodiment of the present invention may further have the following additional technical features:

Further, the one-time optimized image is a depth image.

Further, the step of performing image synthesis on a plurality of the one-time optimized images includes: selecting one of the one-time optimized images as a first reference image according to the second preset rule; obtaining the first reference A target image region having abnormal depth information in the image; obtaining target depth information of the same region as the target image region from other one-time optimized images, and using the target depth information to correct the depth information of the target image region.

Further, the preset rule includes any one of the following rules: a rule that preferentially selects an image with the least abnormal depth information; a rule that preferentially selects an image with the largest field of view; a rule that preferentially selects an image with the smallest field of view; priority Select a composite image of images taken by a specific two cameras; or a rule that preferentially selects the image with the highest image quality.

Further, the main camera is a first color camera, and all the sub cameras include at least a black and white camera and a second color camera, and there is a difference in the equivalent focal lengths of the first color camera and the second color camera. .

Further, the step of synthesizing the first optimized image by the images captured by the first color camera and the black and white camera includes: according to the first preset rule, a first color image captured from the first color camera And one of the second luminance signal images captured by the black and white camera is selected as a reference image; the first color image is split into a first luminance signal image and a first chrominance signal image; based on the selected A reference image, performing noise reduction synthesis on the first luminance signal image and the second luminance signal image to obtain a third luminance signal image after noise reduction; combining the first chrominance signal image and the third luminance signal image The brightness signal image is synthesized to obtain the once optimized image after noise reduction.

Further, the step of synthesizing the first optimized image by the images captured by the first color camera and the second color camera includes: according to the first preset rule, a first image captured by the first color camera A color image, and a second image selected from the second color camera as a reference image; determining the first color according to the focal lengths of the first color camera and the second color camera The subject and background relationship between the image and the second color image; and selectively performing a virtual correction on the first color image and the second color image based on the selected reference image and the determined subject and background relationship Chemical synthesis to obtain a clear one-time optimized image.

Further, the arrangement manner of the cameras on the terminal is any one of the following situations: when two secondary cameras are provided on the terminal, the connection between the main camera and the two secondary cameras When the three sub cameras are provided on the terminal, all the cameras are arranged in a rectangular arrangement, and the four cameras are located at four corner points of the rectangle.

Further, a pin connected to the synchronization signal line of the main camera is used as a synchronization signal output terminal, a pin connected to each synchronization signal line of each of the sub cameras is used as a synchronization signal input terminal, and the frame rate of each camera is the same.

An image processing system according to an embodiment of the present invention is applied to a terminal. The terminal is provided with a main camera and at least two sub cameras. All cameras are connected through a synchronization signal line, and the main camera is connected to each of the cameras. There is a common shooting area between the sub cameras, and the system includes: an image acquisition module for acquiring a main image taken by the main camera, and separately acquiring sub images taken by each of the sub cameras simultaneously; A compositing module for sequentially synthesizing each of the sub-photograph images with one of the main-photograph images, and in each compositing, one of the images is selected as a reference image according to a first preset rule to synthesize multiple images A single optimized image; a second composition module, configured to synthesize multiple first optimized images according to a second preset rule to synthesize a second optimized image.

In addition, an image processing system according to the above embodiment of the present invention may also have the following additional technical features:

Further, the one-time optimized image is a depth image.

Further, the secondary synthesis module includes: a reference selection unit, configured to select one of the primary optimized images as a first reference image according to the second preset rule; and an abnormality acquisition unit, configured to acquire all A target image region having abnormal depth information in the first reference image; a depth calibration unit, configured to obtain target depth information in the same region as the target image region from other one-time optimized images, and use the target depth information to Correcting depth information of the target image area.

Further, the second preset rule includes any one of the following rules: a rule that preferentially selects an image with the least abnormal depth information; a rule that preferentially selects an image with the largest field of view; a rule that preferentially selects an image with the smallest field of view ; Preferentially select the composite image of the images captured by the specific two cameras; or the rule of preferentially selecting the image with the highest image quality.

Further, the main camera is a first color camera, and all the sub cameras include at least a black and white camera and a second color camera, and there is a difference in the equivalent focal lengths of the first color camera and the second color camera. .

Further, the one-time composition module includes a first selection unit configured to, according to the first preset rule, a first color image captured from the first color camera, and a second color image captured by the black and white camera. An image is selected from the luminance signal images as a reference image; an image splitting unit is configured to split the first color image into a first luminance signal image and a first chrominance signal image; a first synthesizing unit is configured to Performing noise reduction synthesis on the first luminance signal image and the second luminance signal image to obtain a noise-reduced third luminance signal image; a second synthesis unit configured to combine the first chrominance signal image with the first luminance signal image; The third brightness signal image is synthesized to obtain the once optimized image after noise reduction.

Further, the one-time synthesis module further includes: a second selection unit configured to, according to the first preset rule, a first color image captured from the first color camera, and a second color camera capture One of the second color images is selected as a reference image; the relationship determining unit is configured to determine the first color image and the first color image according to the focal lengths of the first color camera and the second color camera; Subject and background relationship between two color images; an image blurring unit for selecting the first color image and the second color image based on the selected reference image and the determined subject and background relationship Sexual blurring synthesis to obtain a clear one-time optimized image.

Further, the arrangement manner of the cameras on the terminal is any one of the following situations: when two secondary cameras are provided on the terminal, the connection between the main camera and the two secondary cameras When the three sub cameras are provided on the terminal, all the cameras are arranged in a rectangular arrangement, and the four cameras are located at four corner points of the rectangle.

Further, a pin connected to the synchronization signal line of the main camera is used as a synchronization signal output terminal, a pin connected to each synchronization signal line of each of the sub cameras is used as a synchronization signal input terminal, and the frame rate of each camera is the same.

The present invention also proposes a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the image processing method as described above is implemented.

The present invention also provides a terminal including a memory, a processor, and a computer program stored on the memory and executable on the processor. The terminal is provided with a main camera and at least two sub cameras, and all cameras pass a synchronization signal. The main camera and each of the sub cameras have a common shooting area, and the processor implements the method as described above when the processor executes the program.

The image processing method, system, readable storage medium, and terminal described above, by arranging multiple secondary cameras and having a common shooting area between the primary camera and each secondary camera, acquire a primary image and multiple images simultaneously. Sub-shooting images, and then synthesizing each sub-shooting image with a main-shooting image to obtain multiple optimized images at once. This time the combination is a dual-shot combination, which can synthesize multiple depth-of-field images and / or at least one clear image. Image and at least one noise reduction image, and then synthesizing multiple once-optimized images to output a second-optimized image. Because the synthesis of multiple depth-of-field images can achieve calibration of depth-of-field information, and clear images and noise reduction The images can be superimposed and synthesized with each other, so the image processing method, system, readable storage medium and terminal can output a more accurate depth of field map to avoid error blurring, and at the same time can output low noise and high definition images, which can improve the overall quality of the captured images. Picture quality.

[Brief Description of the Drawings]

FIG. 1 is a flowchart of an image processing method in a first embodiment of the present invention;

2 is a flowchart of an image processing method in a second embodiment of the present invention;

3 is an arrangement structure of each camera in a second embodiment of the present invention;

4 is an arrangement structure of each camera in another embodiment of the present invention;

5 is a flowchart of an image processing method in a third embodiment of the present invention;

FIG. 6 is a schematic structural diagram of an image processing system in a fourth embodiment of the present invention.

Explanation of main component symbols:

Image acquisition module	11	One-time synthesis module	12
Secondary Synthesis Module	13	Benchmark Selection Unit	131
Exception Obtaining Unit	132	Depth Calibration Unit	133
First selected unit	121	Second selected unit	125
Image splitting unit	122	First synthesis unit	123
Second synthesis unit	124	Relationship determination unit	126
Image blurring unit	127	Zh	Zh

The following specific embodiments will further explain the present invention in combination with the above drawings.

【detailed description】

In order to facilitate understanding of the present invention, the present invention will be described more fully hereinafter with reference to the accompanying drawings. The figures show several embodiments of the invention. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

It should be noted that when an element is called “fixed to” another element, it may be directly on the other element or a centered element may exist. When an element is considered to be "connected" to another element, it can be directly connected to the other element or intervening elements may also be present. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

Please refer to FIG. 1, which shows an image processing method according to a first embodiment of the present invention, which is applied to a terminal. The terminal is provided with a main camera and at least two sub cameras, and all cameras are connected through a synchronization signal line. There is a common shooting area between the main camera and each of the sub cameras, and the image processing method includes steps S01 to S03.

In step S01, a main camera image captured by the main camera is acquired, and a sub camera image captured by each of the sub cameras is acquired separately.

In specific implementation, the pins connected to the synchronization signal line of the main camera can be used as the synchronization signal output terminal, and the pins connected to the synchronization signal line of each sub camera are used as the synchronization signal input terminal, and the frame rate of each camera is the same. When the main camera starts the exposure of a frame of images, it outputs a synchronization signal to trigger the exposure of each sub-camera at the same time, so that the frames output by all cameras are collected simultaneously, providing a basis for subsequent image synthesis.

In addition, it should be noted that the selection type of each camera can be determined according to the requirements of the final output image. For example, the final output image needs to be a more accurate depth map. Then the type of each camera can be a depth camera. The more requirements for output images, the more the number and type of cameras.

In step S02, image synthesis is performed on each of the sub-photograph images and one of the main-photograph images in turn, and one of the images is selected as a reference image in accordance with the first preset rule for each combination to synthesize multiple images once. Optimize the image.

The reference image refers to the image that is selected as the final optimized result from the two synthesized images. For example, the first image and the second image are synthesized. Assuming that the first image is selected as the reference image, the synthesized output is optimized. First image.

It should be noted that, because the main camera and each sub-camera have a common shooting area, the main image and each sub-image acquired simultaneously have the same image area, so each sub-camera can be combined with a The main shot image is image synthesized to optimize the same image area to obtain one optimized image at a time.

At the same time, images taken by different types of cameras have their own characteristics. For example, black and white cameras have the characteristics of low noise, and the combination of images taken by different types of cameras also produces images with different characteristics, such as black and white images taken by black and white cameras. The color camera captures and synthesizes color images to obtain low-noise color images to reduce noise in the image. For example, if two images containing depth-of-field information are combined, a depth-of-field image can be synthesized.

In specific implementation, the first preset rule may be any one of the following rules:

The rule for selecting an image with a larger field of view or a smaller field of view. For example, if the field of view of the first image is larger than the field of view of the second image, the first image is selected as the reference image, and the larger the field of view, the larger the image area;

Rules for selecting images captured by a specific camera, such as always selecting images captured by the main camera as the reference image;

Rules for selecting images with better image quality.

In step S03, according to a second preset rule, a plurality of the primary optimized images are image synthesized to synthesize a secondary optimized image.

It should be noted that the image synthesis in this step is mainly to integrate the characteristics of each primary optimized image onto a secondary optimized image to obtain a better image effect.

In summary, in the image processing method in the foregoing embodiment of the present invention, multiple sub cameras are arranged, and a common shooting area is provided between the main camera and each sub camera, so as to acquire a main image and multiple Sub-shooting images, and then synthesizing each sub-shooting image with a main-shooting image to obtain multiple optimized images at once. This time the combination is a dual-shot combination, which can synthesize multiple depth-of-field images and / or at least one clear image. Image and at least one noise reduction image, and then synthesizing multiple once-optimized images to output a second-optimized image. Because the synthesis of multiple depth-of-field images can achieve calibration of depth-of-field information, and clear images and noise reduction The images can be superimposed and synthesized with each other, so the image processing method, system, readable storage medium and terminal can output a more accurate depth of field map to avoid error blurring, and at the same time can output low noise and high definition images, which can improve the overall quality of the captured images. Picture quality.

Please refer to FIG. 2, which shows an image processing method according to a second embodiment of the present invention, which is applied to a terminal. The terminal is provided with a main camera and two sub-cameras, which are three cameras in total, and each camera is capable of shooting. A camera that can be used to synthesize images of the depth-of-field image is provided. All cameras are connected through a synchronization signal line, and the main camera and each of the sub cameras have a common shooting area. The image processing method includes steps S11 to Step S15.

Please refer to FIG. 3, which shows the arrangement of the cameras on the terminal in this embodiment. Specifically, the connection between the main camera and the two sub cameras is perpendicular to each other. This vertical arrangement can make The image information of the main camera and each sub-camera is more complementary and more beautiful. At the same time, all cameras and flashes are arranged in a square arrangement and are located on the four corners of the rectangle.

Step S11: Acquire a main image taken by the main camera, and acquire a sub-image taken by each of the sub-cameras synchronously.

In specific implementation, the pins connected to the synchronization signal line of the main camera can be used as the synchronization signal output terminal, and the pins connected to the synchronization signal line of each sub camera are used as the synchronization signal input terminal, and the frame rate of each camera can be the same.

It should be noted that, since each camera is a camera capable of capturing an image that can be used to synthesize a depth of field image, both the main camera image and the sub-camera obtained by synchronous shooting contain depth information.

In step S12, image synthesis is performed on each of the sub-photograph images and one of the main-photograph images in turn, and one of the images is selected as a reference image in accordance with the first preset rule for each combination to synthesize multiple images once. Optimize the image.

The one-time optimized image is a depth image, which is synthesized according to the depth information of each image.

Step S13: According to a second preset rule, select one of the primary optimized images as the first reference image.

In specific implementation, the second preset rule includes any one of the following rules: a rule that preferentially selects the image with the least abnormal depth information; a rule that preferentially selects the image with the largest field of view; and preferentially selects the image with the smallest field of view A rule that preferentially selects a composite image of images captured by a specific two cameras, for example, always selecting an optimized image that is a combination of images captured by camera A and camera B as the first reference image; or a rule that preferentially selects the image with the highest image quality.

Step S14: Obtain a target image region with abnormal depth information in the first reference image.

It should be noted that in the depth image, if there is an abnormal phenomenon such as incorrect depth of field information or depth of field information cannot be confirmed in a certain image area, it is usually highlighted in a specific way, such as a specific color, circle, etc., so it is displayed according to each area. In this way, the target image region with abnormal depth information in the first reference image can be obtained.

Step S15: Obtain target depth information of the same region as the target image region from the other optimized images, and use the target depth information to correct the depth information of the target image region to obtain a second optimization. image.

It can be understood that the purpose of this step is to correct the depth information of the corresponding area of the reference image by obtaining the depth information of the same image area in other images, so that the reference image becomes a more accurate depth map, and then output.

In addition, please refer to FIG. 4. In another embodiment, the terminal may also be provided with a main camera and three sub-cameras, which have four cameras as a whole, and each camera may be arranged in a square arrangement for all cameras. Arrangement, the flash is arranged at the center of the area surrounded by each camera. However, it should be noted that the image processing method in the present invention is not limited to three shots and four shots. In other embodiments, it may be five shots or more, which can be determined according to the requirements of the final output image.

In summary, the image processing method in the above embodiments of the present invention can realize self-calibration of the depth of field map during execution, and can output a more accurate depth of field map.

Please refer to FIG. 5, which shows an image processing method according to a third embodiment of the present invention, which is applied to a terminal. The terminal is provided with a main camera and two sub cameras. The main camera is a first color camera. The secondary cameras are respectively a black and white camera and a second color camera. There is a difference in the equivalent focal length of the first color camera and the second color camera. All cameras are connected through a synchronization signal line, and the main camera There is a common shooting area with each of the secondary cameras, and the image processing method includes steps S21 to S29.

Wherein, the arrangement manner of the cameras on the terminal in this embodiment may be arranged with reference to FIG. 3, and details are not described herein again.

Step S21: Acquire a main image taken by the main camera, and acquire a sub-image taken by each of the sub-cameras synchronously.

It should be noted that the first color camera and the second color camera are both color imaging cameras, and the captured images are color images. The color images generally have brightness information and chrominance information that can be split. Black and white cameras are black and white imaging cameras. The captured image is a black and white image, and the black and white image has only brightness information, low noise, and good image stability. Therefore, the main image and one of the sub-images are color images, and the other sub-image is a black and white image.

Step S22: According to a first preset rule, an image is selected as a reference image from a first color image captured by the first color camera and a second brightness signal image captured by the black and white camera.

Step S23: Split the first color image into a first luminance signal image and a first chrominance signal image.

Step S24: Based on the selected reference image, perform noise reduction synthesis on the first luminance signal image and the second luminance signal image to obtain a third luminance signal image after noise reduction.

In specific implementation, a process of performing noise reduction combining the first luminance signal image and the second luminance signal image captured by the black-and-white camera may be calculating the first luminance signal image and the second luminance signal image. The average pixel value of the same pixel point is then used as the final pixel value after the pixel point is synthesized, and the final pixel value is rendered on the reference image to obtain a third brightness signal image.

It should be noted that because the black and white image has only brightness information, the second brightness signal image is a black and white image taken by a black and white camera, and because the second brightness signal has low noise, the first brightness signal image and all After the second luminance signal image is synthesized, the obtained third luminance signal image has a characteristic of low noise.

In step S25, based on the selected reference image, the first chrominance signal image and the third luminance signal image are synthesized to obtain a one-time optimized image after noise reduction.

It can be understood that combining the first chrominance signal image and the third luminance signal image will obtain a color image. Because the third luminance signal image has low noise, a color image with low noise is obtained as a whole. To achieve image noise reduction.

Step S26: According to the first preset rule, select one image as a reference image from the first color image and the second color image captured by the second color camera;

Step S27: Determine the subject and background relationship between the first color image and the second color image according to the focal lengths of the first color camera and the second color camera.

It can be understood that the larger the focal length, the farther the camera shoots, which is usually used to capture the background, and the smaller the focal length, the closer the camera shoots, and it is usually used to capture the subject, so according to the first color camera and the second color camera The focal length of the lens can determine what the first color image and the second color image are as the subject image and what is the background image.

In step S28, based on the selected reference image and the determined relationship between the subject and the background, the first color image and the second color image are selectively blurred and synthesized to obtain a clear primary optimized image.

The selective blurring may be any one of a blurring background, a blurring subject, or a certain image area, which may be preset or determined by the terminal processor.

It should be noted that the main purpose of the above steps S22 to S25 is to synthesize the images captured by the first color camera and the black and white camera once to optimize the images. The above steps S26 to S28 are mainly for synthesizing the images captured by the first color camera and the second color camera into an optimized image. In this embodiment, steps S22-S25 are arranged and executed before steps S26-S28, but the present invention is not limited thereto. In other embodiments, steps S26-S28 can also be arranged and executed before steps S22-S25, or synchronized carried out.

Step S29: According to a second preset rule, image synthesis is performed on a plurality of the primary optimized images to synthesize a secondary optimized image.

The second preset rule may be: firstly selecting a one-time optimized image as a reference image, and then performing multiple pixel synthesis on the same image area for multiple one-time optimized images and rendering them to corresponding areas of the reference image, Get the final output image.

In summary, the image processing method in the above embodiments of the present invention can support effects such as optical zoom and color + black and white noise reduction during execution, and can output low noise and high definition images.

In addition, in another embodiment, the terminal may also be provided with a main camera and three sub-cameras, which are four cameras in total. The arrangement of each camera may be arranged with reference to FIG. 4, and is not described herein again. However, it should be noted that the image processing method in the present invention is not limited to three-shot and four-shot. In other embodiments, it may be five-shot or more, which can be determined according to the requirements of the final output image.

Another aspect of the present invention provides an image processing system. Please refer to FIG. 6, which shows an image processing system in a fourth embodiment of the present invention, which is applied to a terminal. The terminal is provided with a main camera and at least two cameras. Sub-cameras, all cameras are connected by a synchronization signal line, and the main camera and each of the sub-cameras have a common shooting area, the system includes:

An image acquisition module 11 configured to acquire a main image captured by the main camera, and respectively acquire a secondary image captured by each of the secondary cameras synchronously;

A one-time composition module 12 is configured to sequentially synthesize images of each of the sub-photograph images and one of the main-photograph images, and select one of the images as a reference image according to a first preset rule for each combination, and Synthesize multiple optimized images at once;

The secondary synthesis module 13 is configured to perform image synthesis on a plurality of the primary optimized images according to a second preset rule to synthesize a secondary optimized image.

Further, the one-time optimized image is a depth image.

Further, the secondary synthesis module 13 includes:

A reference selecting unit 131, configured to select one of the once optimized images as a first reference image according to the second preset rule;

An abnormality obtaining unit 132, configured to obtain a target image region in which the depth information in the first reference image is abnormal;

A depth calibration unit 133 is configured to obtain target depth information of the same area as the target image area from among the other optimized images, and use the target depth information to correct the depth information of the target image area.

Further, the second preset rule includes any one of the following rules: a rule that preferentially selects an image with the least abnormal depth information; a rule that preferentially selects an image with the largest field of view; a rule that preferentially selects an image with the smallest field of view ; Preferentially select the composite image of the images captured by the specific two cameras; or the rule of preferentially selecting the image with the highest image quality.

Further, the main camera is a first color camera, and all the sub cameras include at least a black and white camera and a second color camera, and there is a difference in the equivalent focal lengths of the first color camera and the second color camera. .

Further, the primary synthesis module 12 includes:

A first selecting unit 121, configured to select an image from a first color image captured by the first color camera and a second brightness signal image captured by the black and white camera according to the first preset rule As a reference image;

An image splitting unit 122, configured to split the first color image into a first luminance signal image and a first chrominance signal image;

A first combining unit 123, configured to perform noise reduction synthesis on the first brightness signal image and the second brightness signal image to obtain a third brightness signal image after noise reduction;

The second synthesizing unit 124 is configured to synthesize the first chrominance signal image and the third luminance signal image to obtain the primary optimized image after noise reduction.

Further, the one-time synthesis module 12 further includes:

The second selecting unit 125 is configured to select one of a first color image captured by the first color camera and a second color image captured by the second color camera according to the first preset rule. Image as the reference image;

A relationship determining unit 126, configured to determine a subject and background relationship between the first color image and the second color image according to a focal length of the first color camera and the second color camera;

An image blurring unit 127 is configured to selectively blur the first color image and the second color image based on the selected reference image and the determined relationship between the subject and the background, so as to obtain a clear image. Describe the optimized image once.

Further, the arrangement manner of the cameras on the terminal is any one of the following situations:

When the two secondary cameras are provided on the terminal, the connection between the main camera and the two secondary cameras is perpendicular to each other;

When three of the secondary cameras are provided on the terminal, all cameras are arranged in a rectangular arrangement, and four cameras are located at four corner points of the rectangle.

Further, a pin connected to the synchronization signal line of the main camera is used as a synchronization signal output terminal, a pin connected to each synchronization signal line of each of the sub cameras is used as a synchronization signal input terminal, and the frame rate of each camera is the same.

The present invention also proposes a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the image processing method as described above is implemented.

The present invention also provides a terminal including a memory, a processor, and a computer program stored on the memory and executable on the processor. The terminal is provided with a main camera and at least two sub cameras, and all cameras pass a synchronization signal. And the shooting area of the main camera is equal to or includes the shooting area of the sub camera, and the processor implements the method as described above when the processor executes the program.

It can be understood that the terminal includes, but is not limited to, a mobile phone, a computer, a tablet, a smart TV, a security device, a smart wearable device, and the like.

Those skilled in the art may understand that the logic and / or steps represented in the flowchart or described herein in other ways, for example, may be considered as a ordered list of executable instructions for implementing logical functions, may be specifically implemented in Any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, device, or device), or Used in conjunction with these instruction execution systems, devices or equipment. For the purposes of this specification, a "computer-readable medium" may be any device that can contain, store, communicate, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device.

More specific examples (non-exhaustive list) of computer readable media include the following: electrical connections (electronic devices) with one or more wirings, portable computer disk cartridges (magnetic devices), random access memory (RAM), Read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program can be printed, because, for example, by optically scanning the paper or other medium, followed by editing, interpretation, or other suitable Processing to obtain the program electronically and then store it in computer memory.

It should be understood that each part of the present invention may be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods may be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it may be implemented using any one or a combination of the following techniques known in the art: Discrete logic circuits, application specific integrated circuits with suitable combinational logic gate circuits, programmable gate arrays (PGA), field programmable gate arrays (FPGA), etc.

In the description of this specification, the description with reference to the terms “one embodiment”, “some embodiments”, “examples”, “specific examples”, or “some examples” and the like means specific features described in conjunction with the embodiments or examples , Structures, materials, or features are included in at least one embodiment or example of the invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

The above-mentioned embodiments only express several implementation manners of the present invention, and their descriptions are more specific and detailed, but they cannot be understood as limiting the scope of the patent of the present invention. It should be noted that, for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can be made, which all belong to the protection scope of the present invention. Therefore, the protection scope of the invention patent shall be subject to the appended claims.

The embodiment of the invention also discloses:

A1. An image processing method is applied to a terminal. The terminal is provided with a main camera and at least two sub cameras, all cameras are connected by a synchronization signal line, and between the main camera and each of the sub cameras Both have a common shooting area, the method includes:

Acquiring a main image taken by the main camera, and separately acquiring a sub-image taken synchronously by each of the sub-cameras;

Synthesizing each of the sub-photograph images and one of the main-photograph images in turn, and selecting one of the images as a reference image in accordance with a first preset rule for each combination to synthesize multiple optimized images at once;

According to a second preset rule, a plurality of the primary optimized images are image synthesized to synthesize a secondary optimized image.

A2. The image processing method according to A1, wherein the one-time optimized image is a depth image.

A3. The image processing method according to A2, wherein the step of performing image synthesis on the plurality of optimized images at one time includes:

Selecting one of the primary optimized images as the first reference image according to the second preset rule;

Obtaining a target image region with abnormal depth information in the first reference image;

Acquiring target depth information of the same region as the target image region from other one-time optimized images, and using the target depth information to correct the depth information of the target image region.

A4. The image processing method according to A3, the second preset rule includes any one of the following rules:

Rules for preferentially selecting images with the least amount of abnormal depth information;

Rules for preferentially selecting the image with the largest field of view;

Rules for preferentially selecting the image with the smallest field of view;

Preferentially select a composite image of images captured by a particular two cameras; or

The rule that preferentially selects the image with the highest image quality.

A5. The image processing method according to A1, the main camera is a first color camera, and all the sub cameras include at least a black and white camera and a second color camera, the first color camera and the second camera There is a difference in the equivalent focal length of color cameras.

A6. The image processing method according to A5, the step of synthesizing the images optimized by the first color camera and the black and white camera into the one-time optimized image includes:

Selecting one image as a reference image from the first color image captured by the first color camera and the second brightness signal image captured by the black and white camera according to the first preset rule;

Splitting the first color image into a first luminance signal image and a first chrominance signal image;

Performing noise reduction synthesis on the first brightness signal image and the second brightness signal image based on the selected reference image to obtain a third brightness signal image after noise reduction;

Combining the first chrominance signal image and the third luminance signal image to obtain the primary optimized image after noise reduction.

A7. The image processing method according to A5, the step of synthesizing the images optimized by the first color camera and the second color camera into the one-time optimized image includes:

Selecting one image from the first color image captured by the first color camera and the second color image captured by the second color camera as a reference image according to the first preset rule;

Determining a subject and background relationship between the first color image and the second color image according to a focal length of the first color camera and the second color camera;

Selectively blurring the first color image and the second color image based on the selected reference image and the determined relationship between the subject and the background to obtain a clear first-time optimized image.

A8. According to the image processing method of A1, the arrangement manner of the cameras on the terminal is any one of the following situations:

When the two secondary cameras are provided on the terminal, the connection between the main camera and the two secondary cameras is perpendicular to each other;

When three of the secondary cameras are provided on the terminal, all cameras are arranged in a rectangular arrangement, and four cameras are located at four corner points of the rectangle.

A9. The image processing method according to A1, wherein a pin connected to the synchronization signal line of the main camera is used as a synchronization signal output terminal, and a pin connected to the synchronization signal line of each of the sub cameras is used as a synchronization signal input. And the frame rate of each camera is the same.

B10. An image processing system applied to a terminal, the terminal is provided with a main camera and at least two sub cameras, all cameras are connected by a synchronization signal line, and between the main camera and each of the sub cameras Both have a common shooting area, and the system includes:

An image acquisition module, configured to acquire a main image captured by the main camera, and respectively acquire a secondary image captured by each of the secondary cameras synchronously;

A one-time composition module, configured to sequentially synthesize each of the sub-photograph images and one of the main-photograph images, and select one of the images as a reference image according to a first preset rule for each composition to synthesize Optimize multiple images at once;

A secondary synthesis module is configured to perform image synthesis on a plurality of the primary optimized images according to a second preset rule to synthesize one secondary optimized image.

B11. The image processing system according to B10, wherein the one-time optimized image is a depth image.

B12. The image processing system according to B11, wherein the secondary synthesis module includes:

A reference selecting unit, configured to select one of the once optimized images as a first reference image according to the second preset rule;

An abnormality obtaining unit, configured to obtain a target image region in which depth information is abnormal in the first reference image;

A depth calibration unit is configured to obtain target depth information of the same area as the target image area from among the other optimized images, and use the target depth information to correct the depth information of the target image area.

B13. The image processing system according to B12, wherein the second preset rule includes any one of the following rules:

Rules for preferentially selecting images with the least amount of abnormal depth information;

Rules for preferentially selecting the image with the largest field of view;

Rules for preferentially selecting the image with the smallest field of view;

Preferentially select a composite image of images captured by a particular two cameras; or

The rule that preferentially selects the image with the highest image quality.

B14. The image processing system according to B10, the main camera is a first color camera, and all the sub cameras include at least a black and white camera and a second color camera, the first color camera and the second camera There is a difference in the equivalent focal length of color cameras.

B15. The image processing system according to B14, wherein the one-time synthesis module includes:

A first selecting unit, configured to select an image from a first color image captured by the first color camera and a second brightness signal image captured by the black and white camera according to the first preset rule Reference image

An image splitting unit, configured to split the first color image into a first luminance signal image and a first chrominance signal image;

A first synthesis unit, configured to perform noise reduction synthesis on the first brightness signal image and the second brightness signal image to obtain a third brightness signal image after noise reduction;

A second synthesizing unit is configured to synthesize the first chrominance signal image and the third luminance signal image to obtain the primary optimized image after noise reduction.

B16. The image processing system according to B14, the one-time synthesis module further includes:

A second selection unit, configured to select an image from a first color image captured by the first color camera and a second color image captured by the second color camera according to the first preset rule As a reference image;

A relationship determining unit, configured to determine a subject and background relationship between the first color image and the second color image according to a focal length of the first color camera and the second color camera;

An image blurring unit, configured to selectively blur the first color image and the second color image based on the selected reference image and the determined relationship between the subject and the background, so as to obtain a clear image Optimize the image at once.

B17. The image processing system according to B10, wherein the arrangement manner of the cameras on the terminal is any one of the following situations:

When the two secondary cameras are provided on the terminal, the connection between the main camera and the two secondary cameras is perpendicular to each other;

When three of the secondary cameras are provided on the terminal, all cameras are arranged in a rectangular arrangement, and four cameras are located at four corner points of the rectangle.

B18. The image processing system according to B10, wherein a pin connected to the synchronization signal line of the main camera is used as a synchronization signal output terminal, and a pin connected to the synchronization signal line of each of the sub cameras is used as a synchronization signal input. And the frame rate of each camera is the same.

C19. A computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method according to any of A1 to A9.

D20. A terminal, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor. The terminal is provided with a main camera and at least two sub cameras, and all cameras are connected through a synchronization signal line And the main camera and each of the sub cameras have a common shooting area, and when the processor executes the program, the method according to any one of A1 to A9 is implemented.

Claims (20)

1. An image processing method is applied to a terminal, wherein the terminal is provided with a main camera and at least two sub cameras, all cameras are connected by a synchronization signal line, and between the main camera and each of the sub cameras Both have a common shooting area, the method includes:

   Acquiring a main image taken by the main camera, and separately acquiring a sub-image taken synchronously by each of the sub-cameras;

   Synthesizing each of the sub-photograph images and one of the main-photograph images in turn, and selecting one of the images as a reference image in accordance with a first preset rule for each combination to synthesize multiple optimized images at once;

   According to a second preset rule, a plurality of the primary optimized images are image synthesized to synthesize a secondary optimized image.
2. The image processing method according to claim 1, wherein the one-time optimized image is a depth image.
3. The image processing method according to claim 2, wherein the step of performing image synthesis on a plurality of the optimized images at a time comprises:

   Selecting one of the primary optimized images as the first reference image according to the second preset rule;

   Obtaining a target image region with abnormal depth information in the first reference image;

   Acquiring target depth information of the same region as the target image region from other one-time optimized images, and using the target depth information to correct the depth information of the target image region.
4. The image processing method according to claim 3, wherein the second preset rule comprises any one of the following rules:

   Rules for preferentially selecting images with the least amount of abnormal depth information;

   Rules for preferentially selecting the image with the largest field of view;

   Rules for preferentially selecting the image with the smallest field of view;

   Preferentially select a composite image of images captured by a particular two cameras; or

   The rule that preferentially selects the image with the highest image quality.
5. The image processing method according to claim 1, wherein the main camera is a first color camera, and all the sub cameras include at least a black and white camera and a second color camera, the first color camera and the second camera. There is a difference in the equivalent focal length of the second color camera.
6. The image processing method according to claim 5, wherein the step of synthesizing the images captured by the first color camera and the black and white camera into the one-time optimized image comprises:

   Selecting one image as a reference image from the first color image captured by the first color camera and the second brightness signal image captured by the black and white camera according to the first preset rule;

   Splitting the first color image into a first luminance signal image and a first chrominance signal image;

   Performing noise reduction synthesis on the first brightness signal image and the second brightness signal image based on the selected reference image to obtain a third brightness signal image after noise reduction;

   Combining the first chrominance signal image and the third luminance signal image to obtain the primary optimized image after noise reduction.
7. The image processing method according to claim 5, wherein the step of synthesizing the images captured by the first color camera and the second color camera into the one-time optimized image comprises:

   Selecting one image from the first color image captured by the first color camera and the second color image captured by the second color camera as a reference image according to the first preset rule;

   Determining a subject and background relationship between the first color image and the second color image according to a focal length of the first color camera and the second color camera;

   Selectively blurring the first color image and the second color image based on the selected reference image and the determined relationship between the subject and the background to obtain a clear first-time optimized image.
8. The image processing method according to claim 1, wherein an arrangement manner of each camera on the terminal is any one of the following situations:

   When the two secondary cameras are provided on the terminal, the connection between the main camera and the two secondary cameras is perpendicular to each other;

   When three of the secondary cameras are provided on the terminal, all cameras are arranged in a rectangular arrangement, and four cameras are located at four corner points of the rectangle.
9. The image processing method according to claim 1, wherein a pin connected to the synchronization signal line of the main camera is used as a synchronization signal output terminal, and a pin connected to the synchronization signal line of each of the sub cameras is used as synchronization Signal input, and the frame rate of each camera is the same.
10. An image processing system is applied to a terminal, wherein the terminal is provided with a main camera and at least two sub cameras, all cameras are connected through a synchronization signal line, and between the main camera and each of the sub cameras Both have a common shooting area, and the system includes:

    An image acquisition module, configured to acquire a main image captured by the main camera, and respectively acquire a secondary image captured by each of the secondary cameras synchronously;

    A one-time composition module, configured to sequentially synthesize each of the sub-photograph images and one of the main-photograph images, and select one of the images as a reference image according to a first preset rule for each composition to synthesize Optimize multiple images at once;

    A secondary synthesis module is configured to perform image synthesis on a plurality of the primary optimized images according to a second preset rule to synthesize one secondary optimized image.
11. The image processing system according to claim 10, wherein the one-time optimized image is a depth image.
12. The image processing system according to claim 11, wherein the secondary synthesis module comprises:

    A reference selecting unit, configured to select one of the once optimized images as a first reference image according to the second preset rule;

    An abnormality obtaining unit, configured to obtain a target image region in which depth information is abnormal in the first reference image;

    A depth calibration unit is configured to obtain target depth information of the same area as the target image area from among the other optimized images, and use the target depth information to correct the depth information of the target image area.
13. The image processing system according to claim 12, wherein the second preset rule comprises any one of the following rules:

    Rules for preferentially selecting images with the least amount of abnormal depth information;

    Rules for preferentially selecting the image with the largest field of view;

    Rules for preferentially selecting the image with the smallest field of view;

    Preferentially select a composite image of images captured by a particular two cameras; or

    The rule that preferentially selects the image with the highest image quality.
14. The image processing system according to claim 10, wherein the main camera is a first color camera, and all the sub cameras include at least a black and white camera and a second color camera, the first color camera and the second camera. There is a difference in the equivalent focal length of the second color camera.
15. The image processing system according to claim 14, wherein the one-time synthesis module comprises:

    A first selecting unit, configured to select an image from a first color image captured by the first color camera and a second brightness signal image captured by the black and white camera according to the first preset rule Reference image

    An image splitting unit, configured to split the first color image into a first luminance signal image and a first chrominance signal image;

    A first synthesis unit, configured to perform noise reduction synthesis on the first brightness signal image and the second brightness signal image to obtain a third brightness signal image after noise reduction;

    A second synthesizing unit is configured to synthesize the first chrominance signal image and the third luminance signal image to obtain the primary optimized image after noise reduction.
16. The image processing system according to claim 14, wherein the one-time synthesis module further comprises:

    A second selection unit, configured to select an image from a first color image captured by the first color camera and a second color image captured by the second color camera according to the first preset rule As a reference image;

    A relationship determining unit, configured to determine a subject and background relationship between the first color image and the second color image according to a focal length of the first color camera and the second color camera;

    An image blurring unit, configured to selectively blur the first color image and the second color image based on the selected reference image and the determined relationship between the subject and the background, so as to obtain a clear image Optimize the image at once.
17. The image processing system according to claim 10, wherein an arrangement manner of each camera on the terminal is any one of the following situations:

    When the two secondary cameras are provided on the terminal, the connection between the main camera and the two secondary cameras is perpendicular to each other;

    When three of the secondary cameras are provided on the terminal, all cameras are arranged in a rectangular arrangement, and four cameras are located at four corner points of the rectangle.
18. The image processing system according to claim 10, wherein a pin connected to the synchronization signal line of the main camera is used as a synchronization signal output terminal, and a pin connected to the synchronization signal line of each of the sub cameras is used as synchronization. Signal input, and the frame rate of each camera is the same.
19. A computer-readable storage medium having stored thereon a computer program, wherein when the program is executed by a processor, the method according to any one of claims 1-9 is implemented.
20. A terminal includes a memory, a processor, and a computer program stored on the memory and executable on the processor. The terminal is provided with a main camera and at least two sub cameras, and all cameras are connected through a synchronization signal line. And the main camera and each of the sub cameras have a common shooting area, and when the processor executes the program, the method according to any one of claims 1-9 is implemented.

Images