KR102278474B1 - Multi-camera module-based image processing method, device, device and medium - Google Patents

Abstract

An embodiment of the present invention provides a multi-camera module-based image processing method, apparatus, device, and medium, and in this embodiment, the first camera module is in the current main shooting state, and the target in the framing area of the first camera module is When the subject to be photographed does not exist in the initial framing area of the second camera module, the framing field of view of the second camera module is adjusted so that the adjusted framing area of the second camera module covers the target photographing object, and the second camera module The sub-image collected by , and the main image collected by the first camera module are fused.

Description

Multi-camera module-based image processing method, device, device and medium

Cross-referencing of related applications

This application is filed based on a Chinese patent application with an application number of 201910855143.3 and an filing date of September 10, 2019, claiming priority to the Chinese patent application, and citing the entire contents of the Chinese patent application in this application. Reference.

The present application relates to the field of terminal technology, and more particularly, to a multi-camera module-based image processing method, apparatus, device, and medium.

With the rapid development of terminal technology, all smart terminals such as smart phones and tablet computers are configured with cameras, and through the cameras configured in mobile terminals, the smart terminals can take images. In order to improve the quality of the captured image, some smart terminals are configured with multiple cameras, and the most common one is that the smart terminals have dual cameras. The smart terminal may select one camera from the dual cameras according to a shooting scenario to take an image. For example, the dual cameras configured in the smart terminal are a wide-angle camera and a long-focus camera, respectively. However, since different cameras are used for different shooting scenarios, images captured by a camera having a wide shooting range may have non-ideal image quality.

In order to solve the problems existing in the related art, an embodiment of the present invention provides an image processing method, apparatus, apparatus and medium based on a multi-camera module.

According to a first aspect of an embodiment of the present invention, there is provided an image processing method based on a multi-camera module, the method comprising:

When the first camera module is in the current main shooting state, and the target shooting target in the framing area of the first camera module does not exist in the initial framing area of the second camera module, by adjusting the framing field of the second camera module , allowing the adjusted framing area of the second camera module to cover the target photographing object, wherein the viewing angle of the second camera module is smaller than the viewing angle of the first camera module; and

and fusing the sub-image and the main image, wherein the sub-image is an image collected by the second camera module, and the main image is an image collected by the first camera module.

In an optional embodiment, adjusting the framing field of view of the second camera module comprises:

determining a current framing field of view of the second camera module; and

adjusting the current framing field of view of the second camera module to a maximum range through an Optical Image Stabilizer (OIS);

Here, the target shooting target is an arbitrary shooting point in the non-overlapping part of the maximum framing area and the initial framing area after adjustment of the second camera module.

In an optional embodiment, the target photographing object is an object of interest in a framing area, and adjusting the framing field of view of the second camera module comprises:

determining a target direction according to a positional relationship between the target shooting target and the initial framing area; and

and adjusting the second camera module to operate along the target direction through OIS.

In an optional embodiment, the target shooting target is a focus point in the framing area of the first camera module, a focus area determined by the focus point, a designated partial area in the framing area of the first camera module, or the and a framing area of the first camera module.

In an optional embodiment, the second camera module comprises one camera; the step of allowing the framing area of the second camera module to cover the target photographing object includes the step of allowing the target photographing object to exist in the same framing area or a specific position in the same framing area of the second camera module; The specific position is a preset position or a position determined by a detected manipulation signal.

In an optional embodiment, the second camera module includes one camera, and the size of the target shooting target is larger than the size of the framing area of the second camera module; the target direction includes at least two sub-target directions;

The step of adjusting the second camera module to operate along the target direction through the OIS includes, in each sub-target direction, adjusting the second camera module to operate along the sub-target direction through the OIS and;

the target shooting object is in a first stitching framing area, the first stitching framing area is obtained by stitching the adjusted framing areas obtained by all secondary adjustments;

The sub-images collected by the second camera module include sub-images collected after the second camera module is adjusted according to the sub-target direction each time.

In one optional embodiment, the second camera module includes at least two cameras, the size of the target shooting target is larger than the size of the framing area of the second camera module; the target direction includes at least two sub-target directions;

The step of adjusting the second camera module to operate along the target direction through the OIS includes adjusting each of the cameras in the second camera module to operate along the corresponding sub-target direction through the OIS and;

the target shooting object exists in a second stitching framing area, and the second stitching framing area is obtained by stitching the adjusted framing area of each camera;

The sub-images collected by the second camera module include images collected by the camera after all adjustments.

In an optional embodiment, the method comprises:

When the first camera module is in the current main shooting state and the target shooting target in the framing area of the first camera module does not exist in the initial framing area of the second camera module, the sub-image collected by the second camera module and It further includes the step of fusion processing the main image by the first camera module.

In an optional embodiment, the fusion processing step is executed during a framing process or after a shooting operation command is received.

In an optional embodiment, the multi-camera module includes a wide-angle camera module, a general-purpose camera module, and a long-focus camera module whose viewing angle is gradually reduced.

According to a second aspect of an embodiment of the present invention, there is provided an image processing apparatus based on a multi-camera module, the apparatus comprising:

When the first camera module is in the current main shooting state, and the target shooting target in the framing area of the first camera module does not exist in the initial framing area of the second camera module, by adjusting the framing field of the second camera module , a field of view adjustment module configured such that the adjusted framing area of the second camera module covers the target photographing object, wherein the viewing angle of the second camera module is smaller than the viewing angle of the first camera module; and

and a fusion processing module configured to fuse the sub-image and the main image, wherein the sub-image is the image collected by the second camera module, and the main image is the image collected by the first camera module.

According to a third aspect of an embodiment of the present invention, there is provided an electronic device based on a multi-camera module, the electronic device comprising:

at least two camera modules having different viewing angles;

processor; and

the processor includes a memory configured to store executable instructions;

Here, the processor,

When the first camera module is in the current main shooting state, and the target shooting target in the framing area of the first camera module does not exist in the initial framing area of the second camera module, by adjusting the framing field of the second camera module , such that the adjusted framing area of the second camera module covers the target shooting object, wherein the viewing angle of the second camera module is smaller than the viewing angle of the first camera module;

and fusing the sub-image and the main image, wherein the sub-image is an image collected by the second camera module, and the main image is an image collected by the first camera module.

According to a fourth aspect of an embodiment of the present invention, there is provided a computer readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the steps of the method of any one of the above are implemented.

The technical solution provided in the embodiment of the present invention may include the following advantageous effects.

In an embodiment of the present invention, when the first camera module is in the current main shooting state and the target shooting target in the framing area of the first camera module does not exist in the initial framing area of the second camera module, the second camera module By adjusting the framing field of view, so that the adjusted framing area of the second camera module covers the target shooting target, the sub-image collected by the second camera module and the main image collected by the first camera module are fused, Image quality improvement effect can be achieved, and furthermore, by adjusting the second camera module, because the shooting range of the second camera module is expanded, the framing area obtained after adjustment can cover an area not included in the initial framing area. By doing so, it is possible to implement image quality improvement for at least a part of an image area in which image quality cannot be improved.

It should be understood that the above general description and the following detailed description are merely illustrative and interpretative, and do not limit the embodiments of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings are incorporated in and constitute a part of this specification, and illustrate embodiments consistent with the present invention, and together with the specification serve to interpret the principles of the embodiments of the present invention.
1 is a representation of each camera framing area on the screen.
Fig. 2 is a flowchart of an image processing method based on a multi-camera module shown according to an exemplary embodiment of the present invention.
Fig. 3 is an exemplary diagram of the largest framing area shown according to an exemplary embodiment of the present invention.
4 is an exemplary diagram of a focus point shown according to an exemplary embodiment of the present invention.
Fig. 5 is a diagram illustrating a comparison before and after adjustment of a framing area shown according to an exemplary embodiment of the present invention.
Fig. 6A is an exemplary diagram of a first stitched framing area shown according to an exemplary embodiment of the present invention.
6B is an exemplary diagram of a second stitching framing area shown according to an exemplary embodiment of the present invention.
7 is a flowchart of another multi-camera module-based image processing method according to an exemplary embodiment of the present invention.
8 is a block diagram of a multi-camera module-based image processing apparatus according to an exemplary embodiment of the present invention.
9 is a structural diagram of an image processing apparatus based on a multi-camera module shown according to an exemplary embodiment of the present invention.

Exemplary embodiments are described in detail below, examples of which are shown in the drawings. When referring to drawings in the description below, the same numbers in different drawings indicate the same or similar elements, unless otherwise indicated. The embodiments described in the exemplary embodiments below do not represent all embodiments consistent with the embodiments of the present invention. To the contrary, these are merely examples of apparatus and methods consistent with some aspects of embodiments of the invention, as detailed in the claims.

The terms used in the embodiments of the present invention are only for describing specific embodiments, and are not intended to limit the embodiments of the present invention. As used in the examples and claims of the present invention, the singular forms “a”, “the” and “therefore” also include the plural forms, unless the text clearly indicates otherwise. The term “and/or” as used herein means and includes any or all possible combinations of one or more related listed items.

It should be understood that although various information is described using terms such as “first”, “second”, “third” and the like in the embodiments of the present invention, such information should not be limited to these terms. These terms are only used to distinguish each other using the same type of information. For example, without departing from the scope of the embodiments of the present invention, first information may be referred to as second information, and likewise, second information may also be referred to as first information. This is determined according to the word usage context, for example, the word "if" used herein may be interpreted as meaning "when" or "when" or "in response to a decision".

Currently, in everyday life, more and more users are accustomed to taking images using terminals such as smart phones, tablet computers, and the like, and at the same time, demands for the quality of images taken by terminals are also increasing. With the development of camera hardware technology and image processing technology, in order to further satisfy the needs of users, multiple cameras have been configured in some terminals. Here, each of the multiple cameras may have its own advantages. For example, one camera is more suitable for shooting one scenario, and the other camera is more suitable for shooting another scenario. Accordingly, in the case of image capturing, the terminal can select different cameras from multiple cameras according to different shooting scenarios to take pictures, so that images of different scenarios can be captured with the same terminal. In order to cope with different shooting scenarios, the viewing angle of the camera is often different, that is, the viewing range is different, and accordingly, the framing field of view is also different. The image quality of images taken by cameras having different viewing angles may be different.

Based on this, an embodiment of the present invention proposes an image processing method based on a multi-camera module. In a general situation, the framing range (photographing range) of the camera module is fixed, but in the embodiment of the present invention, the framing range of the camera module can be extended by adjusting the framing area of the camera module. When the first camera module is in the current main shooting state and the target shooting target in the framing area of the first camera module does not exist in the initial framing area of the second camera module, by adjusting the framing field of view of the second camera module, The adjusted framing area of the second camera module covers the target shooting target, and the sub-image collected by the second camera module and the main image collected by the first camera module are fused to improve image quality. Further, by adjusting the second camera module, since the shooting range of the second camera module is expanded, the framing area obtained after adjustment can cover an area not included in the initial framing area, so that the image quality It is possible to implement image quality enhancement for at least a part of an image area that cannot be enhanced.

The multi-camera module-based image processing method provided in this embodiment may be implemented through software, may be implemented in a form executed by a combination of software and hardware or hardware, and the mentioned hardware may be two or a plurality of physical entities. may be formed by, or may be formed by one physical entity. Illustratively, the method of this embodiment may be executed by photographing software in an electronic device. Here, the electronic device may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a personal digital assistant (PDA), a wearable device, or the like.

The camera module may be configured in an electronic device to form an integrated device with the electronic device; The camera module may also be mounted on an electronic device as an independent device. The multi-camera module may be at least two camera modules. The camera modules are configured to collect images, and each camera module may include one camera, and may also include a plurality of cameras. For example, the multi-camera module includes one or various modules of a wide-angle camera module, a long-focus camera module, a general-purpose camera module, a color camera, a black-and-white camera, and the like. Here, the general-purpose camera module may also be referred to as a main camera module configured to photograph in an electronic device, and the general-purpose camera module includes a main camera. In an embodiment of the present invention, in order to improve the image collected by the camera module having a large viewing angle by using the image collected by the camera module having a small viewing angle, multiple camera modules may be camera modules having different viewing angles, so that the image achieve the effect of improving the picture quality. For example, the multi-camera module may be three camera modules including a wide-angle photographing (DW) module, a general-purpose camera (W) module, and a long-focus camera (Tele) module in which the viewing angle is gradually reduced. For example, the field of view (FOV) of the wide-angle camera in the wide-angle camera module may be 120°, and the pixels may be 13M-48M; The viewing angle of the universal camera of the universal camera module may be 80°, and the pixels may be 48M-64M; The viewing angle of the long focus camera module may be 30° to 20°, and the pixels may be 13M-8M. The representation of the framing area of each camera on the screen is as shown in Fig. 1, the wide-angle camera has the largest viewing angle, the general-purpose camera (main camera) has the next, and the long-focus camera has the smallest viewing angle, Accordingly, the size of the framing area is also gradually reduced. It may be understood that FIG. 1 is only for explaining a camera having different viewing angles in which differences in framing areas exist, and in fact, the proportional relationship between the different framing areas may be determined according to an actual situation.

In some application scenarios, when shooting with different camera modules, an object of interest when shooting with one camera module may or may not exist within the viewing range of the other camera module, To this end, the embodiment of the present invention prevents an abnormal situation in which the fusion effect caused by the object of interest does not exist within the viewing range of the other camera module, and adjusts the framing area of the camera module, Make the framing area cover the target image that could not be covered originally.

Referring to Fig. 2, Fig. 2 is a flowchart of an image processing method based on a multi-camera module shown according to an exemplary embodiment of the present invention, and the method includes the following steps.

In step 202, when the first camera module is in the current main shooting state and the target shooting target in the framing area of the first camera module does not exist in the initial framing area of the second camera module, the By adjusting the framing field of view, the adjusted framing area of the second camera module covers the target photographing object, and the viewing angle of the second camera module is smaller than the viewing angle of the first camera module.

In step 204, a sub-image and a main image are fused, wherein the sub-image is an image collected by the second camera module, and the main image is an image collected by the first camera module.

In the above embodiment, when one camera module is selected to be photographed, the other camera module having a smaller viewing angle is used as a sub camera module, thereby improving the image forming effect. In order to distinguish the two modules, the camera module selected to shoot may be referred to as a first camera module, and the other camera module having a smaller viewing angle may be referred to as a second camera module. When the first camera module is selected to shoot, the first camera module may be regarded as the main shooting state. When the second camera module is used as a sub-capturing module, the second camera module may be regarded as a sub-capturing state. Accordingly, the image collected by the first camera module may be referred to as a main image, and the image collected by the second camera module may be referred to as a sub-image.

Taking the wide-angle camera module, the general-purpose camera module, and the long-focus camera module as an example, in a general situation, if the viewing angle is arranged in decreasing order, it is a wide-angle camera module, a general-purpose camera module, and a long-focus camera module. Accordingly, when the wide-angle camera module is used as the first camera module, the second camera module may be a general-purpose camera module or even a long-focus camera module. When the general-purpose camera module is used as the first camera module, the second camera module may be a long focus camera module.

When the first camera module is currently in the main photographing state, a target photographing target in the framing area of the first camera module may be determined. The target photographing target may be a photographing point, a photographing area, or the like. Exemplarily, the target photographing object is an object of interest in the framing area. The target shooting target in the framing area of the first camera module may be one or a plurality of targets.

Regarding the adjustment of the framing field of view of the second camera module, the adjustment may be such that the adjusted framing area of the second camera module covers the target photographing object.

In some scenarios, an optical image stabilization device is set in the image collection device of the mobile device to solve the problem of blurring of the captured image due to shake when holding the camera. The optical image stabilization device may be an image stabilization controller in a mobile device, may also be referred to as an OIS controller, detects a fine movement through a sensor in the camera lens, and then sends a signal to a microprocessor, which the microprocessor needs to correct By calculating the displacement amount and correcting the “optical axis offset” by controlling the optical lens float on the principle of further correction according to the direction and amount of displacement of the lens through a correction lens set, blur caused by shaking of mobile devices Video can be effectively overcome. Accordingly, as an example, the framing field of view of the second camera module may be adjusted through OIS.

For example, in order to expand the frameable range of the camera, the OIS of the camera can be designed at a large angle. For example, since the OIS of the camera in the second camera module can be designed at a large angle, the framing range of the second camera module (a range corresponding to a set of a plurality of framing areas) is the framing range of the first camera module. implemented to cover it. Taking the long-focal camera module as an example, Tele's frameable range can be corrected through OIS, so that the Tele's frameable range becomes larger than the original, for example, 3%-5%.

In the above embodiment, when the framing field of view of the second camera module is adjusted through the OIS and the framing range of the second camera module is extended, the cost can be reduced, and there is no need to add other hardware.

It should be understood that the framing field of view of the second camera module can also be adjusted by other means, for example, in some terminals, since the lens of the second camera module is rotatable, rotating the lens when the terminal is not moving Through this, the framing field of view of the second camera module may be adjusted. In the case of other adjustment means, since the second camera module only needs to be able to photograph the target photographing object, the description will not be repeated here any longer. When the framing field of view of the second camera module is further adjusted, the framing field of view of the first camera module may be maintained unchanged.

In one embodiment, the adjusting the framing field of view of the second camera module comprises: determining a current framing field of view of the second camera module; and adjusting the current framing field of view of the second camera module to a maximum range through OIS. The target shooting target is an arbitrary shooting point of a non-overlapping portion of the adjusted maximum framing area and the initial framing area of the second camera module.

Here, the maximum range is the maximum range that can be adjusted by the OIS. For example, in order to maximally cover the framing area of another camera with a larger viewing angle, the camera's OIS may be designed with a large angle.

The maximum framing area is the set of framing areas of the second camera module after each adjustment by OIS. The maximum framing area is determined by the adjustment width of the OIS and the framing range of the second camera module, and is formed, for example, as a set of adjusted framing areas of the second camera module after M times adjustment. As shown in Fig. 3, this is an exemplary view of the largest framing area shown according to an exemplary embodiment. In the above illustration, taking M=4 as an example, a solid line block is an initial framing area. Make 4 adjustments, with each adjustment you get one adjusted framing area, indicated by the dashed block. The four adjusted framing regions form the largest framing region and are like the four dashed-line blocks in FIG. 3 . It should be understood that M is not limited to 4 and may be any other number. Also, there may or may not be overlapping portions between the adjusted framing regions in the aggregation, as determined by the adjustment width. 3 exemplifies a case in which the maximum framing area is smaller than the framing area of the first camera module, and in some scenarios, the maximum framing area may be greater than or equal to the framing area of the first camera module. Do not duplicate explanations.

Correspondingly, in order to implement improvement of the main image quality by using one or more frames, the sub-images mentioned in the fusion process include the sub-images collected by the adjusted second camera module each time.

The above embodiment may directly adjust the current framing field of view of the second camera module to the maximum range through OIS, and obtain the maximum framing area to realize maximum improvement of image quality.

In another embodiment, it is also possible to adjust the framing field of view of the second camera module to move toward the target photographing object through the OIS, so that the adjusted framing area of the second camera module covers the target photographing object.

For example, the target photographing object may be an object of interest in the framing area. The adjusting of the framing field of view of the second camera module may include: determining a target direction according to a positional relationship between the target shooting target and the initial framing area; and adjusting the second camera module to operate according to the target direction through OIS.

In the above embodiment, by controlling the second camera module to be operated according to the target direction, the target photographing object may be photographed by the second camera module, thereby improving the image quality of the target photographing object.

For example, the target photographing target includes a focus point in a framing area of the first camera module or a focus area determined by the focus point. The focus point may be determined by automatically focusing the photographing software according to the framing area, or may be determined by the detected focus control signal. It is to be understood that the focus point may also be determined by other means, but will not be repeated here any further.

Referring to Fig. 4, Fig. 4 is an exemplary view of a focus point shown according to an exemplary embodiment of the present invention. In the exemplary diagram, the multi-camera module may be a wide-angle camera module including a wide-angle camera, a general-purpose camera module including a main camera, and a long-focus camera module including a long-focus camera. Taking the first camera module as a general-purpose camera module and the second camera module as a long focus camera module, if the focus area in the framing area of the first camera module does not exist in the framing area of the second camera module, the second camera By adjusting the framing area of the module, the focus area is present in the framing area of the second camera module.

In the above embodiment, since the target shooting target includes a focus point or a focus area determined by the focus point in the framing area of the first camera module, accordingly, the focus point or It is possible to improve the resolution and image quality of the focus area. Resolution is, in other words, the ability to distinguish the details of an object. Resolution is a physical quantity for describing the ability of a microfilm camera system to reproduce a minute part of an original to be photographed, and is an important index for evaluating image resolution. The higher the resolution, the better the details can be distinguished and can be used to reflect the sharpness of the captured image.

For example, the target photographing target includes a designated partial area in the framing area of the first camera module. The designated partial area may be a pre-designated area by a software developer, an area designated by a user through a setting page, or the like. For example, since the region often of interest by the user is the central region on the screen, some regions designated accordingly may be the central region, thereby improving the resolution and image quality of the central region with the image collected by the second camera module. implement it to Also, for example, the designated partial region may be the top, bottom, left, right, edge, etc. of the screen, and will not be repeated any longer.

In the above embodiment, since the target shooting target includes a partial area designated from the framing area of the first camera module, the resolution and image quality of the partial area designated as the image collected by the second camera module can be improved accordingly. have.

For example, the target shooting target may include a framing area of the first camera module. For example, the first camera module is a general-purpose camera module, the second camera module is a long-focus camera module, and the target shooting target may be the entire framing area of the first camera module, 1 Improve the image quality of the entire screen of the image collected by the camera module.

It should be understood that the target photographing object includes, but is not limited to, the above object, and may be another object, which will not be repeatedly described herein.

Adjusting the framing field of view of the second camera module is to allow the target photographing object to be photographed by the second camera module. When the second camera module includes only one camera, the size of the target shooting target may be smaller than or equal to the size of the framing area of the second camera module, for example, the target shooting target is the framing of the first camera module. When it includes a focus point of an area, a focus area determined by the focus point, or a designated partial area in the framing area of the first camera module, the size of the target shooting target is smaller than the size of the framing area of the second camera module In other words, the target photographing object may appear completely in the same framing area of the second camera module.

In one example, the second camera module includes one camera; The step of allowing the framing area of the second camera module to cover the target photographing object includes allowing the target photographing object to exist in the same framing area or a specific position in the same framing area of the second camera module. The specific position may be a preset position or a position determined by a detected manipulation signal.

For example, by adjusting the framing field of view of the second camera module, the target shooting target appears in the framing area of the second camera module, or appears at a preset position (eg, central position) of the framing area of the second camera module as much as possible. let it be done As shown in FIG. 5 , it is a contrast illustration of a framing area before and after adjustment shown according to an exemplary embodiment of the present invention. In the example diagram, a circle indicates a target photographing target determined by a focus point, and a small solid line block is for indicating a framing area (referred to as an initial framing area) of the second camera module before adjustment, and the target photographing target is If it does not exist within the small solid line block, an adjustment is made to the framing field of view of the second camera module. The small dotted line block is for indicating the framing area (referred to as the adjusted framing area) of the second camera module after adjustment, and if the target photographing object is within the small dotted line block, that is, the target photographing object is the same framing of the second camera module is within the realm.

In a situation where the adjustment width of the second camera module is sufficiently large, the target shooting target may be limited to be located at a specific position of the framing area of the second camera module, and the specific position may be a preset position, for example, a developer It is the same as the location preset by , or the same as the location preset by the user. For example, a center position, a center top position, and the like. The specific position may be a position determined by the detected manipulation signal. For example, in the framing process, notification information is popped up to inform the user to set a location, etc., indicated by the target shooting target in the framing area of the second camera module.

In the above embodiment, since the same framing area of the second camera module can cover the target shooting target, by adjusting the framing field of view of the second camera module, the target shooting target is the same framing area of the second camera module Alternatively, the main image collected by the first camera module is made to exist at a specific position in the same framing area, so that the images collected within the same time by the first camera module and the second camera module after adjustment are fused directly in the future to improve

When the second camera module includes only one camera, in some scenarios, the size of the target shooting target is larger than the size of the framing area of the second camera module, for example, the target shooting target is the framing area of the first camera module In the case of the designated partial area or the framing area of the first camera module, the case where the range of the target shooting target is larger than the range of the framing area of the second camera module may appear, that is, the same framing area of the second camera module can only cover some target shooting targets.

For example, the second camera module includes one camera, and the size of the target shooting target is larger than the size of the framing area of the second camera module; The target direction includes at least two sub-target directions.

The step of adjusting the second camera module to operate along the target direction through the OIS includes, in each sub-target direction, adjusting the second camera module to operate along the sub-target direction through the OIS do.

The target photographing object exists in a first stitched framing area, and the first stitched framing area is obtained by stitching an adjusted framing area obtained through each adjustment. Correspondingly, making the adjusted framing area of the second camera module cover the target photographing object is such that the first stitching framing area obtained by all secondary adjustments covers the target photographing object.

If the target photographing object is not displayed within the same framing area of the second camera module, the target photographing object may be photographed by the second camera module in the form of multiple adjustments. For example, after each adjustment, the second camera module may be adjusted to cover a partial target shooting target, so that all the partial target shooting targets are stitched to obtain a complete target shooting target. Assuming that at least N framing areas of the second camera module are required to cover the entire target photographing target, adjust N times so that the target photographing object exists within the first stitching framing area of the second camera module, and the first stitching The framing area is obtained by stitching the adjusted framing area obtained with all secondary adjustments. Framing areas after different secondary adjustments may or may not have overlapping screens, and the specifics are set according to demand.

In the subsequent fusion process, the sub-images collected by the second camera module may include the sub-images collected after adjusting the second camera module according to the sub-target direction each time. For example, after adjusting the framing field of view each time, the sub-image and the main image collected by the second camera module may each be fused; In addition, after each adjustment, the sub-images collected by the second camera module are first stitched to obtain a stitching dependent image, and then fused together with the main image to improve the quality of the main image.

Illustratively, the framing area of the second camera module adjusted several times may be executed by an adjustment strategy, and in order to adjust the second camera module to be operated according to the sub-target direction, the adjustment strategy is adjusted each time the adjustment contains commands. The adjustment strategy may be predefined, or may be generated by a positional relationship between the target photographing target and the initial framing area.

For example, the target photographing target may be a framing area of the first camera module. In the above embodiment, even if the framing area of the second camera module is smaller than the framing area of the first camera module, through adjusting the framing field of view of the second camera module, the set of framing areas obtained by adjustment several times is the first By being able to cover the framing area of the camera module, it is possible to achieve the effect of expanding the framing range of the second camera module. Furthermore, the number of times of adjusting the framing area of the second camera module may be determined by the proportion of the framing range of the first camera module and the framing range of the second camera module.

As shown in Fig. 6A, this is an exemplary view of the first stitching framing area shown according to an exemplary embodiment of the present invention. It is assumed that at least N framing areas of the second camera module need to be stitched to form the framing area of the first camera module. In the above illustration, it will be described with N=4 as an example. The dotted line block on the upper left indicates the adjusted framing area after the first adjustment, the dotted line block on the upper right indicates the adjusted framing area after the second adjustment, and the dotted line block on the lower left indicates the adjusted framing area after the third adjustment , the dotted block in the lower right indicates the adjusted framing area after the fourth adjustment. A solid line block is a framing area of the first camera module. Stitching the adjusted framing area obtained with the 1st, 2nd, 3rd, 4th adjustment to obtain the first stitching framing area, see that the first stitching framing area covers the framing area of the first camera module can It may be understood that the size of the first stitching framing area may be greater than or equal to the size of the framing area of the first camera module, and the drawings are exemplified as having the same size. Since the framing areas of the first camera module and the second camera module are fixed, accordingly, in one embodiment, the adjustment strategy is preset, including adjustment instructions of adjustment each time. For example, including the adjustment instruction of the primary adjustment, the adjustment instruction of the second adjustment, the adjustment instruction of the third adjustment, and the adjustment instruction of the fourth adjustment, to the corresponding position through different adjustment instructions without needing to temporarily determine the adjustment strategy By adjusting, the processing speed can be improved.

In some application scenarios, since the size of the target shooting target may be larger than the size of the framing area of the camera in the second camera module, it is not possible to make the target shooting target exist in the same framing area of the second camera module, and also adjust Due to the limitation of the width, even if the range of the target shooting target is smaller than or equal to the framing range of the camera in the second camera module, there may be a situation in which the target shooting target does not exist in the same framing area of the second camera module. Considering that, in another embodiment, the second camera module includes at least two cameras; The size of the sub-framing area of a single camera may be smaller or larger than the size of the framing area of the target shooting target, and the set of framing areas formed by the sub-framing ranges of at least two cameras may be greater than or equal to the framing area of the target shooting target. can The cameras in the same camera module may be the same type of camera, and as a long focus camera module, for example, a plurality of cameras in the long focus camera module may be a long focus camera.

Correspondingly, the target direction comprises at least two sub-target directions. The step of adjusting the second camera module to operate along the target direction through the OIS includes adjusting each of the cameras in the second camera module to operate along the corresponding sub-target direction through the OIS do.

The target photographing object exists in a second stitched framing area, and the second stitched framing area is obtained by stitching the adjusted framing area of the camera.

The sub-images collected by the second camera module include images collected by the camera after all adjustments.

The embodiment is implemented so that the framing area of the second camera module covers the target shooting target by adjusting a plurality of cameras, so that the size of the target shooting target is relatively large or the adjustment range of a single camera is limited can be avoided

As shown in Fig. 6B, this is an exemplary view of the second stitching framing area shown according to an exemplary embodiment of the present invention. It is assumed that the framing area of the first camera module is formed of at least N framing areas of the second camera module. In the above illustration, it will be described with N=4 as an example. The upper left dotted line block represents the adjusted framing area obtained by adjusting the first camera, the upper right dotted line block represents the adjusted framing area obtained by adjusting the second camera, and the lower left dotted line block is the third The adjusted framing area obtained by adjusting the camera is indicated, and the dotted line block in the lower right corner indicates the adjusted framing area obtained by adjusting the fourth camera. A solid line block is a framing area of the first camera module or a target photographing target. It can be seen that when the adjusted framing area after the first, second, third, and fourth cameras are adjusted is stitched to obtain the second stitched framing area, the target shooting object may appear in the second stitched framing area.

It should be understood that the above adjustment form is merely exemplary, and other existing or future adjustment forms for allowing the second camera module to capture a target photographing object can be applied to the embodiments of the present invention, and all It should be included within the scope of the claims of the inventive embodiment.

Regarding the collection time of the main image and the sub-image, when the second camera module includes one camera, and the size of the target photographing object is smaller than or equal to the size of the framing area of the second camera module, or the second camera When the module includes a plurality of cameras, etc., the sub-image may be an image collected at the same time as the main image. In an embodiment in which the framing field of view of the second camera module is adjusted several times, the sub image may be a sub image collected by the second camera module after adjusting the framing field of view each time. The acquisition time of the main image may be the same as the acquisition time of any one sub-image among the plurality of sub-images, or may be an arbitrary time point during the collection of the plurality of sub-images. Since the frequency of image acquisition is relatively high, the difference between images continuously acquired within a short period of time is relatively small, and accordingly, image quality improvement processing is performed on a single-frame main image with a plurality of frame sub-images.

The embodiment of the present invention is to improve the image collected by the camera module having a large viewing angle to the image collected by the camera module having a small viewing angle, and accordingly, the sub-image collected by the second camera module and the first camera after adjustment By fusing the main image collected by the module, an image after the fusion can be obtained. In some scenarios, the sub-images mentioned in the fusion process also include the sub-images collected by the second camera module before adjustment, and specifically configured according to demand. In the fusion process, all overlapping parts may be fused, and some overlapping parts may be fused. Since the viewing angle of the second camera module is smaller than the viewing angle of the first camera module, by fusing the sub image and the main image, an image of higher quality can be obtained. For example, a fused image may be obtained by fusing all or part of the overlapping portion and the main image using a preset image fusion algorithm. A specific image fusion algorithm may refer to a fusion algorithm in a related art, and will not be repeated any longer herein.

In one embodiment, the fusion processing step may be executed during a framing process or may be executed after a shooting operation command is received. In the above embodiment, regardless of whether the framing process or the shooting operation is performed, the main image after all the fusions can be displayed.

In some application scenarios, when shooting with different camera modules, the object of interest when shooting with one camera module may exist within the field of view of the other camera module. Therefore, the method also includes: When the first camera module is in the current main shooting state and the target shooting target in the framing area of the first camera module does not exist in the initial framing area of the second camera module, the sub-image collected by the second camera module and the second 1 including the step of fusion processing the main image by the camera module.

In the above embodiment, when the target shooting target is in the framing area of the second camera module, the image collected by the first camera is directly captured by the second camera module without the need to adjust the second camera module. The picture quality can be improved.

The various technical features in the above embodiment can be arbitrarily combined as long as there is no conflict or contradiction therebetween, but due to the narrow width limitation, they are not described in detail, so it is also seen that various technical features in the above embodiment are arbitrarily combined It should fall within the scope of the disclosure of the invention.

Here, a combination of two of them will be described as an example.

For example, when shooting, the user is generally interested in the central position or focusing position of the picture, and since W has advantages in image quality and resolution compared to DW, when shooting with DW, at the same time W After taking an image, based on the picture of the next DW, the picture quality of the central area is improved by fusion enhancement of the picture taken with W in the center part of the picture. When shooting by W, one Tele is collected at the same time, and if the focus point of W's photo is within the Tele range, the quality of W focus area can be improved through Tele. If the focus point of W's photo is not within the Tele range, the Tele range can be adjusted so that the focus point of W's photo is within the Tele range.

Illustratively, by designing the OIS of the long focus camera module (Tele) at a large angle, Tele is implemented to cover the entire fov (camera field of view / field of view) of the universal camera module, and the shooting focus point of the universal camera module According to the position, the framing area of the Tele is adjusted to the corresponding position, then taken together, and then fusion processing is performed to improve the resolution of the focus position and improve the image quality.

As another example, referring to FIG. 7 , this is a flowchart of another multi-camera module-based image processing method shown according to an exemplary embodiment of the present invention, wherein the multi-camera module has a wide-angle viewing angle that is gradually reduced. A camera module (DW), a general-purpose camera module (W), and a long focus camera module (Tele), the method comprising the following steps.

In step 702, a photographing state of the camera module is determined.

In step 704, if the wide-angle camera module is currently in the main shooting state, the main image collected by the wide-angle camera module and the sub-image collected by the general-purpose camera module are acquired, and the sub-image and the main image are fused.

In step 706, if the general-purpose camera module is currently in a main shooting state, determine a current framing field of view of the long-focal shooting module; Adjustment to the maximum range for the current framing field of view of the long focus imaging module is carried out through OIS.

In step 708, the sub-image collected by the long-focal camera module after adjustment and the main image collected by the general-purpose camera module are fused.

The above embodiment implemented the improvement of resolution and image quality of shooting in the current scenario when shooting in different scenarios through a combination of different cameras, and in particular, by expanding the framing range of Tele with OIS, the general-purpose camera module is the main shooting state Improve the picture quality in (referred to as the main shooting mode).

For each method embodiment described above, for the sake of brevity, all of them are expressed as a series of operation combinations, but those skilled in the art will recognize that some steps may be performed concurrently or using a different order according to the embodiments of the present invention. , it should be noted that the present invention is not limited by the order of operations described in the embodiment.

Next, those skilled in the art will understand that all of the embodiments described in the specification are selectable embodiments, and the mentioned operations and modules are not necessarily required for the embodiments of the present invention.

Corresponding to the above-described embodiment of the multi-camera module-based image processing method, the embodiment of the present invention also provides embodiments of a multi-camera module-based image processing apparatus, a device to which the apparatus is applied, and a storage medium.

As shown in Fig. 8, this is a block diagram of a multi-camera module-based image processing apparatus according to an exemplary embodiment of the present invention, the apparatus comprising:

When the first camera module is in the current main shooting state, and the target shooting target in the framing area of the first camera module does not exist in the initial framing area of the second camera module, by adjusting the framing field of the second camera module , a field of view adjustment module 820 configured such that the adjusted framing area of the second camera module covers the target photographing object, wherein the field of view of the second camera module is smaller than the field of view of the first camera module; and

a fusion processing module 840 configured to fusion-process the sub-image and the main image, wherein the sub-image is an image collected by the second camera module, and the main image is an image collected by the first camera module include

In an optional embodiment, the step of adjusting the framing field of view of the second camera module by the field of view adjustment module 820 comprises:

determining a current framing field of view of the second camera module; and

adjusting the current framing field of view of the second camera module to a maximum range through OIS;

Here, the target shooting target is an arbitrary shooting point in the non-overlapping part of the maximum framing area and the initial framing area after the second camera module is adjusted.

In an optional embodiment, the target shooting target is an object of interest in a framing area, and the step of adjusting the framing field of view of the second camera module by the field of view adjustment module 820 comprises:

determining a target direction according to a positional relationship between the target shooting target and the initial framing area; and

and adjusting the second camera module to operate along the target direction through OIS.

In an optional embodiment, the target shooting target is a focus point in the framing area of the first camera module, a focus area determined by the focus point, a designated partial area in the framing area of the first camera module, or the and a framing area of the first camera module.

In an optional embodiment, the second camera module comprises one camera; the step of allowing the framing area of the second camera module to cover the target photographing object includes the step of allowing the target photographing object to exist in the same framing area or a specific position in the same framing area of the second camera module; The specific position is a preset position or a position determined by a detected manipulation signal.

In an optional embodiment, the second camera module includes one camera, and the size of the target shooting target is larger than the size of the framing area of the second camera module; The target direction includes at least two sub-target directions.

The step of adjusting the field of view adjustment module 820 to operate the second camera module along the target direction through the OIS includes, in each sub-target direction, the second camera module along the sub-target direction through the OIS. adjusting to actuate;

the target shooting object is in a first stitching framing area, the first stitching framing area is obtained by stitching the adjusted framing areas obtained by all secondary adjustments;

The sub-images collected by the second camera module include sub-images collected after adjusting the second camera module according to the sub-target direction each time.

In one optional embodiment, the second camera module includes at least two cameras, the size of the target shooting target is larger than the size of the framing area of the second camera module; the target direction includes at least two sub-target directions;

The step of adjusting the field of view adjustment module 820 so that the second camera module is operated along the target direction through the OIS operates along the sub-target direction corresponding to the camera in the second camera module through the OIS adjusting each to be possible;

the target shooting object is in a second stitching framing area, the second stitching framing area is obtained by stitching the adjusted framing area of the camera;

The sub-images collected by the second camera module include images collected by the camera after all adjustments.

In an optional embodiment, the fusion processing module 840 also includes:

When the first camera module is in the current main shooting state and the target shooting target in the framing area of the first camera module exists in the initial framing area of the second camera module, the sub-images collected by the second camera module and the first It is configured to fuse the main image by the camera module.

In an optional embodiment, the fusion processing step may be executed during a framing process or may be executed after a photographing manipulation command is received.

In an optional embodiment, the multi-camera module includes a wide-angle camera module, a general-purpose camera module, and a long-focus camera module whose viewing angle is gradually reduced.

For the specific details of the implementation process of the functions and actions of each module in the apparatus, reference may be made to the implementation process of the corresponding step of the method, which will not be repeated any longer.

In the case of an apparatus embodiment, since it almost corresponds to a method embodiment, reference may be made to the partial description of the method embodiment for related parts. The device embodiments described above are merely exemplary, where a module described as a separate component may or may not be physically separate, and a component shown as a module may or may not be a physical module, i.e., located in one place, It may be distributed in a plurality of network modules. According to the actual demand, some or all of the modules may be selected to realize the purpose of the solution of the embodiment of the present invention. Those skilled in the art will be able to understand and practice even if creative work has not been done.

Correspondingly, an embodiment of the present invention also provides a computer-readable storage medium having a computer program stored therein, and when the program is executed by a processor, the steps of the multi-camera module-based processing method of any one of the above are implemented. .

The embodiment of the present invention may use the form of a computer program implemented in one or a plurality of storage media (including but not limited to magnetic memory, CD-ROM, optical memory, etc.) including program code. Computer-usable storage media includes permanent and non-persistent, removable and non-removable media, and any method or technology may implement storage of information. The information may be computer readable instructions, data structures, modules of programs, or other data. Examples of storage media of a computer include phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), read-only memory ( Read Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Flash memory or other memory technology, Optical Disc Read Only Memory (CD-ROM), Digital Video Disc (Digital Video Disc) Disc, DVD) or other optical memory, magnetic tape cassette, magnetic tape cassette storage device or other magnetic storage device or any other non-transmission medium, configured to store information visited by a computer device. do.

Correspondingly, an embodiment of the present invention provides an electronic device based on a multi-camera module, the device comprising: a camera module having at least two different viewing angles; processor; and a memory configured to store instructions executable by the processor; Here, the processor,

When the first camera module is in the current main shooting state, and the target shooting target in the framing area of the first camera module does not exist in the initial framing area of the second camera module, by adjusting the framing field of the second camera module , such that the adjusted framing area of the second camera module covers the target shooting object, wherein the viewing angle of the second camera module is smaller than the viewing angle of the first camera module;

and fusing the sub-image and the main image, wherein the sub-image is an image collected by the second camera module, and the main image is an image collected by the first camera module.

Fig. 9 is a structural diagram of an image processing apparatus 900 based on a multi-camera module shown according to an exemplary embodiment. For example, the device 900 may be a user device, specifically a mobile phone, a computer, a digital terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal portable terminal, and a smart watch, smart glasses, smart bracelet. It may be a wearable device such as

Referring to FIG. 9 , an apparatus 900 includes a processing component 902 , a memory 904 , a power component 906 , a multimedia component 908 , an audio component 910 , an input/output (I/O) interface ( 912 , one or more of a sensor component 914 , and a communication component 916 .

Processing component 902 generally controls the overall operation of device 900 , such as operations related to displays, phone calls, data communications, camera manipulation, and record manipulation. The processing component 902 may include one or more processors 920 to execute instructions to complete all or some steps of the method. Further, processing component 902 may include one or a plurality of modules to facilitate interaction between processing component 902 and other components. For example, processing component 902 may include a multimedia module to facilitate interaction between multimedia component 908 and processing component 902 .

Memory 904 is configured to store various types of data to support manipulation of device 900 . Examples of such data include instructions, contact data, phone book data, messages, photos, videos, and the like of any application or method operated on device 900 . Memory 904 includes static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic It may be implemented by any one type of volatile or non-volatile storage device of memory, flash memory, magnetic disk, or optical disk, or a combination thereof.

Power component 906 supplies power to various components of device 900 . Power component 906 may include a power management system, one or more power sources, and other components related to generating, managing, and distributing power for device 900 .

Multimedia component 908 includes a screen that provides one output interface between the device 900 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). When the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors for sensing touch, slide and gesture on the touch panel. The touch sensor may detect a boundary of a touch or slide operation, as well as a duration and pressure associated with the touch or slide operation. In some embodiments, multimedia component 908 includes at least one of a front camera and a rear camera. When the device 900 is in an operation mode such as a shooting mode or a video mode, at least one of the front camera and the rear camera may receive external multimedia data. Each front camera and rear camera may be a single fixed optical lens system or may have focal length and optical zoom functions.

The audio component 910 is configured to at least one of output and input an audio signal. For example, the audio component 910 includes one microphone (MICro Phone, MIC), and when the device 900 is in an operation mode such as a call mode, a recording mode, and a voice recognition mode, the microphone receives an external audio signal. configured to do The received audio signal may be further stored in memory 904 or transmitted by communication component 916 . In some embodiments, the audio component 910 further comprises one speaker configured to output an audio signal.

I/O interface 912 provides an interface between processing component 902 and an external interface module, which may be a keyboard, click wheel, button, or the like. Such buttons include, but are not limited to, a home button, a volume button, a start button, and a lock button.

The sensor component 914 includes one or a plurality of sensors configured to provide various aspects of condition assessment for the device 900 . For example, the sensor component 914 may detect an on/off state of the device 900 , a relative position of the component, eg, the component is a monitor and a keypad of the device 900 , and the sensor component 914 is a device It is possible to detect a change in the position of one component in 900 or in the device 900 , the presence or absence of contact between the user and the device 900 , a direction or acceleration/deceleration of the device 900 , and a change in the temperature of the device 900 . The sensor component 914 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor component 914 may further include an optical sensor, such as a Complementary Metal Oxide Semiconductor (CMOS) image sensor or a Charged Coupled Device (CCD) image sensor configured for use in imaging applications. In some embodiments, the sensor component 914 may also include an acceleration sensor, a gyro sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

Communication component 916 is configured to facilitate communication between device 900 and other devices in a wired or wireless manner. Device 900 may access wireless Internet based on communication criteria such as WiFi, 2G or 3G, 4G Long Term Evolution (LTE), 5G New Radio (NR), or a combination thereof. In one exemplary embodiment, the communication component 916 receives a broadcast signal or broadcast related information from an external broadcast management system by way of a broadcast channel. In one exemplary embodiment, the communication component 916 further includes a Near Field Communication (NFC) module that facilitates near field communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth technology and other technologies. have.

In an exemplary embodiment, device 900 includes one or more ApplicationSpecific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processors (DSPs), Programmable Logic Devices Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor or other electronic component, and device 800 is for carrying out the method.

In an exemplary embodiment, there is provided a non-transitory computer-readable storage medium, such as, for example, memory 904 comprising instructions, when the instructions in the storage medium are executed by the processor 920 of the apparatus 900 , the apparatus To enable the 900 to execute a multi-camera module-based image processing method, the method comprising:

When the first camera module is in the current main shooting state, and the target shooting target in the framing area of the first camera module does not exist in the initial framing area of the second camera module, by adjusting the framing field of the second camera module , allowing the adjusted framing area of the second camera module to cover the target photographing object, wherein the viewing angle of the second camera module is smaller than the viewing angle of the first camera module;

and fusing the sub-image and the main image, wherein the sub-image is an image collected by the second camera module, and the main image is an image collected by the first camera module.

The non-transitory computer-readable storage medium may be a ROM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

A person skilled in the art will be able to readily come up with other implementations of the embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention embodiment is intended to cover any modification, use or adaptability change of the present invention embodiment, and such modification, use or adaptability change follows the general principle of the present invention embodiment, and the present technology not disclosed in the present embodiment It includes known common knowledge or common technical means in the field. The specification and examples are to be regarded as illustrative only, the true scope and spirit of the embodiments of the present invention being indicated by the claims below.

It should be understood that the embodiments of the present invention are not limited to the precise structure described above and shown in the drawings, and various modifications and changes may be made without departing from this scope. The scope of embodiments of the present invention is limited only by the appended claims.

In this embodiment, when the first camera module is in the current main shooting state and the target shooting target in the framing area of the first camera module does not exist in the initial framing area of the second camera module, framing of the second camera module The field of view is adjusted so that the adjusted framing area of the second camera module covers the target shooting target, and the sub-image collected by the second camera module and the main image collected by the first camera module are integrated and processed to obtain an image The effect of improving the picture quality can be achieved. Furthermore, by adjusting the second camera module, the shooting range of the second camera module is expanded so that the framing area after adjustment can cover an area not included in the initial framing area, so image quality cannot be improved It is possible to implement image quality improvement for at least a part of an area.

Claims (13)

A multi-camera module-based image processing method, comprising:
When the first camera module is in the current main shooting state, and the target shooting target in the framing area of the first camera module does not exist in the initial framing area of the second camera module, by adjusting the framing field of the second camera module , allowing the adjusted framing area of the second camera module to cover the target photographing object, wherein the viewing angle of the second camera module is smaller than the viewing angle of the first camera module;
fusion processing of the sub-image and the main image, wherein the sub-image is an image collected by the second camera module, and the main image is an image collected by the first camera module;
Adjusting the framing field of view of the second camera module comprises:
determining a current framing field of view of the second camera module; and
adjusting the current framing field of view of the second camera module to a maximum range through an Optical Image Stabilizer (OIS);
The image processing method, characterized in that the target shooting target is an arbitrary shooting point in the non-overlapping portion of the maximum framing area and the initial framing area after the adjustment of the second camera module. delete According to claim 1,
The target shooting target is a target of interest in a framing area, and adjusting the framing field of view of the second camera module includes:
determining a target direction according to a positional relationship between the target shooting target and the initial framing area; and
and adjusting the second camera module to operate along the target direction through OIS. According to claim 1,
The target shooting target is a focus point in the framing area of the first camera module, a focus area determined by the focus point, a designated partial area in the framing area of the first camera module, or framing of the first camera module An image processing method comprising a region. According to claim 1,
the second camera module includes one camera; the step of allowing the framing area of the second camera module to cover the target photographing object includes the step of allowing the target photographing object to exist in the same framing area or a specific position in the same framing area of the second camera module; The specific position is a predetermined position or an image processing method, characterized in that the position determined by the detected manipulation signal. 4. The method of claim 3,
the second camera module includes one camera, and the size of the target shooting object is larger than the size of the framing area of the second camera module; the target direction includes at least two sub-target directions;
The step of adjusting the second camera module to operate along the target direction through the OIS includes, in each sub-target direction, adjusting the second camera module to operate along the sub-target direction through the OIS and;
the target shooting object is in a first stitching framing area, the first stitching framing area is obtained by stitching the adjusted framing areas obtained by all secondary adjustments;
The sub-images collected by the second camera module include sub-images collected after adjusting the second camera module according to the sub-target direction each time. 4. The method of claim 3,
the second camera module includes at least two cameras, and the size of the target photographing object is larger than the size of the framing area of the second camera module; the target direction includes at least two sub-target directions;
The step of adjusting the second camera module to operate along the target direction through the OIS includes adjusting each of the cameras in the second camera module to operate along the corresponding sub-target direction through the OIS and;
the target shooting object exists in a second stitching framing area, and the second stitching framing area is obtained by stitching the adjusted framing area of each camera;
and the sub-images collected by the second camera module include images collected by the camera after all adjustments. 8. The method according to any one of claims 1 and 3 to 7,
The method is
When the first camera module is in the current main shooting state and the target shooting target in the framing area of the first camera module does not exist in the initial framing area of the second camera module, the sub-image collected by the second camera module and Image processing method, characterized in that it further comprises the step of fusion processing the main image by the first camera module. 8. The method according to any one of claims 1 and 3 to 7,
The fusion processing step is an image processing method, characterized in that it is executed during a framing process or after a shooting manipulation command is received. 8. The method according to any one of claims 1 and 3 to 7,
The multi-camera module is an image processing method, characterized in that it comprises a wide-angle camera module, a general-purpose camera module, and a long-focus camera module whose viewing angle is gradually reduced. As an image processing device based on a multi-camera module,
The device is
When the first camera module is in the current main shooting state, and the target shooting target in the framing area of the first camera module does not exist in the initial framing area of the second camera module, by adjusting the framing field of the second camera module , a field of view adjustment module configured such that the adjusted framing area of the second camera module covers the target photographing object, wherein the viewing angle of the second camera module is smaller than the viewing angle of the first camera module; and
a fusion processing module configured to fuse a sub-image and a main image, wherein the sub-image is an image collected by the second camera module, and the main image is an image collected by the first camera module;
The field of view adjustment module is also
determine a current framing field of view of the second camera module;
configured to adjust the current framing field of view of the second camera module to a maximum range through an Optical Image Stabilizer (OIS),
The image processing apparatus according to claim 1, wherein the target photographing target is an arbitrary photographing point in a non-overlapping portion of the maximum framing area and the initial framing area after adjustment of the second camera module. As an electronic device based on a multi-camera module,
a camera module having at least two different fields of view;
processor; and
a memory configured to store instructions executable by the processor;
The processor is
When the first camera module is in the current main shooting state, and the target shooting target in the framing area of the first camera module does not exist in the initial framing area of the second camera module, by adjusting the framing field of the second camera module , such that the adjusted framing area of the second camera module covers the target shooting object, wherein the viewing angle of the second camera module is smaller than the viewing angle of the first camera module;
and a sub-image and a main image are fused, wherein the sub-image is an image collected by the second camera module, and the main image is an image collected by the first camera module;
The processor is also
determine a current framing field of view of the second camera module;
configured to adjust the current framing field of view of the second camera module to a maximum range through an Optical Image Stabilizer (OIS),
The electronic device, characterized in that the target shooting target is an arbitrary shooting point in the non-overlapping portion of the maximum framing area and the initial framing area after the adjustment of the second camera module. A computer readable storage medium comprising:
A computer readable storage medium having a computer program stored thereon and implementing the steps of the method of any one of claims 1 and 3 to 7 when the program is executed by a processor.

Images