CN112668636A

CN112668636A - Camera shielding detection method and system, electronic equipment and storage medium

Info

Publication number: CN112668636A
Application number: CN202011563466.4A
Authority: CN
Inventors: 高浩波; 李海; 班孝坤
Original assignee: Spreadtrum Communications Shanghai Co Ltd
Current assignee: Spreadtrum Communications Shanghai Co Ltd
Priority date: 2020-12-25
Filing date: 2020-12-25
Publication date: 2021-04-16
Anticipated expiration: 2040-12-25
Also published as: WO2022134957A1; CN112668636B

Abstract

The invention discloses a camera shielding detection method and system, electronic equipment and a storage medium. The camera shielding detection method comprises the following steps: responding to the entering of a non-single-shot shooting mode, and controlling the main camera and the auxiliary camera to shoot; the space areas shot by the main camera and the auxiliary camera are consistent; acquiring a first picture shot by the main camera and a second picture shot by the auxiliary camera; and determining whether the auxiliary camera is shielded or not according to the similarity of the first picture and the second picture. The method and the device can judge whether the auxiliary camera is shielded or not without additionally increasing hardware cost or being limited by a specific camera, and have higher universality and adaptability. In addition, whether the auxiliary camera is shielded or not is judged according to the similarity of pictures shot by the main camera and the auxiliary camera simultaneously, and compared with the subjective judgment of a user, the accuracy is higher.

Description

Camera shielding detection method and system, electronic equipment and storage medium

Technical Field

The present invention relates to the field of electronic devices, and in particular, to a method and a system for detecting camera occlusion, an electronic device, and a storage medium.

Background

With the development of mobile technology, mobile terminals have become common electronic products in daily life. With the dependence of people on mobile terminals becoming higher and higher, the performance requirements of people become higher and higher, especially the photographing performance of mobile terminals. The shooting performance of the mobile terminal with a single camera reaches the limit to a certain extent, and a plurality of cameras are bound to be used when the shooting performance is broken through.

At present, double-camera mobile phones and triple-camera mobile phones are available in the market, and even some mobile phone manufacturers have proposed four-camera mobile phones and five-camera mobile phones. Complex functions such as background blurring, refocusing, night/dim light photographing enhancement, optical zooming, HDR, three-dimensional application, and the like can be realized by using a plurality of cameras, and solutions of different mobile phone manufacturers are different in order to realize the functions. Some solutions provided by mobile phone manufacturers belong to a 'true multi-camera' solution, that is, a result of simultaneous imaging of a plurality of cameras is used for algorithm processing to realize the above functions. However, some solutions provided by mobile phone manufacturers belong to a "false multi-shot" solution, that is, although there are multiple cameras on the hardware, the above functions are realized by using only a single camera to form images. When the user uses the multi-shooting function, the user can not distinguish whether the used mobile phone is 'true multi-shooting' or 'false multi-shooting', if the mobile phone is 'true multi-shooting', the user accidentally shields the auxiliary camera, so that the shooting quality is poor, and at the moment, if the user is not prompted to shield the auxiliary camera, the shooting experience of the user is reduced.

In order to solve the above problems, there are several methods for determining whether an auxiliary camera in a multi-camera mobile phone is blocked in the prior art:

firstly, whether the auxiliary camera is shielded or not is judged by checking the photographing effect under a multi-photographing scene. The method depends on subjective judgment of a user, and the judgment accuracy is low.

And secondly, judging whether the auxiliary camera is shielded or not according to the brightness of the ambient light. The method needs to be realized by means of ambient light brightness, and ambient light brightness information needs to pass through an aem statistical module, but only a raw sensor has a aem statistical module, and other sensors such as yuv sensors do not have a aem statistical module, so that the brightness information of the ambient light cannot be acquired, and if the sensors are used as a camera of a mobile phone, the judgment function cannot be realized.

And thirdly, a distance sensor or an ultrasonic sensor is arranged on a rear panel of the mobile phone, and the distance from an object to the camera is detected through the sensor, so that whether the auxiliary camera is shielded or not is judged. This method requires additional installation of the sensor, resulting in an increase in the cost of the cellular phone.

Disclosure of Invention

The present invention provides a method and a system for detecting camera occlusion, an electronic device, and a storage medium, to overcome the above-mentioned defects in the prior art.

The invention solves the technical problems through the following technical scheme:

the invention provides a camera shielding detection method in a first aspect, which comprises the following steps:

responding to the entering of a non-single-shot shooting mode, and controlling the main camera and the auxiliary camera to shoot; the space areas shot by the main camera and the auxiliary camera are consistent;

acquiring a first picture shot by the main camera and a second picture shot by the auxiliary camera;

and determining whether the auxiliary camera is shielded or not according to the similarity of the first picture and the second picture.

Optionally, the acquiring a first picture taken by the main camera and a second picture taken by the auxiliary camera specifically includes:

and respectively preprocessing the pictures shot by the main camera and the auxiliary camera to obtain a first picture and a second picture with the same field angle and size.

Optionally, the determining whether the auxiliary camera is occluded according to the similarity between the first picture and the second picture specifically includes:

calculating the similarity of the first picture and the second picture;

judging whether the similarity is lower than a preset value or not;

if yes, determining that the auxiliary camera is shielded;

if not, determining that the auxiliary camera is not shielded.

Optionally, the calculating the similarity between the first picture and the second picture specifically includes:

cutting the first picture into a plurality of first sub-pictures, and cutting the second picture into a plurality of second sub-pictures; wherein the first sub-picture and the second sub-picture are the same in size and number;

respectively judging whether the first sub-picture is similar to a second sub-picture at a corresponding position or not aiming at each first sub-picture;

and calculating the similarity of the first picture and the second picture according to all the judged similarity results.

A second aspect of the present invention provides a camera occlusion detection system, comprising:

the control module is used for responding to the entering of a non-single-shot shooting mode and controlling the main camera and the auxiliary camera to shoot; the space areas shot by the main camera and the auxiliary camera are consistent;

the acquisition module is used for acquiring a first picture shot by the main camera and a second picture shot by the auxiliary camera;

and the determining module is used for determining whether the auxiliary camera is blocked according to the similarity of the first picture and the second picture.

Optionally, the acquiring module is specifically configured to respectively pre-process the pictures taken by the main camera and the auxiliary camera to obtain a first picture and a second picture with the same field angle and size.

Optionally, the determining module specifically includes:

the calculating unit is used for calculating the similarity of the first picture and the second picture;

the determining unit is used for judging whether the similarity is lower than a preset value or not, and determining that the auxiliary camera is shielded under the condition of yes; and determining that the secondary camera is not occluded if not.

Optionally, the computing unit specifically includes:

a cropping sub-unit, configured to crop the first picture into a plurality of first sub-pictures, and crop the second picture into a plurality of second sub-pictures; wherein the first sub-picture and the second sub-picture are the same in size and number;

the judging subunit is used for respectively judging whether the first sub-picture is similar to the second sub-picture at the corresponding position or not according to each first sub-picture;

and the calculating subunit is used for calculating the similarity of the first picture and the second picture according to all the judged similarity results.

A third aspect of the present invention provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the camera occlusion detection method according to the first aspect when executing the computer program.

A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the camera occlusion detection method according to the first aspect.

The positive progress effects of the invention are as follows: after entering a non-single-shot shooting mode, controlling the main camera and the auxiliary camera to shoot simultaneously, and determining whether the auxiliary camera is shielded or not according to the similarity of a first picture shot by the main camera and a second picture shot by the auxiliary camera. Compared with the prior art, the method and the device can judge whether the auxiliary camera is shielded or not without additionally increasing hardware cost or being limited by a specific camera, and have higher universality and adaptability. In addition, whether the auxiliary camera is shielded or not is judged according to the similarity of pictures shot by the main camera and the auxiliary camera simultaneously, and compared with the subjective judgment of a user, the accuracy is higher.

Furthermore, the first picture is cut into a plurality of first sub-pictures, the second picture is cut into a plurality of second sub-pictures, whether the sub-pictures are similar or not is compared respectively, the similarity of the first picture and the second picture is calculated based on the similarity result of the sub-pictures, the accuracy of similarity calculation is improved, the accuracy of judging whether the auxiliary camera is shielded or not is further improved, and the user experience is improved.

Drawings

Fig. 1 is a flowchart of a camera occlusion detection method according to embodiment 1 of the present invention.

Fig. 2 is a flowchart of a specific method of step S103 according to embodiment 1 of the present invention.

Fig. 3 is an effect diagram of cropping the first picture into the first sub-picture according to embodiment 1 of the present invention.

Fig. 4 is an effect diagram of cropping the second picture into the second sub-picture according to embodiment 1 of the present invention.

Fig. 5 is a block diagram of a camera occlusion detection system according to embodiment 2 of the present invention.

Fig. 6 is a schematic structural diagram of an electronic device according to embodiment 3 of the present invention.

Detailed Description

The invention is further illustrated by the following examples, which are not intended to limit the scope of the invention.

The camera shielding detection method provided by the embodiment of the invention can be applied to electronic equipment comprising a main camera and an auxiliary camera. And the space areas shot by the main camera and the auxiliary camera are consistent. In one possible design, the primary camera and the secondary camera are located on the same side of the electronic device, e.g., both rear cameras. The number of the auxiliary cameras can be one, two or even more.

Wherein the primary/secondary camera is used to capture still images or video. In general, a camera may include a photosensitive element such as a lens group including a plurality of lenses (convex or concave lenses) for collecting an optical signal reflected by an object to be photographed and transferring the collected optical signal to an image sensor, and an image sensor. And the image sensor generates an original image of the object to be shot according to the optical signal.

It should be noted that when the camera application of the electronic device is in an open state or an operating state, the preview image displayed on the display screen is an image captured by the main camera. That is, if the primary camera is occluded, it is immediately known by viewing the preview screen, specifically, occlusion of the preview screen indicates that the primary camera is occluded, and non-occlusion of the preview screen indicates that the primary camera is not occluded. Therefore, the camera occlusion detection method provided by the embodiment of the invention is mainly used for detecting whether the auxiliary camera is occluded or not.

Example 1

As shown in fig. 1, the present embodiment provides a method for detecting camera occlusion, which includes the following steps S101 to S103:

and step S101, responding to the entering of the non-single-shot shooting mode, and controlling the main camera and the auxiliary camera to shoot. And the space areas shot by the main camera and the auxiliary camera are consistent.

It should be noted that the non-single-shot shooting mode refers to shooting in a non-single-shot scene, and specifically, after entering the non-single-shot shooting mode, shooting with at least two cameras simultaneously in response to a triggered shooting operation to implement different shooting functions, such as a depth of field mode, an optical zoom, and the like.

In one example implementation, real-time or delayed capture is achieved in response to a user-triggered capture operation, such as by the user clicking on a "capture" icon. In another example of the implementation, the photographing operation is performed after entering a non-single shot photographing mode for a preset time period in response to a system timing photographing operation.

And S102, acquiring a first picture shot by the main camera and a second picture shot by the auxiliary camera.

In an optional implementation manner of step S102, the pictures taken by the main camera and the auxiliary camera are respectively preprocessed to obtain a first picture and a second picture with the same view angle and size.

Since the specific positions of the main camera and the auxiliary camera on the electronic device are different, for example, the main camera and the auxiliary camera are arranged horizontally or longitudinally, the field angles of the main camera and the auxiliary camera are different. The embodiment preprocesses the pictures shot by the main camera and the auxiliary camera so as to keep the angle of view and the size of the two pictures the same.

In an example of the preprocessing implementation, the edges of the two pictures are cut, and then the cut pictures are subjected to homography matrix mapping to obtain a first picture and a second picture with the same view angle and size. Specifically, the proportion of the cut pictures is related to the arrangement mode of the main camera and the auxiliary camera.

In order to improve the processing efficiency of the picture, in an optional implementation manner of step S102, the preprocessing further includes: and converting the pictures shot by the main camera and the auxiliary camera into single-channel gray-scale pictures.

It should be noted that the format of the first picture and the format of the second picture need to be the same, and usually are YUV format or RGB format. In a specific example, the format of the picture output by the main camera is raw format, and the format of the picture output by the auxiliary camera is YUV format, so that format conversion of the picture output by the main camera is required, that is, the raw format is converted into YUV format.

In an optional implementation manner of step S102, if only the first picture captured by the main camera can be obtained, but the second picture captured by the auxiliary camera cannot be obtained, it is described that the electronic device still uses a single camera, that is, the main camera to capture the image in the non-single-shot shooting mode, and real non-single-shot shooting is not implemented. In this case, the subsequent step S103 need not be continuously performed.

And S103, determining whether the auxiliary camera is shielded or not according to the similarity of the first picture and the second picture.

In an optional embodiment, as shown in fig. 2, the step S103 specifically includes the following steps S103a to S103 d:

and step S103a, calculating the similarity of the first picture and the second picture.

In one example of the specific implementation, the step S103a includes the following steps S103a 1S 103a 3:

step S103a1, cutting the first picture into a plurality of first sub-pictures, and cutting the second picture into a plurality of second sub-pictures; wherein the first sub-picture and the second sub-picture are the same in size and number. Wherein, the larger the number of the sub-pictures cut out, the more accurate the similarity of the first picture and the second picture is calculated.

Step S103a2, for each first sub-picture, respectively determining whether the first sub-picture is similar to a second sub-picture at a corresponding position.

In a specific implementation, whether the first sub-picture is similar to the second sub-picture at the corresponding position may be determined according to a hash algorithm, specifically, a hamming distance between the hash value of the first sub-picture and the hash value of the second sub-picture is calculated, and a smaller hamming distance indicates that the first sub-picture is more similar to the second sub-picture.

In a specific example, after the preprocessing, the first picture and the second picture are both 27 × 24 in size, and the first picture is cut into 9 first sub-pictures with 9 × 8 in size, as shown in fig. 3. Similarly, the second picture is also cut into 9 second sub-pictures with size 9 x 8, as shown in fig. 4. And judging whether the first sub-picture 11 is similar to the second sub-picture 21 at the corresponding position by using a difference hash algorithm, if the Hamming distance between the hash value of the first sub-picture 11 and the hash value of the second sub-picture 21 is less than 10, considering that the first sub-picture 11 is similar to the second sub-picture 21, and otherwise, considering that the first sub-picture 11 is not similar to the second sub-picture 21.

Step S103a3, calculating the similarity of the first picture and the second picture according to all the judged similarity results.

In the above example, as shown in fig. 3 and 4, the similarity between the first and second pictures is calculated according to the result of the similarity between the first sub-picture 11 and the second sub-picture 21, the result of the similarity between the first sub-picture 12 and the second sub-picture 22, the result of the similarity between the first sub-picture 13 and the second sub-picture 23, the result of the similarity between the first sub-picture 14 and the second sub-picture 24, the result of the similarity between the first sub-picture 15 and the second sub-picture 25, the result of the similarity between the first sub-picture 16 and the second sub-picture 26, the result of the similarity between the first sub-picture 17 and the second sub-picture 27, the result of the similarity between the first sub-picture 18 and the second sub-picture 28, and the result of the similarity between the first sub-picture 19 and the second sub-picture 29.

In a specific example, the default similarity is set to 1%, and if there is a group of similar first and second sub-pictures, the similarity between the first and second pictures is increased by 11%. If there are 9 groups of similar first sub-pictures and second sub-pictures, the similarity between the first picture and the second picture is 100%. The default similarity and the similarity increased by the first sub-picture and the second sub-picture with a group of similarities can be set according to the number of the first sub-picture and the second sub-picture.

In the embodiment, the first picture is cut into the plurality of first sub-pictures, the second picture is cut into the plurality of second sub-pictures, whether the sub-pictures are similar or not is compared, the similarity of the first picture and the second picture is calculated based on the similarity result of the sub-pictures, the accuracy of similarity calculation is improved, the accuracy of judging whether the auxiliary camera is shielded or not is further improved, and the user experience is improved.

Step S103b, determining whether the similarity is lower than a preset value, if so, performing step S103c, and if not, performing step S103 d. The preset value may be set according to actual conditions, and may be set to 90% or 80%, for example.

And step S103c, determining that the auxiliary camera is blocked. In one example of implementation, in the case that it is determined that the secondary camera is occluded, a prompt message is output to prompt the user that the secondary camera is occluded. The user may adjust the obstruction, such as a human hand, after receiving the prompting message so that the secondary camera no longer continues to be occluded.

And step S103d, determining that the auxiliary camera is not blocked.

In this embodiment, after entering the non-single-shot shooting mode, the main camera and the auxiliary camera are controlled to shoot simultaneously, and whether the auxiliary camera is shielded is determined according to the similarity of a first picture shot by the main camera and a second picture shot by the auxiliary camera. Compared with the prior art, the embodiment does not need to additionally increase hardware cost or is limited by a specific camera, can also judge whether the auxiliary camera is shielded, and has higher universality and adaptability. In addition, whether the auxiliary camera is shielded or not is judged according to the similarity of pictures shot by the main camera and the auxiliary camera simultaneously, and compared with the subjective judgment of a user, the accuracy is higher.

Example 2

The present embodiment provides a camera occlusion detection system 50, as shown in fig. 5, including a control module 51, an obtaining module 52, and a determining module 53.

The control module 51 is configured to control the main camera and the auxiliary camera to perform shooting in response to entering a non-single-shot shooting mode. And the space areas shot by the main camera and the auxiliary camera are consistent.

The obtaining module 52 is configured to obtain a first picture taken by the main camera and a second picture taken by the auxiliary camera.

In an optional implementation manner, the obtaining module is specifically configured to pre-process pictures taken by the main camera and the auxiliary camera respectively to obtain a first picture and a second picture with the same view angle and size.

The determining module 53 is configured to determine whether the secondary camera is occluded according to the similarity between the first picture and the second picture.

In an optional implementation manner, the determining module specifically includes:

In an optional implementation manner, the calculating unit specifically includes:

It should be noted that the camera shielding detection system may be a separate chip, a chip module, or a terminal, or may be a chip or a chip module integrated in a terminal.

The modules/units included in the camera occlusion detection system described in the above embodiments may be software modules/units, or may also be hardware modules/units, or may also be part of the software modules/units and part of the hardware modules/units. For example, for each device and product applied to or integrated in a chip, each module/unit included in the device and product may be implemented by hardware such as a circuit, or at least a part of the modules/units may be implemented by a software program running on a processor integrated in the chip, and the rest of the modules/units may be implemented by hardware such as a circuit; for each device and product applied to or integrated with the chip module, each module/unit included in the device and product may be implemented in a hardware manner such as a circuit, and different modules/units may be located in the same component (e.g., a chip, a circuit module, etc.) or different components of the chip module, or at least a part of the modules/units may be implemented in a software program running on a processor integrated inside the chip module, and the remaining part of the modules/units may be implemented in a hardware manner such as a circuit; for each device and product applied to or integrated in the terminal, each module/unit included in the device and product may be implemented by using hardware such as a circuit, and different modules/units may be located in the same component (e.g., a chip, a circuit module, etc.) or different components in the terminal, or at least part of the modules/units may be implemented by using a software program running on a processor integrated in the terminal, and the rest of the modules/units may be implemented by using hardware such as a circuit.

Example 3

Fig. 6 is a schematic structural diagram of an electronic device provided in this embodiment. The electronic device comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein when the processor executes the program, the camera occlusion detection method of embodiment 1 is realized. The electronic device 3 shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiment of the present invention.

The electronic device in the embodiment of the present invention may be a mobile phone, a tablet computer, a PDA (Personal Digital Assistant), a Point of Sales (POS), a vehicle-mounted terminal, a wearable device, and the like, wherein the wearable device may be a smart band, a smart watch, a wrist band, smart glasses, a necklace, and the like.

The components of the electronic device 3 may include, but are not limited to: the at least one processor 4, the at least one memory 5, and a bus 6 connecting the various system components (including the memory 5 and the processor 4).

The bus 6 includes a data bus, an address bus, and a control bus.

The memory 5 may include volatile memory, such as Random Access Memory (RAM)51 and/or cache memory 52, and may further include Read Only Memory (ROM) 53.

The memory 5 may also include a program/utility 55 having a set (at least one) of program modules 54, such program modules 54 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.

The processor 4 executes various functional applications and data processing, such as the camera occlusion detection method of embodiment 1 of the present invention, by running the computer program stored in the memory 5.

The electronic device 3 may also communicate with one or more external devices 7, such as a keyboard, pointing device, etc. Such communication may be via an input/output (I/O) interface 8. Also, the electronic device 3 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the internet) via the network adapter 9. As shown in fig. 6, the network adapter 9 communicates with other modules of the electronic device 3 via the bus 6. It should be appreciated that although not shown in FIG. 6, other hardware and/or software modules may be used in conjunction with the electronic device 3, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems, etc.

It should be noted that although in the above detailed description several units/modules or sub-units/modules of the electronic device are mentioned, such a division is merely exemplary and not mandatory. Indeed, the features and functionality of two or more of the units/modules described above may be embodied in one unit/module according to embodiments of the invention. Conversely, the features and functions of one unit/module described above may be further divided into embodiments by a plurality of units/modules.

Example 4

The present embodiment provides a computer-readable storage medium on which a computer program is stored, which when executed by a processor implements the steps of the camera occlusion detection method of embodiment 1.

More specific examples, among others, that the readable storage medium may employ may include, but are not limited to: a portable disk, a hard disk, random access memory, read only memory, erasable programmable read only memory, optical storage device, magnetic storage device, or any suitable combination of the foregoing.

In a possible implementation manner, the present invention can also be implemented in the form of a program product, which includes program code for causing a terminal device to execute the steps of implementing the camera occlusion detection method of embodiment 1 when the program product is run on the terminal device.

Where program code for carrying out the invention is written in any combination of one or more programming languages, the program code may be executed entirely on the user device, partly on the user device, as a stand-alone software package, partly on the user device and partly on a remote device or entirely on the remote device.

While specific embodiments of the invention have been described above, it will be appreciated by those skilled in the art that this is by way of example only, and that the scope of the invention is defined by the appended claims. Various changes and modifications to these embodiments may be made by those skilled in the art without departing from the spirit and scope of the invention, and these changes and modifications are within the scope of the invention.

Claims

1. A camera occlusion detection method is characterized by comprising the following steps:

2. The method for detecting camera occlusion according to claim 1, wherein the acquiring a first picture taken by the main camera and a second picture taken by the auxiliary camera specifically includes:

3. The method for detecting occlusion of a camera according to claim 1 or 2, wherein the determining whether the secondary camera is occluded according to the similarity between the first picture and the second picture specifically comprises:

calculating the similarity of the first picture and the second picture;

judging whether the similarity is lower than a preset value or not;

if yes, determining that the auxiliary camera is shielded;

if not, determining that the auxiliary camera is not shielded.

4. The method according to claim 3, wherein the calculating the similarity between the first picture and the second picture specifically comprises:

5. A camera occlusion detection system, comprising:

6. The camera occlusion detection system of claim 5, wherein the obtaining module is specifically configured to pre-process the pictures taken by the main camera and the auxiliary camera, respectively, to obtain a first picture and a second picture with the same field angle and size.

7. The camera occlusion detection system of claim 5 or 6, wherein the determining module specifically comprises:

8. The camera occlusion detection system of claim 7, wherein the computing unit specifically comprises:

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the camera occlusion detection method of any of claims 1-4 when executing the computer program.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the camera occlusion detection method of any one of claims 1-4.