CN110710202A

CN110710202A - Image processing method and device

Info

Publication number: CN110710202A
Application number: CN201780091705.0A
Authority: CN
Inventors: 谢俊
Original assignee: Shenzhen Royole Technologies Co Ltd
Current assignee: Shenzhen Royole Technologies Co Ltd
Priority date: 2017-10-09
Filing date: 2017-10-09
Publication date: 2020-01-17
Also published as: WO2019071403A1

Abstract

An image processing method and device are applied to a terminal device, the terminal device comprises a first lens and a second lens, and the first lens and the second lens are used for capturing an image of a target shooting object, wherein the method comprises the following steps: performing face detection on a first image captured by the first lens to acquire feature information of a face area in the first image; determining a region of interest in a second image captured by the second lens according to feature information of a face region in the first image; and performing face detection on the region of interest in the second image to acquire feature information of the face region in the second image. The embodiment of the invention can effectively reduce the calculated amount of face detection and improve the image processing efficiency.

Description

Image processing method and device

Technical Field

The present invention relates to the field of image processing technologies, and in particular, to an image processing method and apparatus.

Background

With the continuous development and progress of scientific technology, two-dimensional pictures shot by a monocular camera cannot meet the requirements of picture display forms, so that a binocular camera and a multi-lens camera device enter the lives of people, the binocular camera or the multi-lens camera device capture at least two images, the synthesized pictures are various in forms, and similar applications such as facial expression replacement, decoration adding and the like can be realized by combining with Augmented Reality (AR) technology. At present, in order to implement the above applications, face detection needs to be performed on all regions in at least two images captured by a binocular camera or a multi-lens camera, and therefore, the calculation amount is large and the image processing efficiency is low.

Disclosure of Invention

The embodiment of the invention discloses an image processing method and device, which can effectively reduce the calculated amount of face detection and improve the image processing efficiency.

The first aspect of the embodiments of the present invention discloses an image processing method, applied to a terminal device, where the terminal device includes a first lens and a second lens, and the first lens and the second lens are used to capture an image of a target photographic object, where the method includes:

performing face detection on a first image captured by the first lens to acquire feature information of a face area in the first image;

determining a region of interest in a second image captured by the second lens according to feature information of a face region in the first image;

and performing face detection on the region of interest in the second image to acquire feature information of the face region in the second image.

A second aspect of an embodiment of the present invention discloses an image processing apparatus applied to a terminal device, where the terminal device includes a first lens and a second lens, and the first lens and the second lens are used to capture an image of a target photographic object, the apparatus including:

the detection module is used for carrying out face detection on a first image captured by the first lens so as to acquire characteristic information of a face area in the first image;

the determining module is used for determining a region of interest in a second image captured by the second lens according to the feature information of the face region in the first image;

the detection module is further configured to perform face detection on the region of interest in the second image to obtain feature information of the face region in the second image.

A third aspect of the embodiments of the present invention discloses a computer-readable storage medium, in which instructions are stored, and when the instructions are executed on a computer, the instructions cause the computer to execute the image processing method according to the first aspect.

A fourth aspect of the embodiments of the present invention discloses a computer program product containing instructions, which, when run on a computer, cause the computer to execute the image processing method according to the first aspect.

The embodiment of the invention can perform face detection on the first image captured by the first lens to acquire the feature information of the face region in the first image, determine the region of interest in the second image captured by the second lens according to the feature information of the face region in the first image, and perform face detection on the region of interest in the second image to acquire the feature information of the face region in the second image, so that the face detection can be performed only on the region of interest in the second image, and compared with the face detection performed on all regions in the second image, the face detection calculation amount can be effectively reduced, and the image processing efficiency is improved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without inventive labor.

FIG. 1 is a flow chart of an image processing method according to an embodiment of the present invention;

FIG. 2a is a schematic diagram of a face recognition system according to an embodiment of the present invention;

FIG. 2b is a schematic diagram of another face recognition disclosed in the embodiments of the present invention;

fig. 3 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present invention.

Detailed Description

The technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

The embodiment of the invention discloses an image processing method and device, which are applied to terminal equipment, wherein the terminal equipment comprises a first lens and a second lens, and the first lens and the second lens are used for capturing an image of a target shooting object. The image processing method and apparatus provided by the embodiments of the present invention are used to reduce the calculation amount of face detection to improve the image processing efficiency, and are described in detail below.

The image processing method provided by the embodiment of the invention can be implemented by terminal Devices with multiple lenses, such as smart phones (e.g., Android phones, iOS phones, Windows Phone phones, etc.), tablet computers, Mobile Internet Devices (MID), intelligent wearable Devices, and shooting Devices.

Fig. 1 is a schematic flow chart of an image processing method according to an embodiment of the present invention. The image processing method described in the present embodiment includes, but is not limited to, the following steps:

s101, the terminal device carries out face detection on a first image captured by a first lens to obtain feature information of a face area in the first image.

The terminal device is configured with a first lens and a second lens, the terminal device may be a binocular camera, for example, and the first lens may be any one of a plurality of lenses included in the terminal device. In the process that a terminal device user takes a picture (for example, self-timer shooting) of a target shooting object (for example, a human face) by using a first lens and a second lens configured by the terminal device, the first lens can capture a first image of the target shooting object, and the first image can be a first preview image of the target shooting object captured by the first lens on a screen of the terminal device.

In a specific implementation, after a first image of a target shooting object is captured by the first lens, the terminal device performs face detection on all regions in the first image by using a face recognition technology, so as to acquire feature information of the face region in the first image. The feature information of the face region acquired by the terminal device includes: one or more of a position of the face region, an area of the face region, and a position of five sense organs in the face region.

The target photographic subject may be a human being or an animal having a face.

S102, the terminal equipment determines an area of interest in a second image captured by a second lens according to the feature information of the face area in the first image.

Wherein the second lens is another lens other than the first lens among a plurality of lenses included in the terminal device. In the process that a terminal device user utilizes the first lens and the second lens configured by the terminal device to take a picture of a target photographic object, the second lens can capture a second image of the target photographic object, and the second image can be a second preview image of the target photographic object captured by the second lens on a screen of the terminal device.

In specific implementation, the feature information of the face region in the first image acquired by the terminal device includes the position and the area of the face region in the first image, and first, the terminal device determines a target region in the second image captured by the second lens according to the position and the area of the face region in the acquired first image. Then, the terminal device determines a parallax region in the second image based on the area of the face region in the first image and the target region in the second image. Specifically, the terminal device first calculates an arithmetic square root of the area of the face region in the first image, resulting in an arithmetic square root value. Next, the terminal device multiplies the parallax conversion coefficient by the arithmetic square root value to obtain the parallax length of the face region of the target photographic subject in the second image, that is, the distance of the position of the face region of the target photographic subject in the second image with respect to the target region. Wherein, the parallax conversion coefficient can be preset by the terminal device.

Wherein, when the first lens and the second lens are arranged in the horizontal direction, the parallax length may be a position of the face region of the target photographic subject in the second image, with respect to the parallax length in the horizontal direction of the target region; when the first lens and the second lens are disposed in the vertical direction, the parallax length may be a position of the face region of the target photographic subject in the second image, with respect to the parallax length in the vertical direction of the target region. The parallax length may be a parallax length in an arbitrary direction with respect to the target region, which is a position of the face region of the target photographic subject in the second image.

Further, the terminal device may determine the position of the face region of the target photographic subject in the second image relative to the target region based on the relative positions of the first lens and the second lens, where the relative positions of the first lens and the second lens refer to the relative orientation between the first lens and the second lens, for example, whether the first lens is located to the left or to the right of the second lens. If the parallax length is a parallax length in the horizontal direction, it may be determined whether the position of the face area of the target photographic subject in the second image is on the left or right of the target area according to the relative positions of the first lens and the second lens. If the parallax length is a parallax length in the vertical direction, it may be determined whether the position of the face area of the target photographic subject in the second image is above or below the target area according to the relative positions of the first lens and the second lens. If the parallax length is a parallax length in an arbitrary direction, it is possible to determine whether the position of the face area of the target photographic subject in the second image is above left, below left, above right, below right, or the like of the target area, based on the relative positions of the first lens and the second lens. After determining the position of the face region of the target photographic object in the second image relative to the target region, the terminal device determines the parallax region in the second image according to the parallax length, the target region, and the position of the face region in the second image relative to the target region, and then the terminal device may determine the target region and the parallax region in the second image as the region of interest in the second image.

In some possible embodiments, the area of the target region is equal to the area of the face region in the first image, and the position of the target region in the second image is the same as the position of the face region in the first image. The area of the target region may not be equal to the area of the face region in the first image, and the position of the target region in the second image may not be exactly the same as the position of the face region in the first image. The face region in the first image may be a rectangular region including a face in the first image, the face region in the second image may be a rectangular region including a face in the second image, and the target region, the parallax region, and the region of interest in the second image may also be rectangular regions. The face region, the target region, the parallax region, and the region of interest may be other shaped regions such as an elliptical region, and the embodiment of the present invention is not limited thereto.

For example, the above-described face region, target region, parallax region, and region of interest are rectangular regions, and the target photographic subject is a human example. Referring to fig. 2a and 2b together, it is assumed that fig. 2a is a first image of a person captured by the first lens and fig. 2b is a second image of a person captured by the second lens. After the first image is captured by the first lens (fig. 2a), and the second image is captured by the second lens (fig. 2b), the terminal device first performs face detection on all the regions in fig. 2a, can detect that the solid-line rectangular region including the face in fig. 2a is the face region in fig. 2a, and obtains the position and area of the solid-line rectangular region including the face in fig. 2 a. The target area, i.e. the dashed rectangular area comprising the face in fig. 2b, is then determined in fig. 2 b. It is assumed that the position of the dotted rectangular region including the face in fig. 2b is the same as the position of the solid rectangular region including the face in fig. 2a, and the area of the dotted rectangular region including the face is equal to the area of the solid rectangular region including the face. Next, the parallax length of the face region in fig. 2b is determined according to the area of the solid-line rectangular region including the face in fig. 2a and the preset parallax conversion coefficient, as shown by L in fig. 2 b. It is further possible to determine the position of the face region in fig. 2b with respect to the dashed rectangular region including the face based on the relative positions of the first lens and the second lens, and assuming that the first lens and the second lens of the terminal device are horizontally disposed and the first lens is on the left side of the second lens, it is possible to determine that the face region in fig. 2b is on the left side of the dashed rectangular region including the face, as indicated by the region indicated by L in fig. 2 b. And finally, determining a parallax region according to the length of the dotted rectangular region including the human face in the vertical direction and the parallax length L, as shown by the solid-line rectangular region in fig. 2b, and taking the solid-line rectangular region in fig. 2b and the dotted rectangular region including the human face as the region of interest in fig. 2 b.

In some possible embodiments, after determining the target region and the parallax region in the second image, the terminal device may determine a partial region of the target region and the parallax region as the region of interest in the second image, for example, the left half of the solid-line rectangular region and the dashed-line rectangular region including the human face in fig. 2b may be used as the region of interest in fig. 2 b. It should be noted that, for some special photographing scenes, the parallax region in the second image may be directly used as the region of interest.

In some possible embodiments, the parallax conversion coefficient preset by the terminal device may be calculated in advance according to at least one image obtained by taking a picture of the photographic subject by the first lens and at least one image obtained by taking a picture of the photographic subject by the second lens, where the photographic subject may be a subject photographed by the terminal device in a certain photographing process in advance, and the photographic subject may be the same as a target photographic subject photographed this time or different from the target photographic subject photographed this time. Specifically, the terminal device first acquires an image (hereinafter referred to as a third image) obtained by photographing a photographic subject with a first lens and an image (hereinafter referred to as a fourth image) obtained by photographing a photographic subject with a second lens in the same photographing process; then, respectively carrying out face detection on the third image and the fourth image so as to determine the position and the area of the face region in the third image and the position and the area of the face region in the fourth image; comparing the positions of the face area of the object in the third image and the fourth image respectively, for example, overlapping and comparing the third image and the fourth image to obtain the parallax length of the face area of the object in the third image and the fourth image; and finally, calculating to obtain a parallax conversion coefficient by using the area of the face area of the shooting object in the third image or the fourth image and the parallax length of the face area of the shooting object in the third image and the fourth image. It should be noted that, with the above-mentioned manner, the terminal device may obtain a plurality of parallax conversion coefficients according to images acquired by the first lens and the second lens simultaneously in different photographing processes, and the terminal device obtains an average parallax conversion coefficient according to the plurality of parallax conversion coefficients, and uses the average parallax conversion coefficient as a preset parallax conversion coefficient. The preset parallax conversion coefficient may be a value, or a range composed of at least two parallax conversion coefficients.

In some possible embodiments, the parallax conversion coefficient may also be calculated by the terminal device according to the relative position of the first lens and the second lens, the distance from the target photographic subject to the baseline, and the parameter (e.g. the angle of view) of the first lens or the second lens, where the relative position of the first lens and the second lens specifically refers to the length of the connecting line (i.e. the baseline) between the first lens and the second lens.

In some possible embodiments, the parallax conversion coefficient preset by the terminal device may be a maximum parallax conversion coefficient between the first lens and the second lens, which is calculated by the terminal device according to parameters such as a base length between the first lens and the second lens, a distance from the target photographic subject to the base line, and a minimum field angle of the first lens or the second lens, and according to the maximum parallax conversion coefficient, the parallax length between the first image and the second image may be determined to ensure that the region of interest in the second image includes the face region of the target photographic subject.

In some possible embodiments, the base length between the first lens and the second lens of the terminal device is adjustable, that is, the distance between the first lens and the second lens is adjustable. The maximum parallax conversion coefficient is calculated by the terminal device according to parameters such as the maximum base length between the first lens and the second lens, the distance from the target shooting object to the base line, and the minimum field angle of the first lens or the second lens.

S103, the terminal device performs face detection on the region of interest in the second image to acquire feature information of the face region in the second image.

In specific implementation, after determining the region of interest in the second image, the terminal device performs face detection on the region of interest in the second image by using a face recognition technology, so as to obtain feature information of the face region in the second image. The feature information of the face region acquired by the terminal device includes: one or more of a position of the face region, an area of the face region, and a position of five sense organs in the face region. By adopting the mode, the terminal equipment only needs to detect the face of the region of interest in the second image, and compared with the method for detecting the face of all the regions in the second image, the embodiment of the invention can effectively reduce the region for detecting the face, thereby reducing the calculated amount of the face detection and improving the image processing efficiency.

In some possible embodiments, the feature information of the face region in the second image acquired by the terminal device includes the position of the five sense organs of the face region in the second image; the feature information of the face region in the first image acquired by the terminal device includes the position of the five sense organs of the face region in the first image. After the terminal device obtains the feature information of the face region in the second image, the terminal device may perform AR special effect processing on the first image according to the position of the five sense organs of the face region in the first image to obtain a first AR image; and performing AR special effect processing on the second image according to the position of the five sense organs of the face area in the second image to obtain a second AR image. Further, the first AR image and the second AR image may be synthesized to obtain an AR image of the target photographic subject. The obtained AR image of the target shooting object can fuse the characteristic information of the two images of the target shooting object captured by the first lens and the second lens of the terminal device, so that the AR effect is further enhanced.

In the embodiment of the invention, the terminal device performs face detection on the first image captured by the first lens to acquire the feature information of the face region in the first image, then determines the region of interest in the second image captured by the second lens according to the feature information of the face region in the first image, and further performs face detection on the region of interest in the second image to acquire the feature information of the face region in the second image, so that face detection can be performed only on the region of interest in the second image, and compared with face detection performed on all regions in the second image, the face detection method and device can effectively reduce the calculation amount of face detection and improve the image processing efficiency.

In the embodiment of the method, the main purpose of step S102 is to estimate the position of the face region in the second image according to the feature information of the face region in the first image, so as to determine the region of interest in the second image, so as to reduce the region of face detection in the second image, thereby reducing the amount of calculation and improving the image processing efficiency of the second image. Some principles on which embodiments of the invention are based will be explained below. For a terminal device including multiple lenses, when the terminal device captures an image of a target photographic subject using the multiple lenses, a first image and a second image of the target photographic subject may be captured, but positions of the target photographic subject in the first image and the second image calculated by image pixels are different, that is, a parallax may occur in the first image and the second image of the target photographic subject, and the parallax may be a parallax in a horizontal direction or a vertical direction. It can be proved that the parallax of the target photographic object in the first image and the second image is inversely proportional to the distance between the target photographic object and the terminal device, wherein the distance between the target photographic object and the terminal device refers to the distance between the target photographic object and the connection line between the first lens and the second lens included in the terminal device, and is not described in detail later; the parallax size of the target shooting object in the first image and the second image is in direct proportion to the distance between the first lens and the second lens, namely the length of the base line; the parallax of the target object in the first image and the second image is inversely related to the field angle of the first lens or the second lens, and may be inversely proportional to the sine of one half of the field angle, because the smaller the field angle is, the smaller the image of the target object in the image is, which is equivalent to being enlarged, and the parallax is also enlarged; it can also be shown that the pixel length of the target photographic subject in the image is inversely proportional to the distance of the target photographic subject from the terminal device. In summary, it can be concluded that the parallax size of the target photographic object in the first image and the second image is proportional to the pixel length of the target photographic object in the images. Since the shorter the pixel length of the target photographic subject in the image, the farther the representative target photographic subject is from the dual lens, the smaller the parallax of the target photographic subject in the first image and the second image. Further, it can be concluded that the area of the target photographic object in the image is proportional to the square of the magnitude of the parallax of the target photographic object in the first image and the second image.

In summary, the main idea of the embodiment of the present invention is that after the first lens and the second lens of the terminal device capture the first image and the second image of the target photographic object, the terminal device first performs face detection on all regions in the first image by using a face recognition technology to obtain the position and area of the face region in the first image; then determining the parallax length of the target shooting object in the second image according to the area of the face area in the first image; and then the region of interest is determined in the second image. The specific implementation manner may refer to the related description in the above method embodiments, and is not described herein again.

In some possible embodiments, to ensure that the face region in the second image is within the region of interest in the second image for most or all of the scenes, the preset parallax conversion coefficient may be a parallax conversion coefficient of the maximum parallax length of the terminal device, that is, a maximum parallax conversion coefficient between the first lens and the second lens of the terminal device, so that a situation that there is no face or only a partial face region in the region of interest in the second image due to the region of interest in the second image being too small can be avoided. The parallax conversion coefficient that determines the maximum parallax length of the terminal device will be described below. In the actual calculation, if a smaller face area (for example, the face area of a child) is used as a standard, the calculated parallax will be larger, and the range of the region of interest can be increased reasonably. When the actual target photographic subject is an adult, the terminal device considers that the target photographic subject is at a close position, and the calculated parallax is also larger. If a larger face area (for example, the face area of an adult) is used as a standard, and the actual target photographic subject is a child, the terminal device may consider that the target photographic subject is at a far position, and thus the calculated parallax is smaller. Therefore, when determining the parallax conversion coefficient of the maximum parallax length of the terminal device, the smaller face area (for example, the face area of a child) of the face areas should be used as a standard; the field angle of the lens of the terminal device is also in a range (for example, 50 degrees to 67 degrees), and the field angle of the first lens or the second lens should be set to the minimum field angle because the smaller the field angle, the larger the parallax. The distance between the first lens and the second lens of the terminal device is also a certain range or fixed, and for the case that the distance between the first lens and the second lens (i.e. the base length) is variable, the distance between the first lens and the second lens should be set to be the maximum distance, because the larger the distance between the first lens and the second lens is, the larger the parallax is. Further, the terminal device takes a picture of the photographic subject with a smaller face area when the lengths of the base lines of the first lens and the second lens are the largest and the field angle of the first lens or the second lens is the smallest, and then calculates a parallax conversion coefficient corresponding to the maximum parallax length of the terminal device according to the image of the photographic subject taken by the first lens and the image of the photographic subject taken by the second lens, where the specific calculation process may refer to the foregoing description and is not described herein again.

In some possible embodiments, the terminal device may further calculate a range of the parallax conversion coefficient commonly used by the terminal device according to a commonly used scenario of the terminal device (for example, a variation range of a face area of a captured target photographic subject, a variation range of a lens angle, and the like). When determining the region of interest in the second image according to the face area in the first image, the range of the parallax length in the second image may be determined in combination with the range of the parallax conversion coefficient generally used by the terminal device, or the maximum possible parallax and the minimum possible parallax of the face region of the target photographic subject in the second image in the current scene may be calculated, and then the region of interest in the second image may be determined according to the maximum possible parallax and the minimum possible parallax. For the way in fig. 2b the region of interest in the second image is determined, only the maximum possible disparity is used, for which the default is 0.

In some possible embodiments, the parameters of the angle of view and the distance between lenses are known. The parallax conversion coefficient preset by the terminal device may be calculated in advance according to at least two images obtained by photographing a plurality of lenses of the terminal device. As is apparent from the above description, the area of the target photographic subject in the image is proportional to the square of the magnitude of the parallax of the target photographic subject in the first image and the second image, and the following relationship L ═ K × S can be obtained, where L is the parallax length of the target photographic subject in the first image and the second image, S is the arithmetic square root of the area of the face region in the first image, K is the parallax conversion coefficient, and L and S have a linear relationship. Therefore, the terminal device first obtains a first image and a second image of a target shooting object obtained by shooting the first lens and the second lens at the same time, and then performs face detection on the first image and the second image respectively by using, for example, a face recognition technology to determine the position and the area of a face area in the first image and the second image; determining the parallax length L of the target shooting object in the first image and the second image according to the position of the face area in the first image and the position of the face area in the second image; and finally, dividing the parallax length L by the arithmetic square root S of the area of the face area in the first image or the second image to obtain a parallax conversion coefficient K.

In the above-described aspect, the parallax conversion coefficient is calculated when the parameters of the first lens and the second lens of the terminal device are matched and the areas of the face regions of the target photographic subjects in the first image and the second image are matched. For the condition that the parameters of the first lens and the second lens of the terminal device are not consistent, the terminal device may first acquire a first image and a second image of a target photographic object, which are obtained by the terminal device by photographing at the same time, assuming that the first image corresponds to the first lens and the second image corresponds to the second lens; then, for example, face detection is carried out on the first image and the second image respectively by face recognition technology so as to determine the position and the area of the face region in the first image and the second image; determining the parallax length L of the target shooting object in the first image and the second image according to the position of the face area in the first image and the position of the face area in the second image; finally, dividing the parallax length L by the arithmetic square root S1 of the area of the face area in the first image to obtain a parallax conversion coefficient K1 of the first lens; the parallax conversion coefficient K2 of the second lens is obtained by dividing the parallax length L by the arithmetic square root S2 of the area of the face region in the second image.

In some possible embodiments, in order to ensure the accuracy of the calculated parallax conversion coefficient, the parallax conversion coefficient may be calculated again according to at least two images of different target photographic objects photographed by the terminal device at different times in the above manner, so as to verify the previous result.

Fig. 3 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present invention. The image processing apparatus described in the present embodiment includes:

a detection module 301, configured to perform face detection on a first image captured by the first lens to obtain feature information of a face area in the first image.

A determining module 302, configured to determine a region of interest in a second image captured by the second lens according to feature information of a face region in the first image.

The detection module 301 is further configured to perform face detection on the region of interest in the second image to obtain feature information of the face region in the second image.

In some possible embodiments, the feature information includes one or more of a position, an area, and a position of a five sense organ in the face region.

In some possible embodiments, the feature information includes a position and an area, and the determining module 302 includes:

a first determining unit 3021 configured to determine a target region in a second image captured by the second lens according to a position and an area of a face region in the first image.

A second determination unit 3022 configured to determine a parallax region in the second image based on an area of the face region in the first image and the target region in the second image.

A third determination unit 3023 configured to determine the target region and the parallax region as a region of interest in the second image.

A third determination unit 3023 configured to determine a partial region in the target region and the parallax region as a region of interest in the second image.

In some possible embodiments, the area of the target region is equal to the area of the face region in the first image, and the position of the target region in the second image is the same as the position of the face region in the first image.

In some possible embodiments, the second determining unit 3022 is specifically configured to:

determining a first parallax length of the face region of the target photographic subject in the second image according to the area of the face region in the first image and the parallax conversion coefficient;

and determining a parallax area in the second image according to the first parallax length, the relative positions of the first lens and the second lens and a target area in the second image.

In some possible embodiments, the parallax conversion coefficient is calculated according to a minimum field angle of the first lens or the second lens, a relative position of the first lens and the second lens, and a distance between a photographic subject and the terminal device.

In some possible embodiments, the first parallax length includes a parallax length in a horizontal direction or a parallax length in a vertical direction.

In some possible embodiments, the image processing apparatus further comprises an obtaining module 303 and a calculating module 304, wherein:

the obtaining module 303 is configured to obtain at least one image obtained by photographing the first lens, and obtain at least one image obtained by photographing the second lens at the same time.

The detection module 301 is further configured to perform face detection on at least one image obtained by photographing the first lens and at least one image obtained by photographing the second lens, determine a position and an area of a face region in the at least one image obtained by photographing the first lens, and determine a position and an area of a face region in the at least one image obtained by photographing the second lens.

The determining module 302 is further configured to determine a second parallax length of the face area of the photographic subject in the at least one image obtained by photographing the first lens and the at least one image obtained by photographing the second lens according to the position of the face area in the at least one image obtained by photographing the first lens and the position of the face area in the at least one image obtained by photographing the second lens.

The calculating module 304 is configured to calculate at least one parallax conversion coefficient by using the area of the face region in the at least one image obtained by photographing with the first lens, the area of the face region in the at least one image obtained by photographing with the second lens, and the second parallax length.

In some possible embodiments, the feature information further includes positions of five sense organs in the face region, the image processing apparatus further includes a processing module 305 and a synthesizing module 306, wherein:

the processing module 305 is configured to perform augmented reality AR special effect processing on the first image according to the position of the five sense organs in the face area in the first image, so as to obtain a first AR image.

The processing module 305 is further configured to perform AR special effect processing on the second image according to the position of the five sense organs in the face area in the second image, so as to obtain a second AR image.

The synthesizing module 306 is configured to synthesize the first AR image and the second AR image to obtain an AR image of the target photographic subject.

It can be understood that the functions of each functional module and unit of the image processing apparatus in this embodiment may be specifically implemented according to the method in the foregoing method embodiment, and the specific implementation process may refer to the related description of the foregoing method embodiment, which is not described herein again.

In the embodiment of the present invention, the detection module 301 performs face detection on the first image captured by the first lens to obtain feature information of a face region in the first image, the determination module 302 determines a region of interest in the second image captured by the second lens according to the feature information of the face region in the first image, and the detection module 301 performs face detection on the region of interest in the second image to obtain feature information of the face region in the second image, so that face detection can be performed only on the region of interest in the second image, and compared with performing face detection on all regions in the second image, the calculation amount of face detection can be effectively reduced, and the image processing efficiency is improved.

The present invention also provides a computer-readable storage medium having stored therein instructions, which, when run on a computer, cause the computer to perform the image processing method described in the above-mentioned method embodiment.

The present invention also provides a computer program product comprising instructions which, when run on a computer, cause the computer to perform the image processing method as described in the above method embodiments.

The invention further provides a terminal device, which comprises a processor, an input device, an output device and a memory, wherein the processor, the input device, the output device and the memory are connected with each other, the memory is used for storing a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions and execute the image processing method according to the above method embodiment.

It should be noted that, for simplicity of description, the above-mentioned embodiments of the method are described as a series of acts or combinations, but those skilled in the art will recognize that the present invention is not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the invention. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required by the invention.

The steps in the method of the embodiment of the invention can be sequentially adjusted, combined and deleted according to actual needs. The modules and units in the device provided by the embodiment of the invention can be combined, divided and deleted according to actual needs.

Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by associated hardware instructed by a program, which may be stored in a computer-readable storage medium, and the storage medium may include: flash disks, Read-Only memories (ROMs), Random Access Memories (RAMs), magnetic or optical disks, and the like.

While the invention has been described with reference to a preferred embodiment, it will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

Claims

An image processing method applied to a terminal device, the terminal device comprising a first lens and a second lens, the first lens and the second lens being used for capturing an image of a target photographic object, the method comprising:

performing face detection on a first image captured by the first lens to acquire feature information of a face area in the first image;

determining a region of interest in a second image captured by the second lens according to feature information of a face region in the first image;

and performing face detection on the region of interest in the second image to acquire feature information of the face region in the second image.
The method of claim 1, wherein the feature information includes one or more of a location, an area, and a position of a five sense organ in the face region.
The method of claim 2, wherein the feature information comprises the location and the area, and wherein determining the region of interest in the second image captured by the second lens from the feature information of the face region in the first image comprises:

determining a target area in a second image captured by the second lens according to the position and the area of the face area in the first image;

determining a parallax region in the second image according to the area of the face region in the first image and the target region in the second image;

determining the target region and the parallax region as a region of interest in the second image.
The method of claim 2, wherein the feature information comprises the location and the area, and wherein determining the region of interest in the second image captured by the second lens from the feature information of the face region in the first image comprises:

determining a target area in a second image captured by the second lens according to the position and the area of the face area in the first image;

determining a parallax region in the second image according to the area of the face region in the first image and the target region in the second image;

determining a partial region in the target region and the parallax region as a region of interest in the second image.
The method according to claim 3 or 4, wherein the area of the target region is equal to the area of the face region in the first image, and the position of the target region in the second image is the same as the position of the face region in the first image.
The method according to claim 3 or 4, wherein determining the parallax region in the second image according to the area of the face region in the first image and the target region in the second image comprises:

determining a first parallax length of the face region of the target photographic subject in the second image according to the area of the face region in the first image and the parallax conversion coefficient;

and determining a parallax area in the second image according to the first parallax length, the relative positions of the first lens and the second lens and a target area in the second image.
The method according to claim 6, wherein the parallax conversion coefficient is calculated from a minimum field angle of the first lens or the second lens, relative positions of the first lens and the second lens, and a distance between a photographic subject and the terminal device.
The method of claim 6, wherein the first disparity length comprises a disparity length in a horizontal direction or a disparity length in a vertical direction.
The method according to claim 6 or 7, wherein before the face detection of the first image captured by the first lens to obtain the feature information of the face region in the first image, the method further comprises:

acquiring at least one image obtained by photographing through the first lens, and acquiring at least one image obtained by photographing through the second lens;

respectively carrying out face detection on at least one image obtained by photographing the first lens and at least one image obtained by photographing the second lens, determining the position and the area of a face region in the at least one image obtained by photographing the first lens, and determining the position and the area of the face region in the at least one image obtained by photographing the second lens;

determining a second parallax length of the face area of the shooting object in the at least one image obtained by the first lens shooting and the at least one image obtained by the second lens shooting according to the position of the face area in the at least one image obtained by the first lens shooting and the position of the face area in the at least one image obtained by the second lens shooting;

and calculating to obtain at least one parallax conversion coefficient by using the area of the face region in at least one image obtained by photographing through the first lens, the area of the face region in at least one image obtained by photographing through the second lens and the second parallax length.
The method according to claim 3 or 4, characterized in that the method further comprises: the feature information further includes the position of the five sense organs in the face region;

according to the position of the five sense organs of the face area in the first image, carrying out AR special effect processing on the first image to obtain a first AR image;

according to the position of the five sense organs in the face area in the second image, carrying out AR special effect processing on the second image to obtain a second AR image;

and synthesizing the first AR image and the second AR image to obtain an AR image of the target shooting object.
An image processing apparatus applied to a terminal device including a first lens and a second lens for capturing an image of a target photographic subject, the apparatus comprising:

the detection module is used for carrying out face detection on a first image captured by the first lens so as to acquire characteristic information of a face area in the first image;

the determining module is used for determining a region of interest in a second image captured by the second lens according to the feature information of the face region in the first image;

the detection module is further configured to perform face detection on the region of interest in the second image to obtain feature information of the face region in the second image.
The apparatus of claim 11, wherein the feature information comprises one or more of a location, an area, and a position of a five sense organ in the face region.
The apparatus of claim 12, wherein the characteristic information comprises the location and the area, and wherein the determining module comprises:

a first determination unit configured to determine a target region in a second image captured by the second lens, based on a position and an area of a face region in the first image;

a second determination unit configured to determine a parallax region in the second image based on an area of a face region in the first image and a target region in the second image;

a third determination unit configured to determine the target region and the parallax region as a region of interest in the second image.
The apparatus of claim 12, wherein the characteristic information comprises the location and the area, and wherein the determining module comprises:

a first determination unit configured to determine a target region in a second image captured by the second lens, based on a position and an area of a face region in the first image;

a second determination unit configured to determine a parallax region in the second image based on an area of a face region in the first image and a target region in the second image;

a third determination unit configured to determine a partial region in the target region and the parallax region as a region of interest in the second image.
The apparatus according to claim 13 or 14, wherein the area of the target region is equal to the area of the face region in the first image, and the position of the target region in the second image is the same as the position of the face region in the first image.
The apparatus according to claim 13 or 14, wherein the second determining unit is specifically configured to:

determining a first parallax length of the face region of the target photographic subject in the second image according to the area of the face region in the first image and the parallax conversion coefficient;

and determining a parallax area in the second image according to the first parallax length, the relative positions of the first lens and the second lens and a target area in the second image.
The apparatus according to claim 16, wherein the parallax conversion coefficient is calculated from a minimum field angle of the first lens or the second lens, relative positions of the first lens and the second lens, and a distance between a photographic subject and the terminal device.
The apparatus of claim 16, wherein the first disparity length comprises a disparity length in a horizontal direction or a disparity length in a vertical direction.
The apparatus of claim 16 or 17, further comprising an acquisition module and a calculation module, wherein:

the acquisition module is used for acquiring at least one image obtained by photographing the first lens and acquiring at least one image obtained by photographing the second lens;

the detection module is further configured to perform face detection on at least one image obtained by photographing the first lens and at least one image obtained by photographing the second lens, determine a position and an area of a face region in the at least one image obtained by photographing the first lens, and determine a position and an area of a face region in the at least one image obtained by photographing the second lens;

the determining module is further configured to determine a second parallax length of the face area of the photographic subject in the at least one image obtained by the first lens photographing and the at least one image obtained by the second lens photographing according to the position of the face area in the at least one image obtained by the first lens photographing and the position of the face area in the at least one image obtained by the second lens photographing;

the calculation module is configured to calculate at least one parallax conversion coefficient by using the area of the face region in the at least one image obtained by photographing with the first lens, the area of the face region in the at least one image obtained by photographing with the second lens, and the second parallax length.
The apparatus of claim 13 or 14, wherein the feature information further comprises the position of the five sense organs in the face region, the apparatus further comprising a processing module and a synthesizing module, wherein:

the processing module is used for carrying out augmented reality AR special effect processing on the first image according to the position of the five sense organs of the face area in the first image to obtain a first AR image;

the processing module is further configured to perform AR special effect processing on the second image according to the position of the five sense organs in the face area in the second image to obtain a second AR image;

the synthesis module is used for synthesizing the first AR image and the second AR image to obtain an AR image of the target shooting object.
A computer-readable storage medium having stored therein instructions which, when run on a computer, cause the computer to execute the image processing method according to any one of claims 1 to 10.
A computer program product comprising instructions which, when run on a computer, cause the computer to carry out the image processing method according to any one of claims 1 to 10.