WO2019090591A1

WO2019090591A1 - Photography method and photography system based on smart terminal

Info

Publication number: WO2019090591A1
Application number: PCT/CN2017/110170
Authority: WO
Inventors: 詹昌松
Original assignee: 深圳传音通讯有限公司
Priority date: 2017-11-09
Filing date: 2017-11-09
Publication date: 2019-05-16
Also published as: CN111345030A

Abstract

A photography method based on a smart terminal, comprising the following steps: collecting a preview image comprising a face of a photography subject; extracting a facial colour temperature value; determining whether the facial colour temperature value is lower than a preset colour temperature value; and, when the facial colour temperature value is lower than the preset colour temperature value, emitting a compensation light ray. The photography system based on a smart terminal comprises: a collection module, an extraction module, a determining module, and a compensation module.

Description

Photographing method based on intelligent terminal and photographing system

Technical field

The present invention relates to the technical field of intelligent terminals, and in particular, to a photographing method and a photographing system based on a smart terminal.

Background technique

Smart terminals have become an increasingly indispensable part of today's life. With the rapid development of smart terminals, its built-in functions are becoming more diverse. Listening to music, business, games, and taking pictures has become a must-have feature for smart terminals. At present, in order to further improve the functions and user experience of smart terminals, technicians have continuously innovated and optimized existing functions.

For example, for the camera function of the smart terminal, the constant efforts of the technicians have made a lot of contributions from the aspects of pixels, color, beauty, and the like. In order to further increase the shooting effect, the white balance option is often adjusted manually. The so-called white balance is to let the smart terminal shoot white in different ambient color temperatures to represent real white, where white is a standard, because if the white is accurate, then The other colors are also accurate, so it is called white balance. Those white balance modes such as cloudy, flash, fluorescent lights, etc., are all set by the smart terminal in response to different color temperatures, just to make the color of the photo closer to the real scene color. However, this adjustment often requires manual operation by the user, and an additional representation of the color is required by additionally filling the light with the tool or replacing the shooting scene.

To this end, it is necessary to provide a photographing method and a photographing system based on a smart terminal, which can be automatically operated without the need of manual operation, by means of a fill light tool, and by the limitation of the shooting location, by extracting the color temperature value of the face of the subject face. Different levels of compensation light are emitted according to the color temperature value of the face to ensure that the photo can present more realistic colors, providing users with a more optimized shooting effect and shooting experience.

Summary of the invention

The invention is based on a smart terminal photographing method and a photographing system. By extracting the color temperature value of the face of the photographed subject, the compensating light of different intensities can be automatically generated according to the color temperature value of the face to ensure that the photo can be presented more. Real colors provide users with a more optimized shooting and shooting experience.

The invention provides a photographing method for a smart terminal, and the photographing method comprises the following steps:

Collecting a preview image of a face containing a subject;

Extracting a facial color temperature value of the face;

Determining whether the color temperature value of the face is lower than a preset color temperature value;

When the color temperature value of the face is lower than a preset color temperature value, a compensation light is emitted.

Preferably, the step of extracting the color temperature value of the face of the face further comprises:

Performing face recognition on the face to determine an eye position in the face;

The average eye color temperature value of the eye position is detected as a facial color temperature value.

Preferably, the compensation light is emitted by a soft light and/or a flash of the smart terminal.

Preferably, the intensity of the compensated light is adjusted by the flash according to a difference between the facial color temperature value and the preset color temperature value.

Preferably, after the step of emitting a compensation light when the color temperature value of the face is lower than a preset color temperature value, the photographing method further comprises:

Extracting a compensated color temperature value after the face receives the compensation light;

Determining whether the compensated color temperature value is lower than a preset color temperature value;

When the compensated color temperature value is equal to or higher than the preset color temperature value, a reminder to remind the photo is issued.

The present invention further provides a camera system for an intelligent terminal, the camera system comprising: an acquisition module, an extraction module, a determination module, and a compensation module;

The collecting module collects a preview image of a face including a shooting object;

The extraction module is communicatively coupled to the acquisition module and the determined module to extract a facial color temperature value of the face;

The determining module, and the extracting module and the compensation module, determining whether the color temperature value of the face is lower than a preset color temperature value;

When the color temperature value of the face is lower than a preset color temperature value, the compensation module emits a compensation light.

Preferably, the extraction module further comprises:

An identification unit that performs face recognition on the face to determine an eye position in the face;

The detecting unit detects an average eye color temperature value of the eye position as a facial color temperature value.

Preferably, the compensation module emits the compensation light by a soft light and/or a flash.

Preferably, the photographing system further includes a prompting module, and the prompting module is communicatively connected to the determining module;

The extraction module is further connected to the compensation module, and extracts a compensated color temperature value after the face receives the compensation light;

The determining module determines whether the compensated color temperature value is lower than a preset color temperature value;

When the compensated color temperature value is equal to or higher than the preset color temperature value, the prompting module issues a prompt for reminding the photo.

Compared with the prior art, the technical advantages of the present invention are:

1. No need to use the fill light tool;

2. There is no need to manually adjust the white balance option;

3. Not subject to the restrictions of the location;

4. By extracting the color temperature value of the face of the subject's face, the compensation light of different intensity can be automatically generated according to the color temperature value of the face;

5. It can ensure that the photo is taken under a certain standard brightness, providing users with a more optimized shooting effect and shooting experience.

DRAWINGS

1 is a schematic flowchart of a method for photographing based on an intelligent terminal according to an embodiment of the present invention;

2 is a schematic flowchart of a method for photographing based on an intelligent terminal according to an embodiment of the present invention;

3 is a schematic flowchart of a method for photographing based on an intelligent terminal according to an embodiment of the present invention;

4 is a schematic structural diagram of a camera system based on an intelligent terminal according to an embodiment of the present invention;

FIG. 5 is a schematic structural diagram of a camera system based on an intelligent terminal according to an embodiment of the present invention; FIG.

FIG. 6 is a schematic structural diagram of a camera system based on an intelligent terminal according to an embodiment of the present invention.

Detailed ways

The superiority of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. The following description refers to the same or similar elements in the different figures unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with aspects of the present disclosure as detailed in the appended claims.

The terms used in the present disclosure are for the purpose of describing particular embodiments only, and are not intended to limit the disclosure. The singular forms "a", "the" and "the"

The smart terminal can be implemented in various forms. For example, the terminal described in the present invention may include a mobile terminal such as a mobile phone, a smart phone, a notebook computer, a PDA (Personal Digital Assistant), a PAD (Tablet), a PMP (Portable Multimedia Player), a navigation device, and the like, and such as Fixed terminal for digital TV, desktop computer, etc. In the following, it is assumed that the terminal is a smart terminal. However, those skilled in the art will appreciate that configurations in accordance with embodiments of the present invention can be applied to fixed type terminals in addition to components that are specifically for mobile purposes.

In the following description, the use of a suffix such as "module" for indicating an element is merely an explanation for facilitating the present invention, and does not have a specific meaning per se.

1 is a schematic flowchart of a method for photographing based on an intelligent terminal according to an embodiment of the present invention. As shown in the figure, in the embodiment, the photographing method includes the following steps:

- collecting a preview image of a face containing a subject;

When the user needs to capture an image including a human face through the smart terminal, according to the photographing method provided by the present invention, the camera of the smart terminal (including the front camera and the rear camera) needs to be first turned on, and after opening, a preview image is collected through the camera. The preview image includes at least one or more face images of the subject.

Extracting a facial color temperature value of the face;

After acquiring the facial image including one or more photographic subjects, the smart terminal further detects the face in the preview image through the built-in color temperature sensor for the subsequent smart terminal to accurately adjust the facial color temperature value according to the current shooting state. The color temperature value is the color temperature of the face. Color temperature is a feature of visible light that has important applications in photography, video, and publishing. Color temperature is a sign that indicates the color of light, not the temperature of light. The color temperature value is expressed by K. Different light sources exhibit different colors because the spectral components are different, and white light contains equal amounts of red, green, and blue light, that is, the same amount of red, green, and blue light are mixed in white light. The color temperature of this white light is about 5400K. If a light source contains more red light components, its color temperature is lower than 5400K, for example, the tungsten light color temperature is about 2800K; if a light source contains more blue light components, its color temperature is higher than 5400K, such as sunny days. The color temperature can reach about 20000K. Therefore, the simple understanding is that the more red light, the lower the color temperature value; the more blue the light, the higher the color temperature value.

- determining whether the color temperature value of the face is lower than a preset color temperature value;

After collecting the color temperature value of the face in the current shooting state, the smart terminal compares the color temperature value of the face with a preset color temperature value to determine whether the current color temperature value of the face reaches the preset standard, that is, determines the current shooting state. Is it possible to realistically present the color of the picture? Based on this, the smart terminal will calculate whether the current facial color temperature value is lower than the preset color temperature value by calling a function for judgment.

- A compensated light is emitted when the facial color temperature value is below a predetermined color temperature value.

When the intelligent terminal judges that the color temperature value of the face obtained by the smart terminal is lower than a preset color temperature value, the smart terminal determines that the photo collected under the current light state cannot present the true color, and the light condition is not good, which affects the photo quality. . For example, the current shooting weather is cloudy, rainy, or at night. Among them, the setting of the preset color temperature value can refer to the sensitivity of the color temperature of the human eye according to each region. At this time, in order to avoid taking a photo with poor performance, even if the user issues a photographing instruction, the smart terminal does not execute the photographing instruction. On the contrary, the smart terminal automatically fills in the current subject. The method of filling light includes: automatically turning on the soft light and/or the flash to increase the current shooting brightness, and adjusting and adjusting the color temperature value of the face of the subject. Specifically, the smart terminal can adjust the manner of emitting the compensated light and compensate the intensity of the light according to the difference between the color temperature of the face and the preset color temperature value. When the difference is small, the light compensation can be performed only by the soft light; if the difference is large, the light compensation should be combined with the soft light and the flash, and further Steps increase the light intensity of the flash, so that the compensated face color temperature value reaches the standard of the preset color temperature value.

Referring to FIG. 2, it is a schematic flowchart of a method for photographing based on an intelligent terminal according to an embodiment of the present invention. In a preferred embodiment, the step of extracting the color temperature value of the face of the face further includes:

- performing face recognition on the face to determine an eye position in the face;

The step of extracting the face color temperature value of the subject further includes face recognition of the subject. Among them, the face recognition includes the extraction of the feature information of the specific part of the face and the overall feature information, and the representation methods can be roughly divided into three categories:

The first one is a point to represent facial features, that is, some feature points defined according to our understanding of facial features. For example, in ASM and AAM, facial features are defined by points, which we call boundaries. point. This is divided into three kinds of boundaries: extreme points. Generally, such boundaries have only one definition in a local range, such as the pupil of the eye, the tip of the nose and the nostrils. Boundary point, which can be considered as a boundary point on a local boundary, but considering the position and distance between adjacent boundary points, it is generally extracted uniformly over the entire edge, such as a face contour point. , eyebrow contour points, lip contour points, etc. Interpolation points, which have no obvious texture features locally, are calculated by interpolation of other boundary points, such as the center point of the mouth, the eyebrow, and those that are occluded or invisible.

The second is to define facial features with lines or borders. For example, in a deformable template, the outline of the face is defined as a parabola, the boundary of the eyeball is defined as a circle, and so on.

The third type is to define facial features by region. For example, in some methods of segmentation by color or gray value, the region of the pixel with lip color is segmented as a mouth by statistically counting the lip color; eyes, eyebrows, etc. The difference is made by the difference between the corresponding color and brightness and other areas of the face, and then the area composed of the eligible pixels is taken as the corresponding target area. Based on the above three representation methods, in the photographing method of the embodiment of the present invention, the eye position of the photographed person can be obtained by identifying the points, lines, and faces in the face image collected by the smart terminal.

- Detecting the average eye color temperature value of the eye position as a facial color temperature value.

After the face recognition is used to lock the eye position in the face of the subject, the smart terminal detects and extracts the eye color temperature value for the eye position through its built-in color temperature sensor, and then averages the extracted N times eye color temperature value. Calculate, and finally get the color temperature of the face.

Referring to FIG. 3, it is a schematic flowchart of a method for photographing based on an intelligent terminal according to an embodiment of the present invention. In a preferred embodiment, after the step of emitting a compensation light when the color temperature value of the face is lower than a preset color temperature value, the photographing method further comprises:

Extracting a compensated color temperature value after the face receives the compensated light;

After the smart terminal issues the compensation light according to the detected color temperature value of the face, in order to ensure that the compensated facial color temperature value can reach the preset color temperature value, the photographing method provided by the present invention further includes the compensated color temperature after being compensated. The value is further extracted.

- determining whether the compensated color temperature value is lower than a preset color temperature value;

After the smart terminal extracts the compensated color temperature value after the light compensation, the smart terminal compares the compensated color temperature value with a preset color temperature value to determine whether the compensated color temperature value of the current face reaches the preset standard, that is, the judgment Whether the color of the picture can be realistically displayed under the current shooting state. Based on this, the smart terminal will calculate whether the compensated color temperature value of the current face is lower than the preset color temperature value by calling a function.

- when the compensated color temperature value is equal to or higher than the preset color temperature value, a reminder to remind the photo is issued.

When the compensated color temperature value of the face has reached the preset standard preset color temperature value, the smart terminal can detect the current photographing state and can take a photo showing the real color. At this time, the smart terminal will send a prompt to the screen to remind The user can take a photo by taking a photo instruction to capture a preview image to generate a photo corresponding to the preview image.

Referring to FIG. 4, it is a schematic structural diagram of a camera system based on an intelligent terminal according to an embodiment of the present invention. As shown in the figure, in the embodiment, the photographing system includes: an acquisition module, an extraction module, a determination module, and a compensation module;

- the acquisition module, acquiring a preview image of a face including a photographic subject;

When the user needs to capture an image including a human face through the smart terminal, the camera system provided by the present invention needs to first open the camera (including the front camera and the rear camera) in the acquisition module, and after opening, through the acquisition module The camera captures a preview image that includes at least one or more facial images of the subject.

- the extraction module is communicatively coupled to the acquisition module and the determined module to extract a facial color temperature value of the face;

After the acquisition module collects the facial image including one or more photographic objects, the extraction module further detects the color temperature sensor according to the current shooting state, and the extraction module further detects the preview image through its built-in color temperature sensor. The color temperature value of the face is the color temperature of the face. Color temperature is a feature of visible light that has important applications in photography, video, and publishing. Color temperature is a sign that indicates the color of light, not the temperature of light. The color temperature value is expressed by K. Different light sources exhibit different colors because the spectral components are different, and white light contains equal amounts of red, green, and blue light, that is, the same amount of red, green, and blue light are mixed in white light. The color temperature of this white light is about 5400K. If a light source contains more red light components, its color temperature is lower than 5400K, for example, the tungsten light color temperature is about 2800K; if a light source contains more blue light components, its color temperature is higher than 5400K, such as sunny days. The color temperature can reach about 20000K. Therefore, the simple understanding is that the more red light, the lower the color temperature value; the more blue the light, the higher the color temperature value.

- the determining module, and the extracting module and the compensation module, determining whether the color temperature value of the face is lower than a preset color temperature value;

After the extraction module extracts the color temperature value of the face in the current shooting state, the determining module compares the color temperature value of the face with a preset color temperature value to determine whether the current color temperature value of the face reaches a preset standard, that is, determines the current Whether the color of the picture can be realistically displayed in the shooting state. Based on this, the judging module will calculate whether the current facial color temperature value is lower than the preset color temperature value by calling the function for judgment.

- the compensation module emits a compensation light when the facial color temperature value is lower than a preset color temperature value.

When the judging module judges that the color temperature value of the face obtained by the judging module is lower than a preset color temperature value, the judging module judges that the photo taken under the current light state cannot present the true color, the light condition is not good, and the photo quality is affected. . For example, the current shooting weather is cloudy, rainy, or at night. Among them, the setting of the preset color temperature value can refer to the sensitivity of the color temperature of the human eye according to each region. At this time, in order to avoid taking a photo with poor performance, even if the user issues a photographing instruction, the photographing system will not execute the photographing instruction. On the contrary, the compensation module of the photographing system will automatically fill the current subject. The method of filling light includes: automatically turning on the soft light and/or the flash to increase the current shooting brightness, and adjusting and adjusting the color temperature value of the face of the subject. Specifically, the compensation module can adjust the manner of emitting the compensated light and compensate the intensity of the light according to the difference between the color temperature of the face and the preset color temperature value. When the difference is small, the light compensation can be performed only by the soft light in the compensation module; if the difference is large, the soft light and the flash in the compensation module should be combined to compensate the light, and the light intensity change of the flash is further increased. , so that the compensated face color temperature value reaches the standard of the preset color temperature value.

Referring to FIG. 5, it is a schematic structural diagram of a camera system based on an intelligent terminal according to an embodiment of the present invention. As shown, in a preferred embodiment, the extraction module further includes: an identification unit that performs face recognition on the face to determine an eye position in the face;

The extraction module further includes an identification unit and a detection unit. Wherein, the identification unit is used for face recognition of the photographic subject. Among them, the face recognition includes the extraction of the feature information of the specific part of the face and the overall feature information, and the representation methods can be roughly divided into three categories:

The first one is a point to represent facial features, that is, some feature points defined according to our understanding of facial features. For example, in ASM and AAM, facial features are defined by points, which we call boundaries. point. This is divided into three kinds of boundaries: extreme points. Generally, such boundaries have only one definition in a local range, such as eyes. Pupils, nose tips and nostrils. Boundary point, which can be considered as a boundary point on a local boundary, but considering the position and distance between adjacent boundary points, it is generally extracted uniformly over the entire edge, such as a face contour point. , eyebrow contour points, lip contour points, etc. Interpolation points, which have no obvious texture features locally, are calculated by interpolation of other boundary points, such as the center point of the mouth, the eyebrow, and those that are occluded or invisible.

The third type is to define facial features by region. For example, in some methods of segmentation by color or gray value, the region of the pixel with lip color is segmented as a mouth by statistically counting the lip color; eyes, eyebrows, etc. The difference is made by the difference between the corresponding color and brightness and other areas of the face, and then the area composed of the eligible pixels is taken as the corresponding target area. Based on the above three representation methods, in the photographing system of the embodiment of the present invention, the recognition unit of the extraction module can obtain the eye position of the photographed person by the recognition of the points, lines and faces in the facial image collected by the acquisition module.

a detecting unit that detects an average eye color temperature value of the eye position as a facial color temperature value.

After recognizing the face of the subject by the recognition unit and locking the position of the eye in the face of the subject, the detecting unit can detect and extract the color temperature value of the eye for the eye position through the built-in color temperature sensor thereof, and then extract the obtained eye color value. The N-eye color temperature value is averaged, and finally the face color temperature value is obtained.

FIG. 6 is a schematic structural diagram of a camera system based on an intelligent terminal according to an embodiment of the present invention. As shown in the figure, in a preferred embodiment, after the step of emitting a compensation light when the color temperature value of the face is lower than a preset color temperature value, the photographing system further includes a prompting module, the prompt a module, which is in communication with the determining module;

The extraction module is further connected in communication with the compensation module, and extracts a compensated color temperature value after the face receives the compensation light;

After the compensation module sends the compensation light according to the color temperature value of the face detected by the extraction module, in order to ensure that the compensated color temperature value of the face can reach the preset color temperature value, the extraction module in the photographing system provided by the present invention further The compensation module communication connection is further used for further extracting the compensated compensation color temperature value.

- the determining module, determining whether the compensated color temperature value is lower than a preset color temperature value;

After the extraction module extracts the compensated color temperature value after the light compensation, the determining module further compares the compensated color temperature value with a preset color temperature value to determine whether the compensated color temperature value of the current face reaches a preset standard, that is, Determine whether the color of the picture can be realistically displayed under the current shooting state. Based on this, the judgment module will calculate whether the compensated color temperature value of the current face is lower than the preset color temperature value by calling the function for judgment.

- when the compensated color temperature value is equal to or higher than the preset color temperature value, the prompting module issues a reminder to remind the photo.

When the compensated color temperature value of the face has reached the preset standard preset color temperature value, the judging module judges that the photograph of the real color can be captured under the current photographing state, and at this time, the prompting module of the photographing system will issue a photo to the screen. The prompt reminds the user that the photo can be taken by taking a photo instruction, and the preview image is captured to generate a photo corresponding to the preview image.

The invention adopts a smart terminal-based photographing method and a photographing system provided by the invention, and the user does not need to manually adjust the white balance option when taking photos, without using the fill light tool, and is not restricted by the shooting location, and the face of the photographed object is adopted. Face color temperature value extraction, can automatically emit different intensity compensation light according to the color temperature value of the face to ensure that the photo is shot under a certain standard brightness, providing users with a more optimized shooting effect and shooting experience.

It should be noted that the embodiments of the present invention are preferred embodiments, and are not intended to limit the scope of the present invention. Any one skilled in the art may use the above-disclosed technical contents to change or modify the equivalent embodiments. Any modification or equivalent changes and modifications of the above embodiments in accordance with the technical spirit of the present invention are still within the scope of the technical solutions of the present invention.

Claims

A photographing method based on a smart terminal, characterized in that

The photographing method includes the following steps:

Collecting a preview image of a face containing a subject;

Extracting a facial color temperature value of the face;

Determining whether the color temperature value of the face is lower than a preset color temperature value;

When the color temperature value of the face is lower than a preset color temperature value, a compensation light is emitted.
The photographing method according to claim 1, wherein

And the step of extracting the color temperature value of the face of the face, further comprising:

Performing face recognition on the face to determine an eye position in the face;

The average eye color temperature value of the eye position is detected as a facial color temperature value.
The photographing method according to claim 1, wherein

The compensation light is emitted by a soft light and/or a flash of the smart terminal.
The photographing method according to claim 3, wherein

The intensity of the compensated light is adjusted by the flash according to a difference between the facial color temperature value and the preset color temperature value.
The photographing method according to any one of claims 1 to 4, wherein

After the step of emitting a compensation light when the color temperature value of the face is lower than a preset color temperature value, the photographing method further includes:

Extracting a compensated color temperature value after the face receives the compensation light;

Determining whether the compensated color temperature value is lower than a preset color temperature value;

When the compensated color temperature value is equal to or higher than the preset color temperature value, a reminder to remind the photo is issued.
A camera system based on a smart terminal, characterized in that

The photographing system comprises: an acquisition module, an extraction module, a judgment module and a compensation module;

The collecting module collects a preview image of a face including a shooting object;

The extraction module is communicatively coupled to the acquisition module and the determined module to extract a facial color temperature value of the face;

The determining module, and the extracting module and the compensation module, determining whether the color temperature value of the face is lower than a preset color temperature value;

When the color temperature value of the face is lower than a preset color temperature value, the compensation module emits a compensation light.
A photographing system according to claim 6 wherein:

The extraction module further includes:

An identification unit that performs face recognition on the face to determine an eye position in the face;

The detecting unit detects an average eye color temperature value of the eye position as a facial color temperature value.
A photographing system according to claim 6 wherein:

The compensation module emits the compensation light by a soft light and/or a flash.
A photographing system according to claim 8 wherein:

The intensity of the compensated light is adjusted by the flash according to a difference between the facial color temperature value and the preset color temperature value.
A photographing system according to any one of claims 6-9, wherein

The photographing system further includes a prompting module, and the prompting module is communicatively connected to the determining module;

The extraction module is further connected to the compensation module, and extracts a compensated color temperature value after the face receives the compensation light;

The determining module determines whether the compensated color temperature value is lower than a preset color temperature value;

When the compensated color temperature value is equal to or higher than the preset color temperature value, the prompting module issues a prompt for reminding the photo.