WO2019213820A1

WO2019213820A1 - Photographing control method and electronic device

Info

Publication number: WO2019213820A1
Application number: PCT/CN2018/085900
Authority: WO
Inventors: 王星泽
Original assignee: 合刃科技（武汉）有限公司
Priority date: 2018-05-07
Filing date: 2018-05-07
Publication date: 2019-11-14
Also published as: CN111279683A

Abstract

Provided is a photographing control method, comprising: when a user performs a manually controlled photographing operation, a model setting a currently photographed image as a positive sample satisfying a photographing condition, and adjusting a parameter of the model itself according to the positive sample; the model sampling, according to a preset rule, an image frame not obtained by means of a manually controlled photographing operation so as to obtain a negative sample, and adjusting the parameter of the model according to the negative sample; and if it is determined that the training has satisfied a completion condition, ending the training to obtain a trained model for subsequent automatic photographing control. Also provided is an electronic device implementing the photographing control method. In the photographing control method and the electronic device, a model is trained, and the trained model is used to subsequently perform automatic photographing control, such that an image of a scene of interest containing content corresponding to the current preview image can be timely captured.

Description

Shooting control method and electronic device

Technical field

The present application relates to the field of electronic devices, and in particular, to a photographing control method for an electronic device and the electronic device.

Background technique

Now, with the improvement of people's living standards, taking pictures has become a common function that is not indispensable in life. Nowadays, whether it is a camera or an electronic device such as a mobile phone or a tablet computer with a camera function, pixels are getting higher and higher, and the quality of photographs is getting better and better. However, current electronic devices such as cameras and mobile phones often require the user to initiate a photograph by pressing a shutter button or a photographing icon when performing photographing control. Since the user's operation often has a certain lag, it is often impossible to capture the wonderful time in time. In an instant, it is often impossible to take a satisfactory photo, or because of the lag, it has taken an unsatisfactory photo. For example, the subject may be in good condition when framing, but the moment when the photo is taken may appear that the eyes are not open, the smile is relatively stiff, and the final photographs are often unsatisfactory. For example, when taking pictures of a baby, the cute expression of the baby is often fleeting, and it is difficult to take a satisfactory photo in time by the user operating the shutter button or shooting an icon.

Summary of the invention

The present application provides a shooting control method and an electronic device, which can automatically capture shooting control through the model by training the model, and can capture a wonderful moment in time.

In one aspect, a photographing control method is provided, the photographing control method includes: when a user performs manual control photographing, using a model to take a currently photographed image as a positive sample satisfying a photographing condition, and adjusting the model itself according to the positive sample of the current time. The parameter is obtained by sampling the frame frame that is not manually controlled by the model as a reverse sample, and adjusting the parameters of the model according to the reverse sample; and ending the training to obtain the trained after determining that the training completion condition is reached; The model is used for subsequent automatic shooting control.

In another aspect, an electronic device is provided, the electronic device including a memory and a processor. The memory is for storing program instructions. The processor is configured to execute the shooting control method by calling the program instruction, and the shooting control method includes: when the user performs manual control shooting, using the model to take the currently captured image as a positive sample satisfying the shooting condition, and according to the present The positive sample of the second adjusts the parameters of the model itself; the frame frame that is not manually controlled is sampled by the model as a reverse sample, and the parameters of the model itself are adjusted according to the reverse sample; and when it is determined that the training completion condition is reached The training is ended and the trained model is obtained for subsequent automatic shooting control.

In still another aspect, a computer readable storage medium is provided, the computer readable storage medium storing program instructions, the program instructions being executed by a computer to execute a shooting control method, the shooting control method comprising: at a user When performing manual control shooting, the currently taken picture is taken as a positive sample satisfying the shooting condition by the model, and the model's own parameters are adjusted according to the current front sample; the picture frame that is not manually controlled is sampled by the model with a preset rule as The reverse sample, and adjusting the parameters of the model itself according to the reverse sample; and when it is determined that the training completion condition is reached, the training is ended to obtain the trained model for subsequent automatic shooting control.

The shooting control method and the electronic device of the present application can perform the automatic shooting control through the trained model, and can capture the exciting moment including the corresponding content of the current shooting preview screen in time.

DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings to be used in the embodiments will be briefly described below. Obviously, the drawings in the following description are only some embodiments of the present application, Those skilled in the art can also obtain other obvious modifications according to these drawings without any creative work.

FIG. 1 is a flowchart of a model training process in a photographing control method in a first embodiment of the present application.

2 is a flowchart of a model training process in a photographing control method in a second embodiment of the present application

FIG. 3 is a flowchart of a model training process in a photographing control method in a third embodiment of the present application.

4 is a flowchart of a model training process in a photographing control method in a fourth embodiment of the present application.

FIG. 5 is a flowchart of a model training process in a photographing control method in a fifth embodiment of the present application.

FIG. 6 is a flowchart of a model training process in a photographing control method in a sixth embodiment of the present application.

FIG. 7 is a flowchart of a model training process in a photographing control method in a seventh embodiment of the present application.

FIG. 8 is a flowchart of a model training process in the photographing control method in the eighth embodiment of the present application.

FIG. 9 is a flowchart of a model training process in the photographing control method in the ninth embodiment of the present application.

FIG. 10 is a flowchart of a model training process in the photographing control method in the tenth embodiment of the present application.

Figure 11 is a flow chart showing a model training process in the photographing control method in the eleventh embodiment of the present application.

FIG. 12 is a flowchart of a model training process in the photographing control method in the twelfth embodiment of the present application.

Fig. 13 is a flow chart showing automatic shooting control using a model in the photographing control method in the thirteenth embodiment of the present application.

FIG. 14 is a block diagram showing a schematic partial structure of an electronic device according to an embodiment of the present application.

detailed description

The technical solutions in the embodiments of the present application are clearly and completely described in the following with reference to the drawings in the embodiments of the present application. It is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present application without departing from the inventive scope are the scope of the present application.

The photographing control method of the present application can be applied to an electronic device, the electronic device at least including a camera, the electronic device can acquire a photographing preview screen through the camera and display the photographing preview screen, and the electronic device can pass The camera performs operations such as photographing, continuous shooting, and video shooting. The camera includes a front camera and a rear camera, and the photographing, continuous shooting, video shooting and the like may be performed by a rear camera or a self-timer by a front camera.

Please refer to FIG. 1 , which is a flowchart of a model training process in a shooting control method in a first embodiment of the present application. As shown in FIG. 1, in the first embodiment, the shooting control method may include the following steps:

S11: When the user performs manual control shooting, the currently taken picture is taken as a positive sample satisfying the shooting condition by the model, and the parameters of the model itself are adjusted according to the current positive sample.

In some embodiments, the model saves the frontal sample and establishes or updates the positive sample and the correspondence that satisfies the shooting conditions to adjust the parameters of the model itself. Among them, the photographing condition can be marked as a label of the front sample.

Optionally, in one implementation, the user manually controls the shooting to be done by pressing a shutter button or a photo icon.

Optionally, in another implementation, the user manually controls the shooting to be performed by performing a specific operation on a physical button of the electronic device. For example, the electronic device includes a power button, and manual control shooting is achieved by double-clicking the power button.

S13: sampling, by using a preset rule, a frame frame that is not manually controlled to be taken as a reverse sample, and adjusting a parameter of the model according to the reverse sample.

In some embodiments, the model may save the back samples of the samples, and may also establish a correspondence between the back samples and the photographing conditions to adjust the parameters of the model itself. The back surface sample is a screen that does not satisfy the shooting condition; the labeling condition may be used as a label of the back surface sample for marking.

S15: When it is determined that the training completion condition is reached, the training is ended to obtain the trained model for subsequent automatic shooting control.

Optionally, in an implementation manner, the step S11 further includes the step of: entering the model training mode in response to the user input entering the model training. The determining that the training completion condition is reached includes determining that the training completion condition is reached in response to the user inputting the operation of exiting the model training mode.

Optionally, the operation of entering the model training includes a selection operation of a menu option, or a specific operation on a physical button, or a specific touch gesture input on a touch screen of the electronic device. Optionally, the operation of entering the model training in response to the user input controls the entering the model training mode, including: responding to the user's selection operation of the menu option, or performing a specific operation on the physical button, or inputting on the touch screen of the electronic device. The specific touch gesture is controlled to enter the model training mode.

Optionally, in another implementation manner, the determining that the training completion condition is reached includes: determining that the training completion condition is reached when it is determined that the number of times the user manually controls the shooting reaches the preset number of times N1. The preset number of times N1 may be the number of times the system default model training needs to be performed, or may be a user-defined value.

Optionally, in another implementation manner, the determining reaches a training completion condition, including: when the user performs manual control shooting, inputting parameters of the manually controlled captured image to the model to obtain a predicted value; determining whether the predicted value is The shooting condition is satisfied; if the predicted value satisfies the shooting condition, it is determined that the training completion condition is reached.

The above model may be a model such as a neural network model or an image processing algorithm model.

In the present application, the trained model is obtained by training the model in advance, and when the subsequent user turns on the camera for shooting, the shooting can be automatically controlled according to the trained model, and the satisfactory picture desired by the user can be captured in time.

Please refer to FIG. 2 , which is a flowchart of a model training process of the photographing control method in the second embodiment of the present application. As shown in FIG. 2, in the second embodiment, the shooting control method may include the following steps:

S21: Control the entering the model training mode in response to the user inputting the operation of entering the model training.

Optionally, the operation of entering the model training includes a selection operation of a menu option, or a click operation on a specific icon, or a specific operation on a physical button, or a specific touch gesture input on a touch screen of the electronic device. For example, the operation of entering the model training includes: a user selects a function option to enter the model training, or a click operation on a model training mode icon displayed on the electronic device, or a double-click operation on the volume key, or in an electronic An upward sliding touch gesture or the like input on the touch screen of the device.

Optionally, the operation of entering the model training in response to the user input controls the entering the model training mode, including: responding to the user's selection operation of the menu option, or performing a specific operation on the physical button, or inputting on the touch screen of the electronic device. The specific touch gesture is controlled to enter the model training mode.

S23: When the user performs manual control shooting, the currently taken picture is taken as a positive sample satisfying the shooting condition by the model, and the parameters of the model itself are adjusted according to the current front sample.

S25: The screen frame that is not manually controlled is sampled by the model as a reverse sample by using a preset rule, and the parameters of the model itself are further adjusted according to the reverse sample.

S27: Respond to the operation of the exit model training mode input by the user, determine that the training completion condition is reached, and end the training to obtain the trained model for subsequent automatic shooting control.

The operation of the exit model training also includes a selection operation of a menu option, a click operation on a specific icon, or a specific operation on a physical button, or a specific touch gesture input on a touch screen of an electronic device. For example, the operation of the exit model training includes: a user unchecking a function option entering the model training, or a click operation on an exit model training mode icon displayed on the electronic device, or a long press operation on the volume key, Or a downward sliding touch gesture or the like input on the touch screen of the electronic device.

Wherein, steps S23 and S25 in FIG. 2 correspond to steps S11 and S13 in FIG. 1, and related descriptions may be referred to each other.

Please refer to FIG. 3 , which is a flowchart of a model training process of the photographing control method in the third embodiment of the present application. As shown in FIG. 3, in the third embodiment, the shooting control method may include the following steps:

S31: When the user performs manual control shooting, the currently taken picture is taken as a positive sample satisfying the shooting condition by the model, and the parameters of the model itself are adjusted according to the current positive sample.

S33: Determine whether the number of times the user manually controls the shooting reaches a preset number N. If yes, go to step S37, if no, go to step S35.

S35: Sample the frame that has not been manually controlled by sampling according to a preset rule as a reverse sample, and further adjust the parameters of the model according to the reverse sample.

After the execution of step S35, the process returns to step S31.

S37: When it is determined that the training completion condition is reached, the training is ended to obtain the trained model for subsequent automatic shooting control.

Steps S31 and S35 respectively correspond to steps S11 and S13 in the first embodiment shown in FIG. 1. For a more specific introduction, reference may be made to the related descriptions of steps S11 and S13 in FIG.

Please refer to FIG. 4 , which is a flowchart of a model training process of a shooting control method in a fourth embodiment of the present application. As shown in FIG. 4, in the fourth embodiment, the shooting control method may include the following steps:

S41: When the user performs manual control shooting, the parameters of the screen obtained by manually controlling the shooting are input to the model to obtain a predicted value.

S43: Determine whether the predicted value satisfies the shooting condition. If no, step S45 is performed, and if so, step S47 is performed.

S45: If the shooting condition is not satisfied, the screen obtained by the manual control shooting is used as a parameter of the front sample adjustment model itself. After the step S45 is performed, the process returns to step S41.

S47: If the shooting condition is met, it is determined that the training completion condition is reached, and the training is ended to obtain the trained model for subsequent automatic shooting control.

In some embodiments, the step S47 further includes: if the predicted value satisfies the shooting condition, adding one counted number of times the predicted value satisfies the shooting condition to obtain an updated counting number; determining the current counting number (ie, the updated number of times) Whether the current number of counts exceeds the preset number of times; if the current number of counts exceeds the preset number of times, it is determined that the training completion condition is reached, and the training is ended to obtain the trained model for subsequent automatic shooting control. Obviously, if the current number of counts does not exceed the preset number of times, it is determined that the training completion condition is not reached, and the process returns to step S41.

Thus, by determining that the training completion condition is reached after determining that the predicted value satisfies the shooting condition for more than a preset number of times, it is ensured that the accuracy of the training completion is determined.

Please refer to FIG. 5 , which is a flowchart of a model training process of a shooting control method in a fifth embodiment of the present application. As shown in FIG. 5, in the fifth embodiment, the model training process may include the following steps:

S51: After the automatic shooting function is turned on, the framing preview is performed in response to the opening operation of the camera to obtain a shooting preview screen.

Optionally, in some embodiments, the turning on the automatic shooting function may be completed in response to a user setting operation in a menu option of the camera.

Optionally, in other embodiments, the turning on the automatic shooting function may also be performed in response to a specific touch gesture on the touch screen of the electronic device, for example, in response to a double tapping on the touch screen of the electronic device through the knuckles After the operation is completed.

In some embodiments, the operation of turning on the camera may be a click operation on a photo application icon, a specific operation on a physical button of the electronic device, an operation of a preset touch gesture input in any display interface of the electronic device, or the like. .

S53: When it is determined according to the current model that the shooting condition is satisfied, the control performs a shooting operation.

In some embodiments, the step S53 specifically includes: determining, according to the training result in the current model, whether the current shooting framing picture satisfies the shooting condition, and controlling to perform the shooting operation when determining that the shooting condition is satisfied.

The photographing operation includes a photographing operation, a continuous shooting operation, a video shooting operation, and the like.

For a more specific description of step S53, reference may be made to the embodiment shown in the following FIG.

S55: Acquire user satisfaction feedback information about this automatic shooting.

Optionally, in an implementation manner, after the automatic photographing is completed, the user may be prompted to perform satisfaction evaluation on the automatic photographing by generating prompt information, for example, generating a prompt box including “satisfactory” and “unsatisfactory” options. For the user to select, and according to the user's choice, the satisfaction feedback information of the automatic photographing is obtained.

Optionally, in another embodiment, the user's satisfaction with the automatic shooting is obtained by detecting the user's operation on the photo or video obtained by the automatic shooting. For example, if it is detected that the user deletes the photo or video obtained by the automatic shooting, it is determined that the user is not satisfied with the automatic shooting, and the satisfaction feedback information that is unsatisfactory is obtained. For example, if it is detected that the user has set a photo or video obtained by the automatic shooting to a favorite or favorite type setting operation or a sharing operation, it is determined that the user is satisfied with the automatic shooting, and obtains I got feedback on satisfaction with satisfaction.

S57: Output the satisfaction feedback information of the user to the current automatic shooting to the currently used model, so that the currently used model uses the satisfaction feedback information to perform optimization training.

Therefore, in the present application, by collecting user satisfaction feedback information for automatic shooting, the training of the model can be optimized, and the model is continuously optimized, so that the automatic shooting in subsequent use can be more accurate.

The currently used model may be a model that has been confirmed by the training, for example, a model that is confirmed to be completed by the method steps shown in FIG. 1-4, or a model that has not been trained yet. When it is confirmed that the training is completed, the model can be further optimized. When the model is not yet trained, the training can be better achieved.

Correspondingly, the steps S51-S57 in FIG. 5 can be performed after step S15 in FIG. 1, and can also be performed before step S15 in FIG. 1, and can even be performed before step S11 in FIG. When executed before step S11, the currently used model may be an untrained initial model.

In some embodiments, when the preset model is an untrained model, the untrained model automatically acquires a picture every time the user performs shooting, and performs training as a positive sample, or further acquires a shooting time. The shooting parameters are trained together as a positive sample, and the preset model is gradually optimized until the number of training reaches a preset number of times or the satisfaction feedback information of subsequent user feedback is a satisfactory ratio exceeding a preset ratio, and the training is determined to be completed. In this way, since the user himself trains the model without using another person's model, personalization can be better achieved.

Please refer to FIG. 6 , which is a flowchart of a model training process in a shooting control method in a sixth embodiment of the present application. As shown in FIG. 6, in the sixth embodiment, the model training process may include the following steps:

S61: When the user performs manual control shooting, the currently captured picture is taken as a positive sample corresponding to the shooting condition and the shooting parameter by the model, and the parameters of the model itself are adjusted according to the current front sample.

In some embodiments, the model saves the frontal sample and establishes or updates the frontal sample and the correspondence that satisfies the shooting conditions and the shooting parameters to adjust the parameters of the model itself. Wherein, the shooting conditions and the shooting parameters can be simultaneously marked as the labels of the front samples.

The shooting parameters may include parameters such as aperture size, shutter time, and the like.

S63: sample the frame frame that is not manually controlled by the model by using a preset rule as a reverse sample, and adjust the parameters of the model according to the reverse sample.

S65: When it is determined that the training completion condition is reached, the training is ended, and the trained model is obtained.

The step S63 and the step S65 respectively correspond to the step S13 and the step S15 in FIG. 1 . For specific introduction, reference may be made to the description of step S13 and step S15 in FIG. 1 .

Therefore, in still another embodiment, by the training of the model, not only the correspondence between the picture as the front sample and the shooting condition is established, but the correspondence between the picture as the front sample and the shooting parameter is established, and when the automatic shooting is subsequently started. It is possible to automatically determine whether the shooting conditions are satisfied and the setting of the shooting parameters can be automatically performed.

Please refer to FIG. 7 , which is a flowchart of a model training process in a shooting control method in a seventh embodiment of the present application. As shown in FIG. 7, in the seventh embodiment, the shooting control method may include the following steps:

S71: When the user performs manual control shooting, the currently taken picture is taken as a positive sample satisfying the shooting condition by the model, and the parameters of the model itself are adjusted according to the current positive sample.

S73: Sampling the framed picture frame after a positive sample for a period of time as a reverse sample, and adjusting the parameters of the model itself according to the reverse sample.

The time period after the front sample is manually controlled by the user for 2 seconds, 3 seconds, and the like after the front sample taken.

S75: When it is determined that the training completion condition is reached, the training is ended to obtain the trained model for subsequent automatic shooting control.

The steps S71 and S75 are the same as the steps S11 and S15 in FIG. 1 . For specific introduction, reference may be made to the related description in FIG. 1 . Step S73 is a more specific manner of step S13 in Fig. 1, and the relevant points can be referred to each other.

Please refer to FIG. 8 , which is a flowchart of a model training process in a shooting control method in an eighth embodiment of the present application. As shown in FIG. 8, in the eighth embodiment, the shooting control method may include the following steps:

S81: When the user performs manual control shooting, the currently taken picture is taken as a positive sample satisfying the shooting condition by the model, and the parameters of the model itself are adjusted according to the current positive sample.

S83: Sampling the framed picture frame for a period of time before the front side sample as a reverse side sample, and adjusting the parameters of the model itself according to the reverse side sample.

The time before the front sample may be manually controlled by the user for 2 seconds, 3 seconds, and the like before the front sample taken.

Wherein, when sampling the frame frame that is framing for a period of time before the front side sample, the framing picture/shooting preview picture is automatically intercepted in advance and a certain number of pending samples are stored, and after the manual control shooting, the non-manual control is performed. The pending sample taken is determined as the reverse sample.

S85: When it is determined that the training completion condition is reached, the training is ended to obtain the trained model for subsequent automatic shooting control.

The steps S81 and S85 are the same as the steps S11 and S15 in FIG. 1 . For specific introduction, reference may be made to the related description in FIG. 1 . Step S83 is a more specific manner of step S13 in Fig. 1, and the relevant points can be referred to each other.

Referring to Fig. 9, there is shown a flow chart of a model training process in the photographing control method in the ninth embodiment of the present application. As shown in FIG. 9, in the ninth embodiment, the photographing control method may include the following steps:

S91: When the user performs manual control shooting, the currently taken picture is taken as a positive sample satisfying the shooting condition by the model, and the parameters of the model itself are adjusted according to the positive sample of the current time.

S93: Obtain a picture frame that is not manually controlled by random sampling as a reverse sample, and adjust a parameter of the model itself according to the reverse sample.

Thus, the reverse sample can be obtained by randomly sampling the picture frames that are not manually controlled.

S95: When it is determined that the training completion condition is reached, the training is ended to obtain the trained model for subsequent automatic shooting control.

The steps S91 and S95 are the same as the steps S11 and S15 in FIG. 1 . For a specific description, reference may be made to the related description in FIG. 1 . Step S93 is a more specific manner of step S13 in FIG. 1, and the relevant points can be referred to each other.

Please refer to FIG. 10 , which is a flowchart of a model training process in a shooting control method in a tenth embodiment of the present application. As shown in FIG. 10, in the tenth embodiment, the shooting control method may include the following steps:

S101: When the user performs manual control shooting, the currently captured picture is taken as a positive sample satisfying the shooting condition by the model, and the parameters of the model itself are adjusted according to the positive sample of the current time.

S103: When it is determined by the detection result of the sensor that sampling is required, sampling of the picture frame is performed, and the sampled picture frame is used as a reverse sample, and the parameters of the model itself are adjusted according to the reverse sample.

The sensor may be a photosensitive or acoustic sensor for collecting ambient light or sound to determine sampling, and the sampled picture frame is used as a reverse sample. For example, when the sensor is an acoustic sensor, when a sound such as “please be prepared” is collected, it is considered that the shooting ready state has not yet entered, and the frame frame sampled at this time is the reverse sample.

S105: When it is determined that the training completion condition is reached, the training is ended to obtain the trained model for subsequent automatic shooting control.

The steps S101 and S105 are the same as the steps S11 and S15 in FIG. 1 . For a specific description, reference may be made to the related description in FIG. 1 . Step S103 is a more specific manner of step S13 in FIG. 1, and the relevant points can be referred to each other.

Please refer to FIG. 11 , which is a flowchart of a model training process in the photographing control method in the eleventh embodiment of the present application. As shown in FIG. 11, in the eleventh embodiment, the shooting control method may include the following steps:

S111: When the user performs manual control shooting, the currently taken picture is taken as a positive sample satisfying the shooting condition by the model, and the parameters of the model itself are adjusted according to the current positive sample.

S113: Collect and save the picture frame that has not been manually controlled, perform composition analysis on the saved picture frame, determine a picture frame as a reverse sample, and adjust the parameters of the model according to the reverse sample.

The composition analysis may determine that the picture frame is a reverse sample when the expression in the picture frame is closed, unnatural, or the like.

S115: When it is determined that the training completion condition is reached, the training is ended to obtain the trained model for subsequent automatic shooting control.

The steps S111 and S115 are the same as the steps S11 and S15 in FIG. 1 . For a specific description, reference may be made to the related description in FIG. 1 . Step S113 is a more specific manner of step S13 in Fig. 1, and the correlation can be referred to each other.

Please refer to FIG. 12 , which is a flowchart of a model training process in a photographing control method in a twelfth embodiment of the present application. As shown in FIG. 12, in the twelfth embodiment, the photographing control method may include the following steps:

S121: When the user performs manual control shooting, the currently taken picture is taken as a positive sample satisfying the shooting condition by the model, and the parameters of the model itself are adjusted according to the current positive sample.

S123: Sampling a frame frame obtained by framing between two adjacent manual control shots at a preset time interval as a reverse sample, and adjusting parameters of the model itself according to the reverse sample.

The preset time interval may be 1 second or 2 seconds.

Optionally, in one implementation, two adjacent manual control shots may be two adjacent manual control shots during the framing shot performed by the same camera opening.

Optionally, in another implementation manner, the two adjacent manual control shots may also be two different manual control shots during the framing shooting. For example, after the user turns on the camera and completes the first manual control shooting, the camera is turned off, and the next time the camera is turned on to complete the second manual control shooting, the framing screen between the first manual control shooting and the second manual control shooting is The currently used model is saved at a preset time interval as a negative sample.

S125: When it is determined that the training completion condition is reached, the training is ended to obtain the trained model for subsequent automatic shooting control.

The steps S121 and S125 are the same as the steps S11 and S15 in FIG. 1 . For a specific description, reference may be made to the related description in FIG. 1 . Step S123 is a more specific manner of step S13 in FIG. 1, and the relevant points can be mutually cross-referenced.

Please refer to FIG. 13 , which is a flowchart of an automatic shooting control process in the shooting control method in the thirteenth embodiment of the present application.

S131: Acquire a shooting preview screen.

In some embodiments, the operation of acquiring the shooting preview screen is performed by the camera in response to the operation of turning on the camera, that is, the shooting preview screen is acquired by the camera.

In some embodiments, the operation of turning on the camera is a click operation on the photographing application icon, that is, when the camera is turned on in response to a click operation on the photographing application icon, the shooting preview screen is acquired by the camera.

Alternatively, in other embodiments, the operation of turning on the camera is a specific operation of a physical button of the electronic device. For example, the electronic device includes a volume up button and a volume down button, and the operation of turning on the camera is to increase the volume and volume. Simultaneous pressing of small keys. Further, the operation of turning on the photographing application is an operation of pressing the volume up key and the volume down key in a preset time (for example, 2 seconds).

In other embodiments, the operation of turning on the camera may also be an operation of a preset touch gesture input in any display interface of the electronic device. For example, on the main interface of the electronic device, the user may input a circular touch track. Turn on the camera with a touch gesture.

In other embodiments, the operation of turning on the camera may also be an operation of a preset touch gesture input on the touch screen when the electronic device is in a black screen state.

In some embodiments, when the electronic device is a camera, the operation of turning on the camera is an operation that presses the shutter button/power button of the camera to trigger the camera to be in an activated state.

Optionally, in the present application, acquiring a shooting preview screen is to obtain a shooting preview screen in real time through a camera.

S133: analyzing the shooting preview image by using a preset model to obtain an analysis result.

The preset model may be a trained model or an untrained model.

Optionally, in some embodiments, the preset model is a trained neural network model; and the preset preview model is used to analyze the captured preview image to obtain an analysis result, and further includes: analyzing the captured preview image by using a neural network model, The result of this analysis of satisfaction is obtained. Among them, the satisfaction degree is that the neural network model outputs the satisfaction result by processing the pre-trained model by taking all the pixels of the preview image as input.

Optionally, in other embodiments, the preset model is a trained image processing algorithm model, and the preset preview model is used to analyze the captured preview image to obtain an analysis result, and further includes: using a trained image processing algorithm model The shooting preview screen is compared with the reference picture, and the analysis result of the similarity between the preview picture and the reference picture is analyzed. The reference picture may be a standard picture preset by the user with a specific expression such as smile, laughter, sadness, anger, yawning, and the like.

Further, the trained image processing algorithm model includes the trained target object feature model, and the trained image processing algorithm model is used to compare the captured preview image with the reference image, and the similarity between the captured preview image and the reference image is analyzed, including The face recognition technology is used to analyze the target object in the shooting preview image to generate a corresponding target object feature vector; and the captured preview image and the reference are calculated according to the trained target object feature model and the target object feature vector corresponding to the captured preview image. The similarity of the picture.

In some embodiments, the calculating the similarity between the captured preview image and the reference image according to the trained target object feature model and the target object feature vector corresponding to the captured preview image comprises: capturing the target object feature vector corresponding to the preview image As the input information of the trained expression feature model, the similarity between the shooting preview picture and the reference picture is calculated by the target object feature vector.

In another embodiment, comparing the shooting preview image with the reference image by using the trained image processing algorithm model, and analyzing the analysis result of the similarity between the preview image and the reference image, comprising: acquiring pixel information of the shooting preview image; The pixel information of the shooting preview screen is compared with the pixel information of the reference picture, and the similarity between the shooting preview picture and the reference picture is analyzed. That is, in other embodiments, the similarity between the captured preview picture and the reference picture is obtained by comparing the pixel information such as the pixel grayscale value of the two images.

In another embodiment, when the preset model is a trained image processing algorithm model, the analyzing the captured preview image by using the preset model to obtain the analysis result includes: using the face recognition technology to capture the preview image. The facial expression is analyzed to generate a corresponding expression feature vector; the expression feature vector is used as input information of the image processing algorithm model, and an analysis result including identification information indicating whether the shooting condition is currently satisfied is obtained. For example, the identification information may include an identifier of 1, 0, etc., which identifies whether the analysis result of the shooting condition is currently satisfied. More specifically, when the identification information is the identifier 1, it indicates that the shooting condition is satisfied, and when the identification information is the identifier 0, it indicates that the shooting condition is not satisfied.

In some embodiments, the trained target object feature model is completed by training in a plurality of photos having different expression faces in the initial training set; and using the image recognition technology to provide a plurality of pieces in the initial training set The target object in the photo performs emoticon analysis to generate a corresponding target object feature vector Xi. For example, when the target object feature vector is an expression feature vector, X1 represents the size of the eye opening, X2 represents the degree of the mouth angle rising, and X3 represents the mouth opening. The size of the opening; the training sample set is established based on the generated target object feature vector and the similarity label between the corresponding photo and the reference picture; and the sample set is used for training learning to obtain the trained target object feature model.

In some embodiments, in the present application, the analysis result is that the analysis result is obtained according to the real-time acquired shooting preview screen. In some other embodiments, the step of analyzing the shot preview image by using the preset model to obtain an analysis result is that the currently acquired shot preview screen is analyzed every time by a preset time (for example, 0.2 seconds) to obtain a current analysis result. .

S135: Determine whether the shooting condition is currently satisfied according to the analysis result. If yes, step S137 is performed, otherwise, it returns to step S133 or the process ends.

In some embodiments, when the analysis result is obtained according to the neural network model, determining whether the shooting condition is currently satisfied according to the analysis result includes: determining that the shooting condition is currently satisfied when determining that the satisfaction exceeds the satisfaction preset threshold. The satisfaction preset threshold may be 80%, 90%, and the like.

In some embodiments, when the analysis result is obtained according to the image processing algorithm model, determining whether the shooting condition is currently satisfied according to the analysis result includes: determining that the shooting condition is currently satisfied when determining that the similarity exceeds the similarity preset threshold. Alternatively, determining whether the shooting condition is currently satisfied according to the analysis result may further include: determining that the shooting condition is currently satisfied when the analysis result includes identifying the identification information that currently meets the shooting condition.

In the present application, the target object may be a hand, a face, a specific scene, etc.; the target object feature model includes a gesture feature model, an expression feature model, and a scene feature model, etc., and the analyzed target object feature vector may include a gesture. Feature vectors, expression feature vectors, and scene feature vectors.

S137: Control performs a photographing operation when it is determined that the photographing condition is satisfied.

In some implementations, the photographing operation is a photographing operation, and controlling to perform the photographing operation includes: controlling to perform a photographing operation to obtain a photograph corresponding to the current photograph preview screen.

In other implementations, the shooting operation is a continuous shooting operation, and controlling to perform a shooting operation includes: controlling to perform a continuous shooting operation, and obtaining a plurality of photos including a photo corresponding to the current photo preview screen. Optionally, after performing the continuous shooting operation, further steps may be further included: analyzing a plurality of photos obtained by the continuous shooting operation to determine the best photo; and retaining the best photo, and performing the continuous shooting operation Get other photos to delete.

In other implementations, the photographing operation is a video photographing operation, and controlling to perform a photographing operation includes: controlling to perform a video photographing operation to obtain a video file that uses the current photograph preview screen as a starting video frame frame. Optionally, after the video recording operation is performed to obtain the video file, the method may further include: after the video file is captured, the plurality of video frame frames in the captured video file may be compared to determine an optimal screen. Frame; and intercept the best picture frame to save as a photo.

Therefore, in the present application, by analyzing the photographing preview screen to determine whether the photographing condition is currently satisfied, whether or not the photograph desired by the user is photographed can be determined according to the content in the photograph preview screen, so that the current exciting moment can be captured in time.

In some embodiments, step S15 includes determining a shooting parameter including a shutter time and an aperture size based on the analysis result, in addition to determining whether the shooting condition is currently satisfied according to the analysis result.

Controlling the execution of the photographing operation when it is determined that the photographing condition is satisfied further includes: performing a photographing operation according to the determined photographing parameter when it is determined that the photographing condition is satisfied.

Please refer to FIG. 14 , which is a block diagram showing a schematic partial structure of an electronic device 100 according to an embodiment of the present application. As shown in FIG. 14, the electronic device 100 includes a processor 10, a memory 20, and a camera 30. The camera 30 includes at least a rear camera 31 and a front camera 32. The rear camera 31 is used for capturing an image behind the electronic device 100, and can be used for a user to take a photographing operation such as a person. The front camera 32 is used to capture an image in front of the electronic device 100, and can be used to implement a self-photographing and the like.

In some embodiments, the models described in FIGS. 1-13 may be programs such as specific algorithm functions running in processor 10, such as neural network algorithm functions, image processing algorithm functions, and the like. In other embodiments, the electronic device 100 may further include a model processor independent of the processor 10, and the models described in FIGS. 1 to 13 are in an operation and model processor, and the processing is performed. The controller 10 can generate a corresponding instruction to trigger the model processor to run the corresponding model, and the output result of the model is output to the processor 10 through the model processor for use by the processor 10, and performs control operations such as shooting operations. .

Program instructions are stored in the memory 20.

The processor 10 is configured to invoke a program instruction stored in the memory 20 to execute a model training process in the shooting control method in any of the embodiments shown in FIGS. 1 to 6, and execute as shown in FIGS. 7-14. The process of the automatic shooting control using the model in the shooting control method in any of the embodiments.

For example, the processor 10 is configured to call a program instruction stored in the memory 20 to execute the following shooting control method:

When the user performs manual control shooting, the currently taken picture is taken as a positive sample satisfying the shooting condition by the model, and the parameters of the model itself are adjusted according to the positive sample of the current time;

Sampling a frame frame that has not been manually controlled by a model by a preset rule as a reverse sample, and adjusting a parameter of the model itself according to the reverse sample;

Upon determining that the training completion condition is reached, the training is ended and the trained model is obtained for subsequent automatic shooting control.

In some embodiments, when the user performs manual control shooting by the processor 10, the processor 10 calls the currently taken picture as a positive sample satisfying the shooting condition by the model, and adjusts the model's own parameters according to the current positive sample. The method includes: saving the positive sample by the model, and establishing or updating the correspondence between the front sample and the shooting condition to adjust the parameter of the model itself. Among them, the photographing condition can be marked as a label of the front sample.

In one implementation, the processor call program instructions are further executed to enter the model training mode in response to user input of the entered model training. The determining that the processor invokes the execution of the program instruction reaches the training completion condition includes: determining that the training completion condition is reached in response to the user inputting the operation of exiting the model training mode.

Optionally, in another implementation manner, the determining that the processor invokes the execution of the program instruction reaches the training completion condition, and: determining that the training completion condition is reached when determining that the number of times the user manually controls the shooting reaches the preset number of times N1. The preset number of times N1 may be the number of times the system default model training needs to be performed, or may be a user-defined value.

Optionally, in another implementation manner, the determining that the processor invokes the execution of the program instruction reaches a training completion condition, including: using the positive sample of the current to test the model, determining whether the test result reaches a preset threshold, and testing The result reaches a preset threshold and determines that the training completion condition is reached.

The processor 10 can be a microcontroller, a microprocessor, a single chip, a digital signal processor, or the like.

The memory 20 can be any storage device that can store information such as a memory card, a solid state memory, a micro hard disk, an optical disk, or the like.

As shown in FIG. 14, the electronic device 100 further includes an input unit 40 and an output unit 50. The input unit 40 may include a touch panel, a mouse, a microphone, a physical button including a power button, a volume button, and the like. The output unit 50 can include a display screen, a speaker, and the like. In some embodiments, the touch panel of the input unit 40 and the display screen of the output unit 50 are integrated to form a touch screen while providing the functions of touch input and display output.

In some embodiments, the present application further provides a computer readable storage medium, where the computer readable storage medium stores a plurality of program instructions, and the program instructions are executed by the main processing unit 20, and executed as shown in FIG. - all or part of any of the shooting control methods shown in FIG. In some embodiments, the computer storage medium is the memory 20, and may be any storage device that can store information such as a memory card, a solid state memory, a micro hard disk, an optical disk, or the like.

The shooting control method and the electronic device 100 of the present application can perform model training first, and are used for subsequent automatic shooting control after the model training is completed, and capture the wonderful moments in time. Specifically, the model can automatically determine whether the shooting condition is satisfied according to the shooting preview screen, and when the shooting condition is satisfied, the exciting moment including the corresponding content of the current shooting preview screen can be captured in time.

Although the present invention has been described herein in connection with the embodiments of the present invention, it will be understood by those skilled in the <RTIgt; Other variations of the disclosed embodiments are achieved. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill several of the functions recited in the claims. Certain measures are recited in mutually different dependent claims, but this does not mean that the measures are not combined to produce a good effect.

Those skilled in the art will appreciate that embodiments of the present invention can be provided as a method, apparatus (device), or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or a combination of software and hardware. Moreover, the invention can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code. The computer program is stored/distributed in a suitable medium, provided with other hardware or as part of the hardware, or in other distributed forms, such as over the Internet or other wired or wireless telecommunication systems.

The present invention has been described with reference to flowchart illustrations and/or block diagrams of the methods, apparatus, and computer program products of the embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or FIG. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing device to produce a machine for the execution of instructions for execution by a processor of a computer or other programmable data processing device. Means for implementing the functions specified in one or more of the flow or in a block or blocks of the flow chart.

The above disclosure is only an embodiment of the present application, and of course, the scope of the application should not be limited thereto, and those skilled in the art can understand all or part of the process of implementing the above embodiments, and according to the claims of the present application. The equivalent change is still within the scope of the application.

Claims

A shooting control method, characterized in that the shooting control method comprises:

When the user performs manual control shooting, the currently taken picture is taken as a positive sample satisfying the shooting condition by the model, and the parameters of the model itself are adjusted according to the positive sample of the current time;

The image frame that is not manually controlled is sampled by the model as a reverse sample by using a preset rule, and the parameters of the model itself are adjusted according to the reverse sample;

Upon determining that the training completion condition is reached, the training is ended and the trained model is obtained for subsequent automatic shooting control.
The photographing control method according to claim 1, wherein the photographing control method further comprises: controlling to enter the model training mode in response to an operation of entering the model training input by the user;

When it is determined that the training completion condition is reached, the training is ended to obtain the trained model for subsequent automatic shooting control, including:

In response to the user-entered operation of the exit model training mode, it is determined that the training completion condition is reached, and the training is ended to obtain the trained model for subsequent automatic shooting control.
The photographing control method according to claim 1, wherein, when it is determined that the training completion condition is reached, the training is ended to obtain a trained model for subsequent automatic photographing control, including: determining user manual control When the number of times of shooting reaches a preset number of times, it is determined that the training completion condition is reached, and the training is ended to obtain the trained model for subsequent automatic shooting control.
The photographing control method according to claim 1, wherein, when it is determined that the training completion condition is reached, the training is ended to obtain the trained model for subsequent automatic photographing control, including:

When the user performs manual control shooting, the parameters of the picture obtained by manually controlling the shooting are input to the model to obtain a predicted value;

Determining whether the predicted value meets the shooting conditions;

If the predicted value satisfies the shooting condition, it is determined that the training completion condition is reached, and the training is ended to obtain the trained model for subsequent automatic shooting control.
The photographing control method according to claim 4, wherein if the predicted value satisfies the photographing condition, it is determined that the training completion condition is reached, and the training is ended to obtain the trained model for subsequent automatic photographing control, including :

If the predicted value satisfies the shooting condition, the number of counts in which the predicted value satisfies the shooting condition is incremented by one to obtain an updated count number of times;

Determine whether the current number of counts exceeds a preset number of times;

If the current number of counts exceeds the preset number of times, it is determined that the training completion condition is reached, and the training is ended to obtain the trained model for subsequent automatic shooting control.
The photographing control method according to claim 1, wherein when the user performs manual control photographing, the currently photographed image is taken as a positive sample satisfying the photographing condition by the model, and the model itself is adjusted according to the current positive sample. Parameters, including:

When the user performs manual control shooting, the currently captured picture is taken as a positive sample corresponding to the shooting conditions and the shooting parameters by the model, and the parameters of the model itself are adjusted according to the current front sample.
The photographing control method according to any one of claims 1 to 4, wherein the photographing control method further comprises the steps of:

After the automatic shooting function is turned on, the framing preview is performed in response to the opening operation of the camera to obtain a shooting preview screen;

When it is determined according to the current model that the preview screen information satisfies the shooting condition, controlling to perform an automatic shooting operation;

Obtain feedback feedback from users on this automatic shooting;

The satisfaction feedback information of the user for the automatic shooting is output to the currently used model, so that the currently used model performs optimization training using the satisfaction feedback information and the preview screen information.
The photographing control method according to any one of claims 1 to 4, wherein the sampling of the frame frame that is not manually controlled by the model by the preset rule as the back surface sample comprises: sampling adjacent two at a preset time interval The frame frame obtained by the framing between the shots is taken as the reverse sample.
The photographing control method according to any one of claims 1 to 6, wherein the sampling of the frame frame that is not manually controlled by the model by the preset rule as the reverse sample includes: after the front sample is in a period of time or before The framed picture framed for a period of time is sampled as a reverse sample.
The photographing control method according to any one of claims 1 to 6, wherein the sampling of the frame frame that is not manually controlled by the model by the preset rule as the reverse sample comprises: obtaining the uncontrolled shooting by random sampling. The picture frame is used as the reverse side sample.
The photographing control method according to any one of claims 1 to 4, wherein the sampling of the frame frame that is not manually controlled by the model by the preset rule as the back surface sample comprises: determining, by the detection result of the sensor, that sampling is required At the same time, the sampling of the picture frame is performed, and the sampled picture frame is taken as the reverse side sample.
The photographing control method according to any one of claims 1 to 6, wherein the sampling of the frame frame that is not manually controlled by the model by the preset rule as the back surface sample comprises: collecting the frame frame that is not manually controlled and capturing The save is performed, and the saved picture frame is subjected to composition analysis, and the picture frame as the reverse side sample is determined.
An electronic device, the electronic device comprising:

a memory for storing program instructions;

A processor for invoking the program instruction to perform the photographing control method according to any one of claims 1-12.
A computer readable storage medium, wherein the computer readable storage medium stores program instructions for performing a photographing control according to any one of claims 1 to 12 after being invoked by a computer method.