WO2021179831A1 - 拍照方法、装置、电子设备及存储介质 - Google Patents

拍照方法、装置、电子设备及存储介质 Download PDF

Info

Publication number
WO2021179831A1
WO2021179831A1 PCT/CN2021/074205 CN2021074205W WO2021179831A1 WO 2021179831 A1 WO2021179831 A1 WO 2021179831A1 CN 2021074205 W CN2021074205 W CN 2021074205W WO 2021179831 A1 WO2021179831 A1 WO 2021179831A1
Authority
WO
WIPO (PCT)
Prior art keywords
point
sub
model
composition
target
Prior art date
Application number
PCT/CN2021/074205
Other languages
English (en)
French (fr)
Inventor
罗彤
Original Assignee
Oppo广东移动通信有限公司
上海瑾盛通信科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oppo广东移动通信有限公司, 上海瑾盛通信科技有限公司 filed Critical Oppo广东移动通信有限公司
Publication of WO2021179831A1 publication Critical patent/WO2021179831A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/61—Control of cameras or camera modules based on recognised objects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
    • H04N23/631—Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
    • H04N23/632—Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
    • H04N23/633—Control of cameras or camera modules by using electronic viewfinders for displaying additional information relating to control or operation of the camera
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/64—Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image

Definitions

  • This application relates to the field of image processing technology, and in particular to a photographing method, device, electronic equipment, and storage medium.
  • the embodiments of the present application provide a photographing method, device, electronic equipment, and storage medium. Be able to provide composition suggestions for the subject and take pictures.
  • an embodiment of the present application provides a photographing method, which includes:
  • the shooting scene is photographed to obtain a photographed image.
  • an embodiment of the present application provides a photographing device, which includes:
  • An acquisition module configured to acquire a preview image of a shooting scene, and call a key point recognition model to perform key point detection on the preview image, and obtain the target key points of the shooting subject in the shooting scene;
  • the first determining module is configured to determine the current composition type according to the target key point, and determine the anchor point corresponding to the photographing subject according to the composition type and the target key point;
  • a second determining module configured to determine a composition point corresponding to the photographing subject according to the positioning point and the composition type
  • the prompt module is configured to output prompt information for instructing to adjust the shooting posture of the electronic device when the positioning point does not match the composition point;
  • the photographing module is used for photographing the photographing scene when the positioning point matches the composition point to obtain a photographed image.
  • the storage medium provided by the embodiment of the present application has a computer program stored thereon, and when the computer program runs on the computer, the computer is caused to execute the photographing method provided in any embodiment of the present application.
  • an embodiment of the present application provides an electronic device, including a memory and a processor, the memory stores a computer program, and the processor invokes the computer program stored in the memory to execute:
  • the shooting scene is photographed to obtain a photographed image.
  • FIG. 1 is a schematic diagram of the first process of a photographing method provided by an embodiment of the present application.
  • FIG. 2 is a schematic diagram of a second process of a photographing method provided by an embodiment of the present application.
  • Fig. 3 is a schematic structural diagram of a key point recognition model provided by an embodiment of the present application.
  • FIG. 4 is a schematic diagram of a first structure of a second recognition sub-model provided by an embodiment of the present application.
  • FIG. 5 is a schematic diagram of a second structure of a second recognition sub-model provided by an embodiment of the present application.
  • FIG. 6 is a schematic diagram of a third structure of a second recognition sub-model provided by an embodiment of the present application.
  • Fig. 7 is a schematic flow chart of determining a composition type provided by an embodiment of the present application.
  • FIG. 8 is a schematic diagram of a composition three-point line provided by an embodiment of the present application.
  • Fig. 9 is a schematic diagram of composition information provided by an embodiment of the present application.
  • FIG. 10 is a schematic diagram of a first structure of a photographing device provided by an embodiment of the present application.
  • FIG. 11 is a schematic diagram of a second structure of a photographing device provided by an embodiment of the present application.
  • FIG. 12 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.
  • module used in this article can be regarded as a software object executed on the computing system.
  • different modules, engines and services can be regarded as the implementation objects on the computing system.
  • the embodiment of the present application provides a photographing method, and the execution subject of the photographing method may be the photographing device provided in the embodiment of the present application, or an electronic device integrated with the photographing device.
  • the electronic device may be a smart phone, a smart wearable device, a tablet computer, a PDA (Personal Digital Assistant), etc. Detailed descriptions are given below.
  • FIG. 1 is a schematic diagram of a first process of a photographing method provided by an embodiment of the present application.
  • the photographing method can provide composition suggestions for the subject and take pictures.
  • the photographing method may include the following steps:
  • a preview image is generated on the screen of the electronic device so that the photographer can view the current picture information at any time.
  • the electronic device can automatically detect whether there is a subject in the preview image, and when there is a subject in the preview image, it can automatically call the key point recognition model to identify the key points of the preview image.
  • the subject can be a variety of subjects that can be photographed, such as people, animals, plants, dolls, and dolls.
  • the key point recognition model can identify one of the subjects, and it can also identify multiple subjects.
  • the preview image may include one or more shooting subjects, and the one or more shooting subjects have corresponding key points.
  • the complete doll image includes multiple key points, such as the head, chest, limbs, neck, and joints.
  • the key points identified by the key point recognition model are not necessarily key points on the subject, but may also be key points on other subjects, such as buildings on the side of the road and passers-by when shooting portraits.
  • the key points identified by the key point recognition model can be screened to obtain the target key points of the subject.
  • the key points of the subject are in different parts of the subject. For example, if the subject is a doll, the head, hands, legs and other parts of the doll have key points. These key points can be used to determine the composition type of the subject.
  • the specific part of the subject can be identified according to the target key points of the subject. For example, if the target key point is a key point on the head of the puppet, then the current composition type can be determined as a portrait according to the shooting part. . If the photographed parts identified according to the key points of the target include the head, hands, torso, legs, and feet, it means that the current composition type is a full-length image.
  • the composition type can be determined according to the number of target key points or the ratio between the target key points. If the number of target key points on the head is greater than the number of target key points on the torso, confirm that the head is the main shooting location. Or if the ratio of the number of target key points on the head to the number of target key points on the trunk is greater than the preset ratio threshold, it is confirmed that the head is the main shooting part. Then the composition type of the subject is portrait.
  • the composition type can also be directly determined according to the distribution of the target key points.
  • the head, hands, torso, legs, feet and other parts of the figure corresponding to the target key points are based on the target key points.
  • the distribution of can determine that the composition type of the subject is a full-length image. It should be noted that for different shooting subjects, there may be different parts and different composition types.
  • the dolls in the embodiments of the present application are only examples.
  • the positioning point of the subject may be determined according to the composition type and the key points of the target.
  • the anchor point may be a point representing the photographed subject, and the anchor point contains position information of the photographed subject.
  • the obtained composition type is a portrait
  • the midpoint of the two target key points with the furthest face apart from the target key points can be selected as the anchor point.
  • the composition type is a full-length image
  • composition point can be generated to guide the photographer and the subject to take a photo.
  • the anchor point has detailed position information in the preview image.
  • the preview image is a rectangular image
  • a rectangular coordinate system is established for the rectangular image
  • the positioning point has specific coordinate position information in the rectangular image.
  • This coordinate position information can represent the position information of the photographing subject in the preview image.
  • composition types such as full-length portrait, bust portrait, portrait, etc.
  • the composition point can be determined according to the composition type and positioning point. For example, first determine the specific position of the subject in the preview image according to the anchor point, and then generate a visible composition point in the preview image according to the position and composition type. For example, after obtaining the location of the anchor point, you can select the composition database according to the composition type. The composition database contains the matching information of the anchor point and the position of the composition point. After the location of the anchor point is determined, you can directly search the database according to the location of the anchor point. Corresponding composition point position.
  • the position of the composition point may be directly calculated by a preset algorithm, and then the position of the composition point is displayed on the preview image.
  • the position of the composition point corresponding to the positioning point may be one or more.
  • the anchor point and any composition point meet the preset conditions, it means that the composition of the subject is successful.
  • the positioning point and the composition point have been matched; if the Euclidean distance between the positioning point and the composition point is not within the preset distance range, the positioning point is indicated Does not match the composition point.
  • the positioning point and the composition point may be matched.
  • marks of the positioning point and the composition point are displayed on the preview image, and the user is prompted to adjust the shooting posture.
  • an indicator arrow is set between the positioning point and the composition point to indicate the direction or/and distance of the positioning point to be adjusted, and then the user can adjust the shooting posture of the electronic device or adjust the shooting position of the subject.
  • the shooting scene is photographed to obtain a photographed image.
  • the Euclidean distance between the positioning point and the composition point is within the preset distance range, indicating that the subject meets the composition conditions at this time, and the shooting scene can be directly photographed to obtain the captured image.
  • the target key points of the subject in the shooting scene are obtained; then the current composition type is determined according to the target key points, And according to the composition type and key points of the target to determine the corresponding positioning point of the subject; finally according to the positioning point and composition type to determine the corresponding composition point of the subject; when the positioning point does not match the composition point, output for instructions to adjust the electronic device Prompt information of the shooting posture; when the positioning point matches the composition point, the shooting scene is shot to obtain the shooting image.
  • the positioning points and corresponding composition points of the subject are obtained, and the composition suggestions during shooting are realized through the positioning points and composition points, and when the positioning points and composition points meet the preset conditions Take pictures automatically.
  • FIG. 2 is a schematic diagram of the second process of the photographing method provided by the embodiment of the present application.
  • the photographing method can provide composition suggestions for the subject and take pictures.
  • the photographing method may include the following steps:
  • a preview image is generated on the camera interface of the electronic device, and the preview image changes according to the shooting posture of the electronic device.
  • the preview interface when the preview interface is captured, the preview interface can be identified to determine whether there is a photographed subject, and if there is a photographed subject, a detection frame is generated for the photographed subject.
  • the detection frame contains the shooting subject, and the shape of the detection frame can be a regular shape such as a rectangle, a circle, an ellipse, etc., or an irregular shape, such as a stroke shape of the shooting subject.
  • the area of the detection frame can be calculated, and then the area of the detection frame is divided by the area of the preview screen to obtain the area ratio of the preview image occupied by the measurement frame. Then it is judged whether the area ratio exceeds the preset threshold.
  • step 202 is continued.
  • the key point recognition model includes a first sub-model and a second sub-model. Please refer to FIG. 3 for details.
  • FIG. 3 is a schematic structural diagram of a key point recognition model provided by an embodiment of the present application. Among them, after the preview image is obtained, the preview image can be directly input to the first sub-model for recognition, and a feature map of the preview image is obtained.
  • the first sub-model may be a mobilenet v2 model, which is lighter in weight and faster in processing pictures.
  • the first sub-model can also use a higher-precision model, such as the VGG19 model, the resnet50 model, etc. Using these models to extract features can improve the detection accuracy of key points of the human body.
  • the second sub-model includes multiple, and the multiple second sub-models are connected in sequence.
  • the first sub-model is connected to the first second sub-model, that is, the first sub-model is connected to the second sub-model (1) in the figure.
  • each second sub-model can output corresponding location features and connection features.
  • the location feature can be a three-dimensional matrix or a three-dimensional matrix of height*width*keypoints, where height represents the height of the picture, width represents the width of the picture, and keypints represents the number of key points, where the picture corresponds to each connection feature picture of.
  • the specific location feature can be a heatmap.
  • the connection feature can be a three-dimensional matrix or a three-dimensional matrix of height*width*limbs. Among them, height represents the height of the image, width represents the width of the image, and limbs represents the number of connectors.
  • the connecting body may be the connecting area between two associated key points.
  • the connecting body of the left eye and the right eye may be a connecting body, and the connecting body may be the connecting area between the left eye and the right eye.
  • Each connection body corresponds to a three-dimensional matrix of height*width*2.
  • the connection feature can be considered as a dual-channel heat map. Each position in the dual-channel heat map includes two values, such as x value and y value. Compose a vector (x, y), which can indicate the direction of the connected body at the corresponding position. When both the x value and the y value are zero, it means that there is no limb at the position.
  • the input of the first second sub-model is the feature map output by the first sub-model.
  • the first second sub-model processes the feature map to obtain the location features and connection features output by the first second sub-model.
  • FIG. 4 is a schematic diagram of the first structure of the second sub-model provided by an embodiment of the present application.
  • the first and second sub-model processes the feature map, it outputs the location features and connection features output by the first and second sub-models. That is, the second sub-model (1) processes the feature map to obtain the location feature (1) and the connection feature (1).
  • the location feature (2), the connection feature (2), and the feature map into the third second sub-model, that is, input into the second sub-model (3), and get the output position of the second sub-model (3) Features (3) and connection features (3).
  • the remaining second sub-models except for the first second sub-model all the connection features and location features output by the previous second sub-model and the feature map output by the first sub-model are used as input.
  • the second sub-model outputs its corresponding connection feature and location feature. Until the last second sub-model outputs the target connection feature and the target location feature.
  • FIG. 5 is a schematic diagram of a second structure of the second sub-model provided by an embodiment of the present application. Specifically, Figure 5 shows a schematic diagram of the structure of the first and second sub-models. Among them, the feature map output by the first sub-model is used as input, and input into the first second sub-model.
  • the second sub-model includes a connection module and a position module. Both the connection module and the position module are modules composed of multiple different types of convolutional layers.
  • connection module includes multiple first convolutional layers and multiple second convolutional layers, multiple first convolutional layers are connected in sequence, multiple second convolutional layers are connected in sequence, and the last one is the first convolutional layer. Connect with the first second convolutional layer.
  • the position module includes multiple first convolutional layers and multiple second convolutional layers. Multiple first convolutional layers are connected in sequence, multiple second convolutional layers are connected in sequence, and the last first convolutional layer is connected to the first convolutional layer. A second convolutional layer connection.
  • the first convolutional layer may be a 3*3 convolutional layer
  • the second convolutional layer may be a 1*1 convolutional layer
  • the connection module there may be three first convolutional layers and two second convolutional layers, and the structure of the position module may be the same as that of the connection module.
  • the type and quantity of the first convolutional layer and the second convolutional layer can be changed according to actual requirements.
  • connection modules and location modules can process the feature maps separately , Thus get the connection feature (1) and the location feature (1).
  • FIG. 6 is a schematic diagram of the third structure of the second sub-model provided by an embodiment of the present application. Specifically, FIG. 6 shows a schematic structural diagram of the remaining second sub-models of the first second sub-model.
  • Each of the second sub-models includes a connection module and a position module, and the connection module and the position module include a plurality of different types of convolutional layers.
  • connection module includes multiple third convolutional layers and multiple second convolutional layers, multiple third convolutional layers are connected in sequence, multiple second convolutions are connected in sequence, and the last third convolutional layer and the first convolutional layer are connected in sequence.
  • the position module includes five third convolutional layers and two second convolutional layers. Multiple third convolutional layers are connected in sequence, multiple second convolutional layers are connected in sequence, and the last third convolutional layer and the first The second convolutional layer is connected.
  • the third convolutional layer may be a 7*7 convolutional layer
  • the second convolutional layer may be a 1*1 convolutional layer.
  • the connection module there may be five third convolutional layers and two second convolutional layers, and the structure of the position module may be the same as that of the connection module.
  • the type and quantity of the first convolutional layer and the second convolutional layer can be changed according to actual requirements.
  • each second sub-model is the connection feature output by the previous second sub-model, the location feature and the output of the first sub-model Feature map, that is, connecting feature (M-1) and location feature (M-1).
  • Each second sub-model can output its corresponding connection feature and location feature, that is, connection feature (M) and location feature (M). It should be noted that the output of the last second sub-model is the target location feature and the target connection feature.
  • the third convolutional layer in the remaining second sub-models except the first second sub-model, can be replaced with the first convolutional layer, thereby reducing the amount of calculation and parameters, so that the second Sub-models process tasks faster.
  • the position of the maximum value in the target location feature may be selected as the candidate key point.
  • the candidate key point is selected as the candidate key point in the heat map (heatmap).
  • the heat map can be pooled to the maximum, and then the heat maps before and after pooling are compared, and the positions with the same values in the heat maps before and after pooling are used as candidate key points.
  • the candidate key points can be connected according to the direction of the connector in the target connection feature to obtain a complete individual.
  • the target connection feature corresponding to one connector can be acquired at a time, and the candidate key points at both ends of the connector can be connected.
  • the confidence that the two candidate key points are from the same individual can be expressed by the following confidence formula:
  • P(u) is the position of interpolation between the two candidate key points
  • L c is the value of P(u) in the target connection feature
  • a key point association set can be generated, and the association set has a set of candidate key points for each individual. For example, there are candidate key points corresponding to the eyes, candidate key points corresponding to the nose, and so on on the candidate key points. In each of the photographed individuals, both eyes and nose have corresponding candidate key points, and both wrist and elbow have corresponding key points. Multiple candidate key points can form an individual representing the photographed subject. Multiple candidate keypoint sets form an association set. You can find an optimal association set in it, such as:
  • j 1 , j 2 represent the key point category (eyes, nose, wrist, etc.), and m and n represent the key point numbers in the corresponding key point category.
  • E c is the total confidence of all connected connectors, that is, the total confidence of individuals formed by connecting multiple connectors.
  • the Hungarian algorithm can be used to match to get the best association set.
  • Confidence is used to determine the degree of association between candidate key points, so as to determine the target key points on the subject. That is, when the confidence is higher, the correlation between candidate key points is higher, and the more likely it is from the same individual.
  • the target key points can be identified, for example, whether the target key points are mainly concentrated on the head, or the feet, chest, whole body, partial body, and so on.
  • the specific part of the subject can be identified according to the target key points of the subject. For example, if the target key point is a key point on the head of the puppet, then the current composition type can be determined as a portrait according to the shooting part. . If there are head, hands, torso, legs, and feet that are identified according to the key points of the target, the current composition type is a full-body image.
  • FIG. 7 is a schematic diagram of a process for determining a composition type according to an embodiment of the present application.
  • the composition type can be determined according to the attributes of the target key points. for example:
  • step 302. Determine whether the target key point includes the head key point. If the key points of the head are not included, go to step 303, indicating that it is a close-up of a part of the body.
  • step 304 If the head key points are included, go to step 304 to detect whether only the head key points are included. If only the key points of the head are included, step 305 is entered, indicating that it is a close-up of the face.
  • step 306 If it does not only include the key points of the head, proceed to step 306 to detect whether the key points of the feet are included. If the key points of the feet are included, go to step 307, indicating that it is a close-up of the whole body.
  • step 308 determine whether the key points of the hip joint are included.
  • step 309 If the key points of the hip joint are included, go to step 309, indicating that it is a close-up of the chest
  • step 310 indicating that it is a close-up of the seven avatars.
  • the composition type corresponds to different preset positioning point selection methods. For example, if it is a close-up of a part of the body, take the center point of the detection frame as the positioning point. It is understandable that different composition types correspond to different ways of selecting preset anchor points.
  • the corresponding anchor point can be determined according to the coordinate information of the key point. For example, if the selection method of the preset anchor point is the selection method corresponding to the face close-up, the coordinate information of the target key point is obtained. If the mean value of the abscissa of the key points of the eyes, nose and mouth is within the 1/4 length range of the label frame in the horizontal direction Inside, it is considered that the side face is shot, and the center of the detection frame is taken as the anchor point.
  • the composition types can be divided into two categories, one is a close-up of the face that only includes the head, and the other is a close-up of the body that includes the body.
  • the head may or may not be included in the close-up of the body.
  • FIG. 8 is a schematic diagram of a composition three-point line provided by an embodiment of the present application.
  • the two three-point lines A and B connect the side length of the preview image with the longer side
  • the two three-point lines C and D connect the side length of the preview image with the shorter side.
  • the preview image is horizontal; when C and D are vertical lines, the preview image is vertical.
  • the horizontal and vertical screen information of the preview image can be obtained.
  • the candidate composition point corresponding to the subject can be selected as the exact center of the preview image, the midpoint of the three-point line C, and the intersection point of the three-point line C and the three-point lines A and B.
  • the candidate composition point may be the center of the preview image and the midpoint of the third line A.
  • the horizontal and vertical screen information of the preview image can be obtained.
  • the candidate composition point corresponding to the subject can be selected as the exact center of the preview image and the midpoint of the third line C.
  • the candidate composition point selects the intersection of the three-pointers A, B, C, and D.
  • the candidate composition point closest to the anchor point can be selected from the candidate composition points as the final composition point. That is, the composition point of the subject.
  • the positioning point and the composition point match. For example, it is determined whether the Euclidean distance between the composition point and the positioning point is less than a preset distance threshold, or whether the Euclidean distance between the composition point and the positioning point is within a preset range.
  • a prompt message can be generated on the screen to prompt the user to adjust the electronic device Shooting pose.
  • FIG. 9 is a schematic diagram of composition information provided by an embodiment of the present application.
  • an indicator arrow pointing the positioning point toward the composition point can be generated to prompt the user to adjust the shooting posture of the electronic device to recompose the picture.
  • the shooting scene is photographed to obtain a photographed image.
  • the prompt information on the preview image can disappear, and the electronic device automatically shoots The subject takes a picture.
  • the prompt message changes color and generates text to prompt the user to take a photo, and the user can manually take a photo according to the prompt message.
  • the key points of the preview image can be automatically recognized according to the key point recognition model, and the key points of the preview image are processed to obtain the target key points of the subject .
  • the composition type is determined according to the target key points
  • the anchor point is determined according to the composition type and the target key point
  • the composition point corresponding to the subject is determined according to the anchor point and the composition type.
  • FIG. 10 is a first structural diagram of the photographing device provided by an embodiment of the present application.
  • the photographing device includes an acquiring module 410, a first determining module 420, a second determining module 430, a prompting module 440, and a photographing module 450.
  • the acquiring module 410 is configured to acquire a preview image of a shooting scene, and call a key point recognition model to perform key point detection on the preview image, and obtain the target key points of the shooting subject in the shooting scene.
  • the acquisition module 410 can automatically detect whether there is a photographed subject in the preview image, and when there is a photographed subject in the preview image, it automatically calls the key point recognition model to perform key point recognition on the preview image.
  • the key points identified by the key point recognition model are not necessarily key points on the subject, but may also be key points on other subjects, such as buildings on the side of the road and passers-by when shooting portraits.
  • the key points identified by the key point recognition model can be screened to obtain the target key points of the subject.
  • FIG. 11 is a schematic diagram of a second structure of a photographing device provided by an embodiment of the present application.
  • the acquiring module 410 includes a first determining sub-module 411 and a second determining sub-module 412.
  • the first determining sub-module 411 is configured to determine candidate key points among the key points according to the target location feature.
  • the first determining sub-module 411 may select the position of the maximum value in the target location feature as the candidate key point, for example, select the candidate key point with the largest pixel value in a heat map.
  • the heat map can be pooled to the maximum, and then the heat maps before and after pooling are compared, and the positions with the same values in the heat maps before and after pooling are used as candidate key points.
  • the second determining sub-module 412 is configured to determine the target key point of the photographing subject according to the target connection feature and the candidate key point.
  • the candidate key points can be connected according to the direction of the connector in the target connection feature to obtain a complete individual.
  • the confidence level between the key points can be obtained, and the higher the confidence level is, the higher the correlation between the key points is, so as to determine the target key point of the subject.
  • the first determining module 420 is configured to determine the current composition type according to the target key point, and determine the anchor point corresponding to the photographing subject according to the composition type and the target key point.
  • the key points of the subject are in different parts of the subject. For example, if the subject is a doll, the head, hands, legs and other parts of the doll have key points. These key points can be used to determine the composition type of the subject.
  • the specific part of the subject can be identified according to the target key points of the subject. For example, if the target key point is a key point on the head of the puppet, then the current composition type can be determined as a portrait according to the shooting part. . If there are head, hands, torso, legs, and feet that are identified according to the key points of the target, the current composition type is a full-body image.
  • the second determining module 430 is configured to determine a composition point corresponding to the subject according to the positioning point and the composition type.
  • composition point can be generated to guide the photographer and the subject to take a photo.
  • the anchor point has detailed position information in the preview image.
  • the preview image is a rectangular image
  • a rectangular coordinate system is established for the rectangular image
  • the positioning point has specific coordinate position information in the rectangular image.
  • This coordinate position information can represent the position information of the photographing subject in the preview image.
  • composition types such as full-length portrait, bust portrait, portrait, etc.
  • the composition point can be determined according to the composition type and positioning point. For example, first determine the specific position of the subject in the preview image according to the anchor point, and then generate a visible composition point in the preview image according to the position and composition type. For example, after obtaining the location of the anchor point, you can select the composition database according to the composition type. The composition database contains the matching information of the anchor point and the position of the composition point. After the location of the anchor point is determined, you can directly search the database according to the location of the anchor point. Corresponding composition point position.
  • the second determination module 430 further includes an acquisition sub-module 431, a third determination sub-module 432 and a selection sub-module 433.
  • the obtaining sub-module 431 is configured to obtain horizontal and vertical screen information of the preview image.
  • FIG. 8 is a schematic diagram of a composition three-point line provided by an embodiment of the present application.
  • the two three-point lines A and B connect the side length of the preview image with the longer side
  • the two three-point lines C and D connect the side length of the preview image with the shorter side.
  • the preview image is horizontal; when C and D are vertical lines, the preview image is vertical.
  • the third determining sub-module 432 is configured to determine a plurality of candidate composition points corresponding to the photographing subject according to the horizontal and vertical screen information of the preview image, the positioning point, and the composition type.
  • the horizontal and vertical screen information of the preview image can be obtained.
  • the candidate composition point corresponding to the subject can be selected as the exact center of the preview image, the midpoint of the three-point line C, and the intersection point of the three-point line C and the three-point lines A and B.
  • the candidate composition point may be the center of the preview image and the midpoint of the third line A.
  • the horizontal and vertical screen information of the preview image can be obtained.
  • the candidate composition point corresponding to the subject can be selected as the exact center of the preview image and the midpoint of the third line C.
  • the candidate composition point selects the intersection of the three-pointers A, B, C, and D.
  • the selection sub-module 433 is configured to select the candidate composition point closest to the positioning point from among the plurality of candidate composition points as the composition point.
  • the prompt module 440 is configured to output prompt information for instructing to adjust the shooting posture of the electronic device when the positioning point does not match the composition point.
  • the positioning point and the composition point may be matched.
  • marks of the positioning point and the composition point are displayed on the preview image, and the user is prompted to adjust the shooting posture.
  • an indicator arrow is set between the positioning point and the composition point to indicate the direction or/and distance of the positioning point to be adjusted, and then the user can adjust the shooting posture of the electronic device or adjust the shooting position of the subject.
  • the photographing module 450 is configured to photograph the photographing scene when the positioning point matches the composition point to obtain a photographed image.
  • the Euclidean distance between the positioning point and the composition point is within the preset distance range, indicating that the subject meets the composition conditions at this time, and the shooting scene can be directly photographed to obtain the captured image.
  • the preview image of the shooting scene is acquired, and the key point recognition model is called to perform key point detection on the preview image to obtain the target key points of the subject in the shooting scene; the current key points are determined according to the target key points The composition type, and determine the positioning point corresponding to the subject according to the composition type and the key points of the target; determine the composition point of the corresponding subject according to the positioning point and the composition type; when the positioning point does not match the composition point, the output is used to instruct the adjustment of the electronic device Prompt information of the shooting posture; when the positioning point matches the composition point, the shooting scene is shot to obtain the shooting image.
  • the key point recognition model is called to perform key point detection on the preview image to obtain the target key points of the subject in the shooting scene
  • the current key points are determined according to the target key points
  • the composition type and determine the positioning point corresponding to the subject according to the composition type and the key points of the target
  • determine the composition point of the corresponding subject according to the positioning point and the composition type when the positioning point does not match the composition point,
  • an embodiment of the present application also provides an electronic device, as shown in FIG. 12, which is a schematic structural diagram of the electronic device provided in its own embodiment.
  • the electronic device may include an input unit 510 including one or more computer-readable storage media, a display unit 520, a power supply 530, a WIFI module 540, a sensor 550, a memory 560, and a processor including one or more processing cores 570 and other parts.
  • an input unit 510 including one or more computer-readable storage media
  • a display unit 520 including one or more computer-readable storage media
  • a power supply 530 including one or more WIFI module 540
  • a sensor 550 a sensor 550
  • a memory 560 a memory
  • a processor including one or more processing cores 570 and other parts.
  • FIG. 12 does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or a combination of certain components, or different component arrangements. in:
  • the input unit 510 can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.
  • the touch-sensitive surface may include two parts: a touch detection device and a touch controller.
  • the touch detection device detects the user's touch position, detects the signal brought by the touch operation, and transmits the signal to the touch controller;
  • the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it To the processor 570, and can receive and execute the commands sent by the processor 570.
  • multiple types such as resistive, capacitive, infrared, and surface acoustic waves can be used to realize touch-sensitive surfaces.
  • the input unit 510 may also include other input devices.
  • the display unit 520 may include a display panel.
  • the display panel may be configured in the form of a liquid crystal display (LCD, Liquid Crystal Display), an organic light emitting diode (OLED, Organic Light-Emitting Diode), etc.
  • the touch-sensitive surface may cover the display panel. When the touch-sensitive surface detects a touch operation on or near it, it is transmitted to the processor 570 to determine the type of the touch event, and then the processor 570 displays the display panel according to the type of the touch event. Corresponding visual output is provided on the panel.
  • the touch-sensitive surface and the display panel are used as two independent components to realize the input and input functions, in some embodiments, the touch-sensitive surface and the display panel may be integrated to realize the input and output functions.
  • WiFi is a short-distance wireless transmission technology. Electronic devices can help users send and receive files, browse web pages, and WiFi positioning through the WiFi module 540. It provides users with wireless broadband Internet access.
  • the electronic device may also include at least one sensor 550, such as a light sensor, a motion sensor, and other sensors.
  • the light sensor may include an ambient light sensor and a proximity sensor.
  • the motion sensor may include sensors such as a gravity acceleration sensor and a gyroscope; the electronic device may also include other sensors such as a barometer, a hygrometer, a thermometer, an infrared sensor, etc., which will not be repeated here.
  • the memory 560 may be used to store software programs and modules.
  • the processor 570 executes various functional applications and data processing by running the software programs and modules stored in the memory 560.
  • the memory 560 may mainly include a program storage area and a data storage area.
  • the program storage area may store an operating system, an application program required by at least one function (such as a sound playback function, an image playback function, etc.), etc.; Data (such as audio data, phone book, etc.) created by the use of electronic devices, etc.
  • the memory 560 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices.
  • the memory 560 may further include a memory controller to provide the processor 570 and the input unit 510 to access the memory 560.
  • the processor 570 is the control center of the electronic device. It uses various interfaces and lines to connect the various parts of the entire mobile phone. By running or executing software programs and/or modules stored in the memory 560, and calling data stored in the memory 560, Perform various functions of electronic equipment and process data to monitor the mobile phone as a whole.
  • the processor 570 may include one or more processing cores; preferably, the processor 570 may integrate an application processor and a modem processor, where the application processor mainly processes the operating system, user interface, and application programs, etc. , The modem processor mainly deals with wireless communication. It can be understood that the foregoing modem processor may not be integrated into the processor 570.
  • the electronic device also includes a power source 530 (such as a battery) for supplying power to various components.
  • a power source such as a battery
  • the power source may be logically connected to the processor 570 through a power management system, so that functions such as charging, discharging, and power management are realized through the power management system.
  • the power supply 530 may also include any components such as one or more DC or AC power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, and a power status indicator.
  • the electronic device may also include a camera, a Bluetooth module, etc., which will not be repeated here.
  • the processor 570 in the electronic device loads the executable file corresponding to the process of one or more application programs into the memory 560 according to the following instructions, and the processor 570 runs and stores the executable file The application program in the memory 560, thereby realizing various functions:
  • the shooting scene is photographed to obtain a photographed image.
  • an embodiment of the present application provides a storage medium in which multiple instructions are stored, and the instructions can be loaded by a processor to execute the steps in any photographing method provided in the embodiments of the present application.
  • the instruction can perform the following steps:
  • the shooting scene is photographed to obtain a photographed image.
  • the storage medium may include: read only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, etc.
  • the instructions stored in the storage medium can execute the steps in any photographing method provided in the embodiments of the present application, the beneficial effects that can be achieved by any photographing method provided in the embodiments of the present application can be achieved , Please refer to the previous embodiment for details, which will not be repeated here.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Human Computer Interaction (AREA)
  • Studio Devices (AREA)

Abstract

本申请公开一种拍照方法、装置、电子设备及存储介质。方法包括：获取拍摄场景的预览图像，调用关键点识别模型对预览图像检测得到拍摄主体的目标关键点；根据目标关键点确定当前构图类型，根据构图类型及目标关键点确定拍摄主体的定位点；根据定位点及构图类型确定拍摄主体的构图点；根据定位点与构图点对拍摄场景进行拍摄。

Description

拍照方法、装置、电子设备及存储介质
本申请要求于2020年03月09日提交中国专利局、申请号202010158602.5、发明名称为“拍照方法、装置、电子设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及图像处理技术领域，具体涉及一种拍照方法、装置、电子设备及存储介质。
背景技术
随着智能设备的迅速发展，越来越多的人开始使用智能设备进行拍照，在拍摄时，被拍摄者只能根据摄影师的经验来指导摆出姿势，并选取合适的角度进行拍摄。但是在绝大多数拍摄者没有摄影经验，拍摄出的照片并不是很具有美感，被拍摄者在没有摄影经验的情况下，也不能摆出合适的拍摄姿势。
发明内容
本申请实施例提供一种拍照方法、装置、电子设备及存储介质。能够对拍摄主体提供构图建议并拍照。
第一方面，本申请实施例提供了一种拍照方法，该方法包括：
获取拍摄场景的预览图像，并调用关键点识别模型对所述预览图像进行关键点检测，得到所述拍摄场景中拍摄主体的目标关键点；
根据所述目标关键点确定当前的构图类型，并根据所述构图类型以及所述目标关键点确定对应所述拍摄主体的定位点；
根据所述定位点以及所述构图类型确定对应所述拍摄主体的构图点；
当所述定位点与所述构图点不匹配时，输出用于指示调整电子设备拍摄姿态的提示信息；
当所述定位点与所述构图点匹配时，对所述拍摄场景进行拍摄，得到拍摄图像。
第二方面，本申请实施例提供了一种拍照装置，该装置包括：
获取模块，用于获取拍摄场景的预览图像，并调用关键点识别模型对所述预览图像进行关键点检测，得到所述拍摄场景中拍摄主体的目标关键点；
第一确定模块，用于根据所述目标关键点确定当前的构图类型，并根据所述构图类型以及所述目标关键点确定对应所述拍摄主体的定位点；
第二确定模块，用于根据所述定位点以及所述构图类型确定对应所述拍摄主体的构图点；
提示模块，用于当所述定位点与所述构图点不匹配时，输出用于指示调整电子设备拍摄姿态的提示信息；
拍照模块，用于当所述定位点与所述构图点匹配时，对所述拍摄场景进行拍摄，得到拍摄图像。
第三方面，本申请实施例提供的存储介质，其上存储有计算机程序，当所述计算机程序在计算机上运行时，使得所述计算机执行如本申请任一实施例提供的拍照方法。
第四方面，本申请实施例提供一种电子设备，包括存储器和处理器，所述存储器中存储有计算机程序，所述处理器通过调用所述存储器中存储的所述计算机程序，用于执行：
获取拍摄场景的预览图像，并调用关键点识别模型对所述预览图像进行关键点检测，得到所述拍摄场景中拍摄主体的目标关键点；
根据所述目标关键点确定当前的构图类型，并根据所述构图类型以及所述目标关键点确定对应所述拍摄主体的定位点；
根据所述定位点以及所述构图类型确定对应所述拍摄主体的构图点；
当所述定位点与所述构图点不匹配时，输出用于指示调整电子设备拍摄姿态的提示信息；
当所述定位点与所述构图点匹配时，对所述拍摄场景进行拍摄，得到拍摄图像。
附图说明
下面结合附图，通过对本申请的具体实施方式详细描述，将使本申请的技术方案及其它有益效果显而易见。
图1是本申请实施例提供的拍照方法的第一流程示意图。
图2是本申请实施例提供的拍照方法的第二流程示意图。
图3是本申请实施例提供的关键点识别模型的结构示意图。
图4是本申请实施例提供的第二识别子模型的第一结构示意图。
图5是本申请实施例提供的第二识别子模型的第二结构示意图。
图6是本申请实施例提供的第二识别子模型的第三结构示意图。
图7是本申请实施例提供的确定构图类型的流程示意。
图8是本申请实施例提供的构图三分线示意图。
图9是本申请实施例提供的构图信息示意图。
图10是本申请实施例提供的拍照装置的第一结构示意图。
图11是本申请实施例提供的拍照装置的第二结构示意图。
图12是本申请实施例提供的电子设备的结构示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
本文所使用的术语「模块」可看做为在该运算系统上执行的软件对象。本文不同模块、引擎及服务可看做为在该运算系统上的实施对象。
本申请实施例提供一种拍照方法，该拍照方法的执行主体可以是本申请实施例提供的拍照装置，或者集成了该拍照装置的电子设备。其中，该电子设备可以是智能手机、智能穿戴设备、平板电脑、掌上电脑(PDA，Personal Digital Assistant)等。以下分别进行详细说明。
请参阅图1，图1是本申请实施例提供的拍照方法的第一流程示意图。该拍照方法能够对拍摄主体提供构图建议并拍照。该拍照方法可以包括以下步骤：
101、获取拍摄场景的预览图像，并调用关键点识别模型对预览图像进行关键点检测，得到拍摄场景中拍摄主体的目标关键点。
可以理解的是，在拍照时，在电子设备的屏幕上会生成一个预览图像，以便拍摄者随时查看当前的画面信息。在用户拍照时，电子设备可以自动检测预览图像中是否有拍摄主体，在预览图像中有拍摄主体时，自动调用关键点识别模型对预览图像进行关键点识别。
需要说明的是，拍摄主体可以是人、动物、植物、人偶、公仔等多种可以拍摄的主体。关键点识别模型可以识别其中一种拍摄主体，也可以识别多种拍摄主体。
在一种实施例中，预览图像中可以包括一个或多个拍摄主体，该一个或多个拍摄主体都有对应的关键点。以拍摄主体为人偶为例，完整的人偶图像上包括多个关键点，比如头部、胸部、四肢、脖子、关节上都存在关键点。
在预览图像中，关键点识别模型识别出的关键点并不一定是拍摄主体上的关键点，也可能是其他拍摄对象的关键点，比如拍摄人像时路边上的建筑物、路人等都不是真正需要拍摄的拍摄对象。可以对关键点识别模型识别出来的关键点进行筛选，从而获得拍摄主体的目标关键点。
102、根据目标关键点确定当前的构图类型，并根据构图类型以及所述目标关键点确定对应拍摄主体的定位点。
拍摄主体的目标关键点处于拍摄主体的不同部位。比如，拍摄主体为人偶，人偶的头部、手部、腿部等多个部位均存在关键点。可以根据这些关键点来确定拍摄主体的构图类型。
在一些实施例中，可以先根据拍摄主体的目标关键点识别出拍摄主体的具体部位，比如，目标关键点是人偶头部上的关键点，则可以根据拍摄部位来确定当前构图类型为人像。如果根据目标关键点识 别出的拍摄部位有头部、手部、躯干、腿部、脚部，则说明当前的构图类型为全身像。
其中，可以根据目标关键点的数量或者目标关键点之间的比例来确定构图类型。若头部的目标关键点数量大于躯干上的目标关键点数量，则确认头部为主要拍摄部位。或者头部的目标关键点数量和躯干的目标关键点数量比值大于预设比例阈值，则确认头部为主要拍摄部位。则拍摄主体的构图类型为人像。
可以理解的是，还可以根据目标关键点的分布情况直接确定构图类型，比如，目标关键点对应的有人偶的头部、手部、躯干、腿部、脚部等部位，则根据目标关键点的分布情况可以确定拍摄主体的构图类型为全身像。需要说明的是，针对不同的拍摄主体，可以有不同的部位以及不同的构图类型，本申请实施例中的人偶为拍摄主体仅为例举。
在一些实施例中，在获取到拍摄主体的构图类型之后，还可以根据构图类型和目标关键点来确定拍摄主体的定位点。定位点可以是代表拍摄主体的点，定位点包含拍摄主体的位置信息。
比如，获取的构图类型为人像，则可以在目标关键点中选取面部相隔最远的两个目标关键点的中点为定位点。若构图类型为全身像，则可以获取拍摄主体的黄金分割线，然后在黄金分割线上选取拍摄主体的定位点。还可以直接根据目标关键点的分布情况，确定目标关键点分布的中心区域，在中心区域中选取离中心区域几何中心最近的目标关键点为定位点。
103、根据定位点以及构图类型确定对应拍摄主体的构图点。
可以理解的是，在确定好构图类型和定位点之后，可以生成一个构图点来引导拍摄者和被拍摄者进行拍照。
在一些实施例中，定位点在预览图像中具有详细的位置信息。比如，预览图像为矩形图像，对矩形图像建立直角坐标系，定位点在矩形图像中有具体的坐标位置信息，这个坐标位置信息可以代表着拍摄主体在预览图像中的位置信息。
在确定好构图类型时，例如当拍摄主体为人偶时，有全身像、半身像、人像等构图类型，可以根据构图类型和定位点来确定构图点。比如，首先根据定位点确定拍摄主体在预览图像中的具体位置，然后根据该位置和构图类型在预览图像中生成一个可见的构图点。例如，在获取到定位点位置后，可以根据构图类型选择构图数据库，构图数据库中有定位点位置和构图点位置的匹配信息，在确定定位点位置后，可以直接根据定位点位置在数据库中查找对应的构图点位置。
在一些实施方式中，在获取到定位点和构图类型之后，还可以直接通过预设算法来计算出构图点位置，然后将构图点位置展现在预览图像上。
可以理解的是，定位点对应的构图点位置可以是一个，也可以是多个。当定位点和任一构图点满足预设条件时，则表示拍摄主体构图成功。当定位点和构图点之间的欧式距离在预设距离范围内，则说明定位点和构图点已经匹配；若定位点与构图点之间的欧式距离不在预设距离范围内，则说明定位点和构图点不匹配。
104、当定位点与构图点不匹配时，输出用于指示调整电子设备拍摄姿态的提示信息。
在一些实施例中，可以将定位点和构图点进行匹配，当定位点与构图点不匹配时，在预览图像上显示有定位点和构图点的标记，并提示用户调整拍摄姿态。比如，在定位点和构图点之间设置有指示箭头，指示定位点需要调整的方向或/和距离，然后用户可以调整电子设备的拍摄姿态，或者调整拍摄主体的拍摄位置。
105、当定位点与构图点匹配时，对拍摄场景进行拍摄，得到拍摄图像。
在定位点和构图点匹配时，比如定位点和构图点之间的欧式距离在预设距离范围内，则说明此时拍摄主体符合构图条件，可以直接对拍摄场景进行拍摄，得到拍摄图像。
本申请实施例中，通过获取拍摄场景的预览图像，并调用关键点识别模型对预览图像进行关键点检测，得到拍摄场景中拍摄主体的目标关键点；然后根据目标关键点确定当前的构图类型，并根据构图类型以及目标关键点确定对应拍摄主体的定位点；最后根据定位点以及构图类型确定对应拍摄主体的构图点；当所述定位点与构图点不匹配时，输出用于指示调整电子设备拍摄姿态的提示信息；当定位点与 构图点匹配时，对拍摄场景进行拍摄，得到拍摄图像。
通过对拍摄主体的目标关键点进行识别，从而得到拍摄主体的定位点和对应的构图点，通过定位点和构图点来实现拍摄时的构图建议，并在定位点和构图点满足预设条件时自动拍照。
请继续参阅图2，图2是本申请实施例提供的拍照方法的第二流程示意图。该拍照方法能够对拍摄主体提供构图建议并拍照。该拍照方法可以包括以下步骤：
201、获取拍摄场景的预览图像，检测预览图像中的拍摄主体，生成拍摄主体对应的检测框。
在拍照时，在电子设备的相机界面会生成一个预览图像，预览图像根据电子设备的拍摄姿态改变而改变。
在一些实施例中，在捕获预览界面的时候，可以对预览界面进行识别，判断是否有拍摄主体，若存在拍摄主体的情况下，则对拍摄主体生成一个检测框。检测框中包含拍摄主体，检测框的形状可以是矩形、圆形、椭圆形等规则形状，也可以是不规则的形状，比如是拍摄主体的描边形状。
202、判断检测框占用预览图像的面积比例是否超过预设阈值。
在生成拍摄主体对应的检测框之后，可以计算检测框的面积，然后用检测框的面积除以预览画面的面积，得到测框占用预览图像的面积比例。然后再判断该面积比例是否超过预设阈值。
若该面积比例不超过预设阈值，则继续执行步骤202。
203、将预览图像输入至第一子模型，得到预览图像的特征图。
本申请实施例中，关键点识别模型包括第一子模型和第二子模型。具体请参阅图3，图3是本申请实施例提供的关键点识别模型的结构示意图。其中，在获取到预览图像之后，可以直接将预览图像输入至第一子模型进行识别，获取预览图像的特征图(feature map)。
在一些实施例中，第一子模型可以为mobilenet v2模型，mobilenet v2比较轻量化，在处理图片时速度较快。在电子设备性能较强，算力较强时，第一子模型还可以采用精度更高的模型，例如VGG19模型，resnet50模型等，使用这些模型提取特征能够提高人体关键点的检测精度。
204、将特征图输入至第二子模型，得到预览图像的目标连接特征和目标位置特征。
由图3可知，第二子模型包括多个，多个第二子模型依次连接。第一子模型和第一个第二子模型连接，即第一子模型与图中的第二子模型(1)连接。
在多个第二子模型中，每个第二子模型都能输出对应的位置特征和连接特征。其中位置特征可以是一个三维矩阵，可以是一个height*width*keypoints的三维矩阵，其中height代表图片的高度，width代表图片的宽度，keypints表示关键点的数量，其中该图片为每个连接特征对应的图片。具体的位置特征可以是热图(heatmap)。
连接特征可以是一个三维矩阵，可以是一个height*width*limbs的三维矩阵。其中height代表图片的高度，width代表图片的宽度，limbs表示连接体的数量。其中连接体可以是相关联的两个关键点之间的连接区域，比如，左眼和右眼的连接可以为一个连接体，连接体为左眼和右眼之间的连接区域。每一个连接体对应一个height*width*2的三维矩阵，可以认为连接特征是一个双通道的热图，该双通道的热图中的每个位置包括两个值，例如x值和y值，组成向量(x，y)，该向量可以表示对应位置的连接体方向，当x值和y值都为零时，则说明该位置没有肢体。
在一些实施例中，第一个第二子模型的输入为第一子模型输出的特征图。第一个第二子模型对特征图进行处理，得到第一个第二子模型输出的位置特征和连接特征。
请一并参阅图4，图4是本申请实施例提供的第二子模型的第一结构示意图。其中第一个第二子模型对特征图处理后，输出第一个第二子模型输出的位置特征和连接特征。也就是第二子模型(1)对特征图进行处理得到位置特征(1)和连接特征(1)。
将位置特征(1)、连接特征(1)以及特征图输入到第二个第二子模型中，即输入至第二子模型(2)中，得到第二子模型(2)输出的位置特征(2)和连接特征(2)。
再将位置特征(2)、连接特征(2)以及特征图输入到第三个第二子模型中，即输入至第二子模型(3)中，得到第二子模型(3)输出的位置特征(3)和连接特征(3)。依此类推，除去第一个第二子 模型的剩余第二子模型中，都是将上一个第二子模型输出的连接特征、位置特征以及第一子模型输出的特征图作为输入，每一个第二子模型输出与其对应的连接特征和位置特征。直至最后一个第二子模型输出目标连接特征和目标位置特征为止。
请一并参阅图5，图5是本申请实施例提供的第二子模型的第二结构示意图。具体的，图5展示的是第一个第二子模型的结构示意图。其中将第一子模型输出的特征图作为输入，输入至第一个第二子模型中。第二子模型包括连接模块和位置模块，连接模块和位置模块均为多个不同类型的卷积层组成的模块。
比如，在连接模块中包括多个第一卷积层和多个第二卷积层，多个第一卷积层依次连接，多个第二卷积层依次连接，最后一个第一卷积层与第一个第二卷积层连接。在位置模块中包括多个第一卷积层和多个第二卷积层，多个第一卷积层依次连接，多个第二卷积层依次连接，最后一个第一卷积层与第一个第二卷积层连接。
在一些实施方式中，第一卷积层可以是3*3的卷积层，第二卷积层可以为1*1的卷积层。在连接模块中第一卷积层可以为三个，第二卷积层为两个，位置模块的结构可以与连接模块的结构相同。在实际的应用中，第一卷积层和第二卷积层的类型、数量都可以根据实际要求发生改变。
由图5可知，将第一子模型输出的特征图输入至第一个第二子模型中，由于第二子模型中存在连接模块和位置模块，连接模块和位置模块可以分别对特征图进行处理，从而得到连接特征(1)和位置特征(1)。
请继续参阅图6，图6是本申请实施例提供的第二子模型的第三结构示意图。具体的，图6所示的是出第一个第二子模型的剩余第二子模型的结构示意图。其中每个第二子模型都包括连接模块和位置模块，连接模块和位置模块中包括多个不同类型的卷积层。
比如，连接模块中包括多个第三卷积层和多个第二卷积层，多个第三卷积层依次连接，多个第二卷积依次连接，最后一个第三卷积层和第一个第二卷积层连接。位置模块中包括五个第三卷积层和两个第二卷积层，多个第三卷积层依次连接，多个第二卷积依次连接，最后一个第三卷积层和第一个第二卷积层连接。
在一些实施方式中，第三卷积层可以是7*7的卷积层，第二卷积层可以为1*1的卷积层。在连接模块中第三卷积层可以为五个，第二卷积层为两个，位置模块的结构可以与连接模块的结构相同。在实际的应用中，第一卷积层和第二卷积层的类型、数量都可以根据实际要求发生改变。
由图6可知，除第一个第二子模型的剩余第二子模型中，每一个第二子模型的输入为上一个第二子模型输出的连接特征、位置特征以及第一子模型输出的特征图，即连接特征(M-1)和位置特征(M-1)。每一个第二子模型可以输出与其对应的连接特征和位置特征，即连接特征(M)和位置特征(M)。需要说明的是，最后一个第二子模型输出的为目标位置特征和目标连接特征。
在一些实施例中，在除了第一个第二子模型之外的剩余第二子模型中，可以将第三卷积层替换为第一卷积层，从而减少计算量和参数，使得第二子模型处理任务更快。
205、根据目标位置特征确定关键点中的候选关键点。
在一些实施例中，可以选取目标位置特征中的最大值的位置为候选关键点，例如在热图(heatmap)中选取像素点数值最大的为候选关键点。在实际应用中，可以对热图进行最大池化，然后将池化前和池化后的热图对比，将池化前和池化后的热图中取值相等的位置作为候选关键点。
206、根据目标连接特征和候选关键点确定拍摄主体的目标关键点。
可以理解的是，在获取到候选关键点之后，能够根据目标连接特征中的连接体的方向来对候选关键点进行连接，以得到完整的个体。
在一些实施例中，可以每次获取一个连接体对应的目标连接特征，连接着个连接体两端的候选关键点。从而求出两个候选关键点来自同一个体的置信度，具体可以用以下置信度公式表示：
Figure PCTCN2021074205-appb-000001
其中
Figure PCTCN2021074205-appb-000002
可以为表示两个不同候选关键点，P(u)为两个候选关键点之间内插的位置，L c为在目 标连接特征中P(u)处的值，P(u)的具体公式为：
Figure PCTCN2021074205-appb-000003
可以理解的是，在实际应用中，会在两个候选关键点之间取多个位置，比如在区间[0,1]上均匀间隔采样得到u，近似求积分。
在预览图像上只有一个拍摄主体时，则可以确定所有的候选关键点都是来自同一拍摄个体的，当所有的候选关键点连接时，就可以表示一个完整的拍摄主体。
在预览图像上有多个拍摄个体时，可以生成一个关键点关联集，关联集中有每一个个体的候选关键点集合。比如在候选关键点上有眼睛对应的候选关键点，鼻子对应的候选关键点等等。其中每个拍摄个体中，眼睛和鼻子两者有对应的候选关键点，手腕和手肘两者有对应的关键点，多个候选关键点可以组成一个代表拍摄主体的个体。多个候选关键点集合形成一个关联集。可以在其中寻找到一个最佳的关联集，比如：
Figure PCTCN2021074205-appb-000004
关联集Z中，j 1,j 2表示关键点类别(眼睛、鼻子、手腕等)，m和n表示对应关键点类别内的关键点编号。利用上述置信度公式使得：
Figure PCTCN2021074205-appb-000005
E c为所有已连接的连接体的总置信度，即多个连接体连接形成的个体的总置信度。在匹配的过程中可以使用匈牙利算法来匹配，得到最佳的关联集。
需要说明的是，当
Figure PCTCN2021074205-appb-000006
为1的时候，表示候选关键点
Figure PCTCN2021074205-appb-000007
来自同一个体，也就是候选关键点
Figure PCTCN2021074205-appb-000008
为同一个拍摄主体上的关键点。
通过置信度来确定候选关键点之间的关联度，从而确定拍摄主体上的目标关键点。即置信度越高的时候，候选关键点之间的关联度越高，越有可能来自同一个个体。
207、根据目标关键点确定当前的构图类型。
在获取到目标关键点之后，可以对目标关键点进行识别，例如目标关键点是否主要集中在头部，或者脚部、胸部、全身、局部身体等等。
在一些实施例中，可以先根据拍摄主体的目标关键点识别出拍摄主体的具体部位，比如，目标关键点是人偶头部上的关键点，则可以根据拍摄部位来确定当前构图类型为人像。如果根据目标关键点识别出的拍摄部位有头部、手部、躯干、腿部、脚部，则说明当前的构图类型为全身像。
具体请参阅图7，图7是本申请实施例提供的确定构图类型的流程示意图。具体的，可以根据目标关键点的属性来确定构图类型。比如：
301、获取目标关键点。
302、判断目标关键点是否包含头部关键点。若不包含头部关键点，则进入步骤303，说明是局部身体特写。
若包含头部关键点，则进入步骤304，检测是否只包含头部关键点。若只包含头部关键点，则进入步骤305，说明是面部特写。
若不只是包含头部关键点，则进入步骤306检测是否包括脚部关键点。若包含脚部关键点，则进入步骤307，说明是全身特写。
若不包含脚部关键点，则进入步骤308，判断是否包含髋关节关键点。
若包含髋关节关键点，则进入步骤309，说明是胸部特写
若不包含髋关节关键点，则进入步骤310，说明是七分身特写。
208、根据构图类型确定构图类型对应的预设定位点选取方式。
在一些实施例中，构图类型对应着不同的预设定位点选取方式。比如，如果是局部身体特写，取检测框的中心点作为定位点。可以理解的是，不同的构图类型对应着不同的预设定位点选取方式。
209、根据预设定位点选取方式以及目标关键点的位置信息确定定位点。
当确定好预设定位点的选取方式后，根据关键点的坐标信息可以确定对应的定位点。比如，预设定位点的选取方式为面部特写对应的选取方式，则获取目标关键点的坐标信息，如果双眼、鼻子与嘴部关键点的横坐标均值在横向方向的标注框1/4长度范围内，则认为是在拍摄侧脸，取检测框的中心作为定位点。
210、根据定位点以及构图类型确定对应拍摄主体的构图点。
在一些实施例中，可以将构图类型分为两大类，一类是只包含头部的面部特写，一种是包括身体的身体特写。身体特写中既可以包括头部，也可以不包括头部。在获取到构图类型之后，可以确定构图类型是面部特写还是身体特写。
获取电子设备的横竖屏信息，在预览图像界面生成有三分线。具体请参阅，图8是本申请实施例提供的构图三分线示意图。其中A、B两条三分线连接的是边长较长的预览图像边长，C、D两条三分线连接的是边长较短的预览图像边长。如图8所示，当C、D为横线时，预览图像为横屏；当C、D为竖线时，预览图像为竖屏。
若构图类型为面部特写，则可以获取预览图像的横竖屏信息。当预览图像为横屏时，则拍摄主体对应的候选构图点可以选取为预览图像的正中心、三分线C的中点、三分线C与三分线A、B的交点。当预览图像为竖屏时，则候选构图点可以为预览图像的中心、三分线A的中点。
若构图类型为身体特写时，则可以获取预览图像的横竖屏信息。当预览图像为横屏时，则拍摄主体对应的候选构图点可以选取为预览图像的正中心、三分线C的中点。当预览图像为竖屏时，则候选构图点选取三分线A、B、C、D的交点。
在获取到候选构图点之后，可以在候选构图点中选取与定位点最近的候选构图点为最终的构图点。也就是拍摄主体的构图点。
211、判断定位点和构图点是否匹配。
判断定位点和构图点是否匹配的方式有多种。比如，判断构图点和定位点之间的欧式距离是否小于预设距离阈值，或者判断构图点和定位点之间的欧式距离是否在预设范围内。
212、若定位点和构图点不匹配，则输出用于指示调整电子设备拍摄姿态的提示信息。
当构图点和定位点之间的欧式距离在不小于预设距离阈值或者不在预设范围内，定位点和构图点不匹配，此时可以在屏幕上生成一个提示信息，提示用户调整电子设备的拍摄姿态。
比如，如图9所示，图9是本申请实施例提供的构图信息示意图。在定位点和构图点不匹配时，可生成一个定位点朝向构图点方向的指示箭头，来提示用户调整电子设备的拍摄姿态来重新构图。
213、若定位点和构图点匹配，则对拍摄场景进行拍摄，得到拍摄图像。
当构图点和定位点之间的欧式距离在小于预设距离阈值或者在预设范围内，则说明定位点和构图点匹配，此时，预览图像上的提示信息可以消失，电子设备自动对拍摄主体进行拍照。或者提示信息改变颜色、生成文字提示用户可以拍照，用户可以根据提示信息手动拍照。
综上所述，本申请实施例中，在拍照时检测到拍摄主体时，可以根据关键点识别模型自动识别预览图像的关键点，并对预览图像的关键点进行处理得到拍摄主体的目标关键点。最终根据目标关键点确定构图类型，根据构图类型和目标关键点确定定位点，根据定位点以及构图类型确定对应拍摄主体的构图点。当定位点与构图点不匹配时，输出用于指示调整电子设备拍摄姿态的提示信息；当定位点与构图点匹配时，对拍摄场景进行拍摄，得到拍摄图像。从而生成对拍摄主体的构图建议并拍照。
请继续参阅图10，图10是本申请实施例提供的拍照装置的第一结构示意图。其中该拍照装置包括获取模块410、第一确定模块420、第二确定模块430、提示模块440、拍照模块450。
获取模块410，用于获取拍摄场景的预览图像，并调用关键点识别模型对所述预览图像进行关键点检测，得到所述拍摄场景中拍摄主体的目标关键点。
可以理解的是，在拍照时，在电子设备的屏幕上会生成一个预览图像，以便拍摄者随时查看当前的画面信息。在用户拍照时，获取模块410可以自动检测预览图像中是否有拍摄主体，在预览图像中有 拍摄主体时，自动调用关键点识别模型对预览图像进行关键点识别。
在预览图像中，关键点识别模型识别出的关键点并不一定是拍摄主体上的关键点，也可能是其他拍摄对象的关键点，比如拍摄人像时路边上的建筑物、路人等都不是真正需要拍摄的拍摄对象。可以对关键点识别模型识别出来的关键点进行筛选，从而获得拍摄主体的目标关键点。
请一并参阅图11，图11是本申请实施例提供的拍照装置的第二结构示意图。获取模块410包括第一确定子模块411和第二确定子模块412。
第一确定子模块411，用于根据所述目标位置特征确定所述关键点中的候选关键点。
在一些实施例中，第一确定子模块411可以选取目标位置特征中的最大值的位置为候选关键点，例如在热图(heatmap)中选取像素点数值最大的为候选关键点。在实际应用中，可以对热图进行最大池化，然后将池化前和池化后的热图对比，将池化前和池化后的热图中取值相等的位置作为候选关键点。
第二确定子模块412，用于根据所述目标连接特征和所述候选关键点确定所述拍摄主体的所述目标关键点。
可以理解的是，在获取到候选关键点之后，能够根据目标连接特征中的连接体的方向来对候选关键点进行连接，以得到完整的个体。
在一些实施例中，可以获取关键点之间的置信度，在置信度越高的情况下，则关键点之间的关联度越高，从而确定出拍摄主体的目标关键点。
第一确定模块420，用于根据所述目标关键点确定当前的构图类型，并根据所述构图类型以及所述目标关键点确定对应所述拍摄主体的定位点。
拍摄主体的目标关键点处于拍摄主体的不同部位。比如，拍摄主体为人偶，人偶的头部、手部、腿部等多个部位均存在关键点。可以根据这些关键点来确定拍摄主体的构图类型。
在一些实施例中，可以先根据拍摄主体的目标关键点识别出拍摄主体的具体部位，比如，目标关键点是人偶头部上的关键点，则可以根据拍摄部位来确定当前构图类型为人像。如果根据目标关键点识别出的拍摄部位有头部、手部、躯干、腿部、脚部，则说明当前的构图类型为全身像。
第二确定模块430，用于根据所述定位点以及所述构图类型确定对应所述拍摄主体的构图点。
可以理解的是，在确定好构图类型和定位点之后，可以生成一个构图点来引导拍摄者和被拍摄者进行拍照。
在一些实施例中，定位点在预览图像中具有详细的位置信息。比如，预览图像为矩形图像，对矩形图像建立直角坐标系，定位点在矩形图像中有具体的坐标位置信息，这个坐标位置信息可以代表着拍摄主体在预览图像中的位置信息。
在确定好构图类型时，例如当拍摄主体为人偶时，有全身像、半身像、人像等构图类型，可以根据构图类型和定位点来确定构图点。比如，首先根据定位点确定拍摄主体在预览图像中的具体位置，然后根据该位置和构图类型在预览图像中生成一个可见的构图点。例如，在获取到定位点位置后，可以根据构图类型选择构图数据库，构图数据库中有定位点位置和构图点位置的匹配信息，在确定定位点位置后，可以直接根据定位点位置在数据库中查找对应的构图点位置。
如图11所示，第二确定模块430还包括获取子模块431、第三确定子模块432和选取子模块433。
获取子模块431，用于获取所述预览图像的横竖屏信息。
获取电子设备的横竖屏信息，在预览图像界面生成有三分线。具体请参阅，图8是本申请实施例提供的构图三分线示意图。其中A、B两条三分线连接的是边长较长的预览图像边长，C、D两条三分线连接的是边长较短的预览图像边长。如图8所示，当C、D为横线时，预览图像为横屏；当C、D为竖线时，预览图像为竖屏。
第三确定子模块432，用于根据所述预览图像的横竖屏信息、所述定位点及所述构图类型确定对应所述拍摄主体对应的多个候选构图点。
例如若构图类型为面部特写，则可以获取预览图像的横竖屏信息。当预览图像为横屏时，则拍摄主体对应的候选构图点可以选取为预览图像的正中心、三分线C的中点、三分线C与三分线A、B的交点。 当预览图像为竖屏时，则候选构图点可以为预览图像的中心、三分线A的中点。
若构图类型为身体特写时，则可以获取预览图像的横竖屏信息。当预览图像为横屏时，则拍摄主体对应的候选构图点可以选取为预览图像的正中心、三分线C的中点。当预览图像为竖屏时，则候选构图点选取三分线A、B、C、D的交点。
选取子模块433，用于在所述多个候选构图点中选取距离所述定位点最近的候选构图点为所述构图点。
提示模块440，用于当所述定位点与所述构图点不匹配时，输出用于指示调整电子设备拍摄姿态的提示信息。
在一些实施例中，可以将定位点和构图点进行匹配，当定位点与构图点不匹配时，在预览图像上显示有定位点和构图点的标记，并提示用户调整拍摄姿态。比如，在定位点和构图点之间设置有指示箭头，指示定位点需要调整的方向或/和距离，然后用户可以调整电子设备的拍摄姿态，或者调整拍摄主体的拍摄位置。
拍照模块450，用于当所述定位点与所述构图点匹配时，对所述拍摄场景进行拍摄，得到拍摄图像。
在定位点和构图点匹配时，比如定位点和构图点之间的欧式距离在预设距离范围内，则说明此时拍摄主体符合构图条件，可以直接对拍摄场景进行拍摄，得到拍摄图像。
综上所述，本申请实施例中，获取拍摄场景的预览图像，并调用关键点识别模型对预览图像进行关键点检测，得到拍摄场景中拍摄主体的目标关键点；根据目标关键点确定当前的构图类型，并根据构图类型以及目标关键点确定对应拍摄主体的定位点；根据定位点以及构图类型确定对应拍摄主体的构图点；当定位点与构图点不匹配时，输出用于指示调整电子设备拍摄姿态的提示信息；当定位点与构图点匹配时，对拍摄场景进行拍摄，得到拍摄图像。从而实现在拍照时提出构图建议并拍照。
相应的，本申请实施例还提供一种电子设备，如图12所示，图12是本身实施例提供的电子设备的结构示意图。该电子设备可以包括、包括有一个或一个以上计算机可读存储介质的输入单元510、显示单元520、电源530、WIFI模块540、传感器550、存储器560以及包括有一个或者一个以上处理核心的处理器570等部件。本领域技术人员可以理解，图12中示出的电子设备结构并不构成对电子设备的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。其中：
输入单元510可用于接收输入的数字或字符信息，以及产生与用户设置以及功能控制有关的键盘、鼠标、操作杆、光学或者轨迹球信号输入。可选的，触敏表面可包括触摸检测装置和触摸控制器两个部分。其中，触摸检测装置检测用户的触摸方位，并检测触摸操作带来的信号，将信号传送给触摸控制器；触摸控制器从触摸检测装置上接收触摸信息，并将它转换成触点坐标，再送给处理器570，并能接收处理器570发来的命令并加以执行。此外，可以采用电阻式、电容式、红外线以及表面声波等多种类型实现触敏表面。除了触敏表面，输入单元510还可以包括其他输入设备。
显示单元520可包括显示面板，可选的，可以采用液晶显示器(LCD，Liquid Crystal Display)、有机发光二极管(OLED，Organic Light-Emitting Diode)等形式来配置显示面板。进一步的，触敏表面可覆盖显示面板，当触敏表面检测到在其上或附近的触摸操作后，传送给处理器570以确定触摸事件的类型，随后处理器570根据触摸事件的类型在显示面板上提供相应的视觉输出。虽然在图12中，触敏表面与显示面板是作为两个独立的部件来实现输入和输入功能，但是在某些实施例中，可以将触敏表面与显示面板集成而实现输入和输出功能。
WiFi属于短距离无线传输技术，电子设备通过WiFi模块540可以帮助用户收发文件、浏览网页和WiFi定位等，它为用户提供了无线的宽带互联网访问。
电子设备还可包括至少一种传感器550，比如光传感器、运动传感器以及其他传感器。具体地，光传感器可包括环境光传感器及接近传感器。运动传感器可包括重力加速度传感器、陀螺仪等传感器；电子设备还可以包括气压计、湿度计、温度计、红外线传感器等其他传感器，在此不再赘述。
存储器560可用于存储软件程序以及模块，处理器570通过运行存储在存储器560的软件程序以及模块，从而执行各种功能应用以及数据处理。存储器560可主要包括存储程序区和存储数据区，其中， 存储程序区可存储操作系统、至少一个功能所需的应用程序(比如声音播放功能、图像播放功能等)等；存储数据区可存储根据电子设备的使用所创建的数据(比如音频数据、电话本等)等。此外，存储器560可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、或其他易失性固态存储器件。相应地，存储器560还可以包括存储器控制器，以提供处理器570和输入单元510对存储器560的访问。
处理器570是电子设备的控制中心，利用各种接口和线路连接整个手机的各个部分，通过运行或执行存储在存储器560内的软件程序和/或模块，以及调用存储在存储器560内的数据，执行电子设备的各种功能和处理数据，从而对手机进行整体监控。可选的，处理器570可包括一个或多个处理核心；优选的，处理器570可集成应用处理器和调制解调处理器，其中，应用处理器主要处理操作系统、用户界面和应用程序等，调制解调处理器主要处理无线通信。可以理解的是，上述调制解调处理器也可以不集成到处理器570中。
电子设备还包括给各个部件供电的电源530(比如电池)，优选的，电源可以通过电源管理系统与处理器570逻辑相连，从而通过电源管理系统实现管理充电、放电、以及功耗管理等功能。电源530还可以包括一个或一个以上的直流或交流电源、再充电系统、电源故障检测电路、电源转换器或者逆变器、电源状态指示器等任意组件。
尽管未示出，电子设备还可以包括摄像头、蓝牙模块等，在此不再赘述。具体在本实施例中，电子设备中的处理器570会按照如下的指令，将一个或一个以上的应用程序的进程对应的可执行文件加载到存储器560中，并由处理器570来运行存储在存储器560中的应用程序，从而实现各种功能：
获取拍摄场景的预览图像，并调用关键点识别模型对所述预览图像进行关键点检测，得到所述拍摄场景中拍摄主体的目标关键点；
根据所述目标关键点确定当前的构图类型，并根据所述构图类型以及所述目标关键点确定对应所述拍摄主体的定位点；
根据所述定位点以及所述构图类型确定对应所述拍摄主体的构图点；
当所述定位点与所述构图点不匹配时，输出用于指示调整电子设备拍摄姿态的提示信息；
当所述定位点与所述构图点匹配时，对所述拍摄场景进行拍摄，得到拍摄图像。
本领域普通技术人员可以理解，上述实施例的各种方法中的全部或部分步骤可以通过指令来完成，或通过指令控制相关的硬件来完成，该指令可以存储于一计算机可读存储介质中，并由处理器进行加载和执行。
为此，本申请实施例提供一种存储介质，其中存储有多条指令，该指令能够被处理器进行加载，以执行本申请实施例所提供的任一种拍照方法中的步骤。例如，该指令可以执行如下步骤：
获取拍摄场景的预览图像，并调用关键点识别模型对所述预览图像进行关键点检测，得到所述拍摄场景中拍摄主体的目标关键点；
根据所述目标关键点确定当前的构图类型，并根据所述构图类型以及所述目标关键点确定对应所述拍摄主体的定位点；
根据所述定位点以及所述构图类型确定对应所述拍摄主体的构图点；
当所述定位点与所述构图点不匹配时，输出用于指示调整电子设备拍摄姿态的提示信息；
当所述定位点与所述构图点匹配时，对所述拍摄场景进行拍摄，得到拍摄图像。
以上各个操作的具体实施可参见前面的实施例，在此不再赘述。
其中，该存储介质可以包括：只读存储器(ROM，Read Only Memory)、随机存取记忆体(RAM，Random Access Memory)、磁盘或光盘等。
由于该存储介质中所存储的指令，可以执行本申请实施例所提供的任一种拍照方法中的步骤，因此，可以实现本申请实施例所提供的任一种拍照方法所能实现的有益效果，详见前面的实施例，在此不再赘述。
以上对本申请实施例所提供的一种拍照方法、装置、存储介质及电子设备进行了详细介绍，本文 中应用了具体个例对本申请的原理及实施方式进行了阐述，以上实施例的说明只是用于帮助理解本申请的方法及其核心思想；同时，对于本领域的技术人员，依据本申请的思想，在具体实施方式及应用范围上均会有改变之处，综上所述，本说明书内容不应理解为对本申请的限制。

Claims (20)

  1. 一种拍照方法，其中，所述方法包括：
    获取拍摄场景的预览图像，并调用关键点识别模型对所述预览图像进行关键点检测，得到所述拍摄场景中拍摄主体的目标关键点；
    根据所述目标关键点确定当前的构图类型，并根据所述构图类型以及所述目标关键点确定对应所述拍摄主体的定位点；
    根据所述定位点以及所述构图类型确定对应所述拍摄主体的构图点；
    当所述定位点与所述构图点不匹配时，输出用于指示调整电子设备拍摄姿态的提示信息；
    当所述定位点与所述构图点匹配时，对所述拍摄场景进行拍摄，得到拍摄图像。
  2. 根据权利要求1所述的拍照方法，其中，所述关键点识别模型包括：第一子模型和第二子模型；
    所述调用关键点识别模型对所述预览图像进行关键点检测，包括：
    将所述预览图像输入至所述第一子模型，得到所述预览图像的特征图；
    将所述特征图输入至所述第二子模型，得到所述预览图像的目标连接特征和目标位置特征；
    根据所述目标连接特征和所述目标位置特征对所述预览图像进行关键点检测。
  3. 根据权利要求2所述的拍照方法，其中，所述关键点识别模型包括多个所述第二子模型，多个所述第二子模型依次连接，所述第一子模型与第一个所述第二子模型连接；
    所述将所述特征图输入至所述第二子模型，得到所述预览图像的目标连接特征和目标位置特征，包括：
    将所述特征图输入至第一个所述第二子模型中，得到第一个所述第二子模型输出的连接特征和位置特征；
    除第一个所述第二子模型的剩余所述第二子模型中，将所述特征图及上一个所述第二子模型输出的连接特征和位置特征输入至下一个所述第二子模型中，得到下一个所述第二子模型输出的连接特征和位置特征，直至获取到最后一个所述第二子模型输出的所述目标连接特征和所述目标位置特征为止。
  4. 根据权利要求3所述的拍照方法，其中，所述第二子模型包括：连接模块和位置模块；
    所述第一个第二子模型的连接模块包括多个第一卷积层和多个第二卷积层，多个所述第一卷积层依次连接，多个所述第二卷积层依次连接，最后一个所述第一卷积层与第一个所述第二卷积层连接，最后一层所述第二卷积层输出所述第一个第二子模型的连接特征；
    所述第一个第二子模型的位置模块包括多个第一卷积层和多个第二卷积层，多个所述第一卷积层依次连接，多个所述第二卷积层依次连接，最后一个所述第一卷积层与第一个所述第二卷积层连接，最后一层所述第二卷积层输出所述第一个第二子模型的位置特征。
  5. 根据权利要求3所述的拍照方法，其中，所述第二子模型包括：连接模块和位置模块；
    所述剩余第二子模型的连接模块包括多个第三卷积层和多个第二卷积层，所述多个第三卷积层依次连接，所述多个第二卷积层依次连接，最后一个所述第三卷积层与第一个所述第二卷积层依次连接，最后一个所述第二卷积层输出所述剩余第二子模型的连接特征；
    所述剩余第二子模型的位置模块包括多个第三卷积层和多个第二卷积层，所述多个第三卷积层依次连接，所述多个第二卷积层依次连接，最后一个所述第三卷积层与第一个所述第二卷积层依次连接，最后一个所述第二卷积层输出所述剩余第二子模型的位置特征。
  6. 根据权利要求2所述的拍照方法，其中，所述调用关键点识别模型对所述预览图像进行关键点检测，得到所述拍摄场景中拍摄主体的目标关键点，包括：
    根据所述目标位置特征确定所述关键点中的候选关键点；
    根据所述目标连接特征和所述候选关键点确定所述拍摄主体的所述目标关键点。
  7. 根据权利要求6所述的拍照方法，其中，所述根据所述目标位置特征确定所述关键点中的候选关键点，包括：
    确定所述目标位置特征中的最大值的位置；
    将所述最大值的位置作为所述候选关键点。
  8. 根据权利要求1-5任一项所述的拍照方法，其中，所述根据所述构图类型以及所述目标关键点确定对应所述拍摄主体的定位点，包括：
    根据所述构图类型确定所述构图类型对应的预设定位点选取方式；
    根据所述预设定位点选取方式以及所述目标关键点的位置信息确定所述定位点。
  9. 根据权利要求1-5任一项所述的拍照方法，其中，在所述根据所述定位点以及所述构图类型确定对应所述拍摄主体的构图点之前，所述方法还包括：
    获取所述预览图像的横竖屏信息；
    所述根据所述定位点以及所述构图类型确定对应所述拍摄主体的构图点，包括：
    根据所述预览图像的横竖屏信息、所述定位点及所述构图类型确定对应所述拍摄主体对应的多个候选构图点；
    在所述多个候选构图点中选取距离所述定位点最近的候选构图点为所述构图点。
  10. 根据权利要求1-5任一项所述的拍照方法，其中，在所述调用关键点识别模型对所述预览图像进行关键点检测，得到所述拍摄场景中拍摄主体的目标关键点之前，所述方法还包括：
    生成所述拍摄主体对应的检测框，所述检测框内包括所述拍摄主体；
    判断所述检测框占用所述预览图像的面积比例是否超过预设阈值；
    若超过，则调用关键点识别模型对所述预览图像进行关键点检测，得到所述拍摄场景中拍摄主体的目标关键点。
  11. 根据权利要求1-5任一项所述的拍照方法，其中，所述当所述定位点与所述构图点匹配时，对所述拍摄场景进行拍摄，得到拍摄图像，包括：
    获取所述定位点和所述构图点之间的距离；
    当所述定位点和所述构图点之间的距离小于预设距离阈值时，所述定位点与所述构图点匹配，对所述拍摄场景进行拍摄，得到拍摄图像。
  12. 一种拍照装置，其中，所述装置包括：
    获取模块，用于获取拍摄场景的预览图像，并调用关键点识别模型对所述预览图像进行关键点检测，得到所述拍摄场景中拍摄主体的目标关键点；
    第一确定模块，用于根据所述目标关键点确定当前的构图类型，并根据所述构图类型以及所述目标关键点确定对应所述拍摄主体的定位点；
    第二确定模块，用于根据所述定位点以及所述构图类型确定对应所述拍摄主体的构图点；
    提示模块，用于当所述定位点与所述构图点不匹配时，输出用于指示调整电子设备拍摄姿态的提示信息；
    拍照模块，用于当所述定位点与所述构图点匹配时，对所述拍摄场景进行拍摄，得到拍摄图像。
  13. 根据权利要求12所述的拍照装置，其中，所述获取模块包括：
    第一确定子模块，用于根据所述目标位置特征确定所述关键点中的候选关键点；
    第二确定子模块，用于根据所述目标连接特征和所述候选关键点确定所述拍摄主体的所述目标关键点。
  14. 根据权利要求12所述的拍照装置，其中，所述第二确定模块包括：
    获取子模块，用于获取所述预览图像的横竖屏信息；
    第三确定子模块，用于根据所述预览图像的横竖屏信息、所述定位点及所述构图类型确定对应所述拍摄主体对应的多个候选构图点；
    选取子模块，用于在所述多个候选构图点中选取距离所述定位点最近的候选构图点为所述构图点。
  15. 一种电子设备，其中，所述电子设备包括处理器和存储器，所述存储器中存储有计算机程序，所述处理器通过调用所述存储器中存储的所述计算机程序，用于执行：
    获取拍摄场景的预览图像，并调用关键点识别模型对所述预览图像进行关键点检测，得到所述拍摄场景中拍摄主体的目标关键点；
    根据所述目标关键点确定当前的构图类型，并根据所述构图类型以及所述目标关键点确定对应所述拍摄主体的定位点；
    根据所述定位点以及所述构图类型确定对应所述拍摄主体的构图点；
    当所述定位点与所述构图点不匹配时，输出用于指示调整电子设备拍摄姿态的提示信息；
    当所述定位点与所述构图点匹配时，对所述拍摄场景进行拍摄，得到拍摄图像。
  16. 根据权利要求15所述的电子设备，其中，所述关键点识别模型包括第一子模型和第二子模型，所述处理器用于执行：
    将所述预览图像输入至所述第一子模型，得到所述预览图像的特征图；
    将所述特征图输入至所述第二子模型，得到所述预览图像的目标连接特征和目标位置特征；
    根据所述目标连接特征和所述目标位置特征对所述预览图像进行关键点检测。
  17. 根据权利要求16所述的电子设备，其中，所述关键点识别模型包括多个所述第二子模型，多个所述第二子模型依次连接，所述第一子模型与第一个所述第二子模型连接，所述处理器用于执行：
    将所述特征图输入至第一个所述第二子模型中，得到第一个所述第二子模型输出的连接特征和位置特征；
    除第一个所述第二子模型的剩余所述第二子模型中，将所述特征图及上一个所述第二子模型输出的连接特征和位置特征输入至下一个所述第二子模型中，得到下一个所述第二子模型输出的连接特征和位置特征，直至获取到最后一个所述第二子模型输出的所述目标连接特征和所述目标位置特征为止。
  18. 根据权利要求16所述的电子设备，其中，所述处理器用于执行：
    根据所述目标位置特征确定所述关键点中的候选关键点；
    根据所述目标连接特征和所述候选关键点确定所述拍摄主体的所述目标关键点。
  19. 根据权利要求18所述的电子设备，其中，所述处理器用于执行：
    确定所述目标位置特征中的最大值的位置；
    将所述最大值的位置作为所述候选关键点。
  20. 一种存储介质，其中，所述存储介质中存储有计算机程序，当所述计算机程序在计算机上运行时，使得所述计算机执行权利要求1-11任一项所述的拍照方法中的步骤。
PCT/CN2021/074205 2020-03-09 2021-01-28 拍照方法、装置、电子设备及存储介质 WO2021179831A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010158602.5A CN111343382B (zh) 2020-03-09 2020-03-09 拍照方法、装置、电子设备及存储介质
CN202010158602.5 2020-03-09

Publications (1)

Publication Number Publication Date
WO2021179831A1 true WO2021179831A1 (zh) 2021-09-16

Family

ID=71187962

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/074205 WO2021179831A1 (zh) 2020-03-09 2021-01-28 拍照方法、装置、电子设备及存储介质

Country Status (2)

Country Link
CN (1) CN111343382B (zh)
WO (1) WO2021179831A1 (zh)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111343382B (zh) * 2020-03-09 2021-09-10 Oppo广东移动通信有限公司 拍照方法、装置、电子设备及存储介质
CN111953907B (zh) * 2020-08-28 2021-11-23 维沃移动通信有限公司 构图方法、装置
CN112843722B (zh) * 2020-12-31 2023-05-12 上海米哈游天命科技有限公司 一种拍摄方法、装置、设备及存储介质
CN113055593B (zh) * 2021-03-11 2022-08-16 百度在线网络技术（北京）有限公司 图像的处理方法和装置
CN113301251B (zh) * 2021-05-20 2023-10-20 努比亚技术有限公司 辅助拍摄方法、移动终端及计算机可读存储介质
CN113643376B (zh) * 2021-07-13 2024-05-03 杭州群核信息技术有限公司 相机视角生成方法、装置、计算设备以及存储介质
CN117135441A (zh) * 2023-02-23 2023-11-28 荣耀终端有限公司 一种图像抓拍方法及电子设备

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009088710A (ja) * 2007-09-27 2009-04-23 Fujifilm Corp 撮影装置、撮影方法及び撮影プログラム
CN104679831A (zh) * 2015-02-04 2015-06-03 腾讯科技（深圳）有限公司 一种匹配人体模型的方法及装置
CN109660719A (zh) * 2018-12-11 2019-04-19 维沃移动通信有限公司 一种信息提示方法及移动终端
CN109788191A (zh) * 2018-12-21 2019-05-21 中国科学院自动化研究所南京人工智能芯片创新研究院 拍照方法、装置、计算机设备和存储介质
CN110472462A (zh) * 2018-05-11 2019-11-19 北京三星通信技术研究有限公司 姿态估计方法、基于姿态估计的处理方法及电子设备
CN111343382A (zh) * 2020-03-09 2020-06-26 Oppo广东移动通信有限公司 拍照方法、装置、电子设备及存储介质

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102332034B (zh) * 2011-10-21 2013-10-02 中国科学院计算技术研究所 一种人像图片检索方法和装置
CN103402058B (zh) * 2013-08-22 2016-08-10 深圳市金立通信设备有限公司 一种拍摄图像的处理方法及装置
JP2015231192A (ja) * 2014-06-06 2015-12-21 オリンパス株式会社 撮像装置および撮像装置の露光制御方法
CN105357436B (zh) * 2015-11-03 2018-07-03 广东欧珀移动通信有限公司 用于图像拍摄中的图像裁剪方法和系统
CN107509032A (zh) * 2017-09-08 2017-12-22 维沃移动通信有限公司 一种拍照提示方法及移动终端
US10932006B2 (en) * 2017-12-22 2021-02-23 Facebook, Inc. Systems and methods for previewing content
CN108769536A (zh) * 2018-07-06 2018-11-06 深圳市赛亿科技开发有限公司 照片拍摄的构图方法、构图装置及计算机可读存储介质
CN110336945A (zh) * 2019-07-09 2019-10-15 上海泰大建筑科技有限公司 一种智能辅助摄影构图方法及系统

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009088710A (ja) * 2007-09-27 2009-04-23 Fujifilm Corp 撮影装置、撮影方法及び撮影プログラム
CN104679831A (zh) * 2015-02-04 2015-06-03 腾讯科技（深圳）有限公司 一种匹配人体模型的方法及装置
CN110472462A (zh) * 2018-05-11 2019-11-19 北京三星通信技术研究有限公司 姿态估计方法、基于姿态估计的处理方法及电子设备
CN109660719A (zh) * 2018-12-11 2019-04-19 维沃移动通信有限公司 一种信息提示方法及移动终端
CN109788191A (zh) * 2018-12-21 2019-05-21 中国科学院自动化研究所南京人工智能芯片创新研究院 拍照方法、装置、计算机设备和存储介质
CN111343382A (zh) * 2020-03-09 2020-06-26 Oppo广东移动通信有限公司 拍照方法、装置、电子设备及存储介质

Also Published As

Publication number Publication date
CN111343382B (zh) 2021-09-10
CN111343382A (zh) 2020-06-26

Similar Documents

Publication Publication Date Title
WO2021179831A1 (zh) 拍照方法、装置、电子设备及存储介质
WO2021175069A1 (zh) 拍照方法、装置、电子设备及存储介质
JP7249390B2 (ja) 単眼カメラを用いたリアルタイム３ｄ捕捉およびライブフィードバックのための方法およびシステム
US11798246B2 (en) Electronic device for generating image including 3D avatar reflecting face motion through 3D avatar corresponding to face and method of operating same
US20210303075A1 (en) Gesture-based shared ar session creation
CN114981844A (zh) 3d身体模型生成
US11508087B2 (en) Texture-based pose validation
WO2020093799A1 (zh) 一种图像处理方法和装置
US11660022B2 (en) Adaptive skeletal joint smoothing
WO2022174594A1 (zh) 基于多相机的裸手追踪显示方法、装置及系统
CN110796083B (zh) 图像显示方法、装置、终端及存储介质
US11887322B2 (en) Depth estimation using biometric data
US20230120037A1 (en) True size eyewear in real time
CN116320721A (zh) 一种拍摄方法、装置、终端及存储介质
US20240098361A1 (en) Recommendations for image capture
CN115880348B (zh) 一种人脸深度的确定方法、电子设备及存储介质
WO2023004682A1 (zh) 身高检测方法、装置及存储介质
CN113763531B (zh) 三维人脸重建方法、装置、电子设备及存储介质
EP4127876B1 (en) Depth estimation using biometric data
EP4390505A2 (en) Depth estimation using biometric data
JP5015109B2 (ja) 情報処理装置、情報処理方法、プログラム及び情報記憶媒体
JP2023108528A (ja) 情報処理装置、情報処理方法、及び情報処理プログラム
CN115802021A (zh) 体积视频生成方法、装置、电子设备及存储介质
CN116630170A (zh) 图像校正方法、装置、电子设备及计算机可读存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21766917

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21766917

Country of ref document: EP

Kind code of ref document: A1