WO2023178589A1

WO2023178589A1 - Filming guiding method, electronic device, system and storage medium

Info

Publication number: WO2023178589A1
Application number: PCT/CN2022/082703
Authority: WO
Inventors: 刘志鹏; 李鑫超
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2022-03-24
Filing date: 2022-03-24
Publication date: 2023-09-28

Abstract

A filming guidance method, an electronic device, a system and a storage medium. The method comprises: acquiring a target video; according to filming information of the target video, determining a template corresponding to the target video, the template being used for guiding a video filming process; and generating filming guidance for one or more missing shots of the target video by utilizing the template. Filming effect improvement is facilitated.

Description

Photography guidance methods, electronic equipment, systems and storage media

Technical field

This application relates to the field of photography technology, specifically, to a photography guidance method, electronic equipment, system and storage medium.

Background technique

With the development and popularization of shooting devices in today's society, users are increasingly inclined to use shooting devices to take pictures when traveling or outing to preserve the beautiful pictures along the way. However, non-professional photographers with insufficient shooting experience may not be able to shoot videos of good quality when using shooting devices. For example, they may shoot randomly due to no shooting ideas, resulting in messy video content and lack of smoothness and coherence. Sex etc. For this type of video, users may delete it and re-shoot the video until the captured video meets the desired requirements. The whole process is relatively cumbersome.

Contents of the invention

In view of this, one of the purposes of this application is to provide a shooting guidance method, electronic device, system and storage medium.

In the first aspect, embodiments of the present application provide a shooting guidance method, including:

Get the target video;

Determine a template corresponding to the target video according to the shooting information of the target video; the template is used to guide the video shooting process;

Using the template, shooting guidelines for one or more missing shots in the target video are generated.

In the embodiments of this application, full utilization of the videos that have been captured by the user can be achieved, and the target video captured by the user in advance can be obtained, and then the template corresponding to the target video is determined based on the shooting information of the target video. The template is usually based on the shooting level, which is reasonable and coherent and smooth. Obtaining excellent films can provide users with shooting ideas, guide the video shooting process, and then generate shooting guidelines for one or more missing shots in the target video, so that users can shoot images related to the missing shots based on the shooting guidelines. /or video to help improve the shooting effect. Moreover, users can only shoot images and/or videos related to the missing lens based on the missing lens shooting guide, without having to start the entire video shooting process from scratch, which helps reduce the user's shooting steps and shooting time, and improves shooting efficiency.

In the second aspect, embodiments of the present application provide a shooting guidance method, including:

Obtain environmental information of the environment where the shooting device is located; and, obtain the target video and determine one or more missing shots in the target video;

According to the environmental information, shooting guidelines for one or more missing shots in the target video are generated.

In the embodiment of the present application, it is helpful to capture images or videos related to the missing lens from the surrounding environment according to the shooting guidelines, and then the target video can be combined with the images or videos related to the missing lens to obtain a coherent and smooth synthetic video, achieving Make full use of targeted videos. Moreover, users can only shoot images and/or videos related to the missing lens based on the shooting guidelines of the missing lens, without having to start the entire video shooting process from scratch, which helps reduce the user's shooting steps and shooting time, and improves shooting efficiency.

In a third aspect, embodiments of the present application provide an electronic device, including:

one or more processors;

memory for storing instructions executable by the processor;

Wherein, the one or more processors execute the executable instructions individually or jointly to perform the method described in the first aspect or the second aspect.

In the fourth aspect, embodiments of the present application provide a system, including the electronic device described in the third aspect and a movable platform; the movable platform is provided with a shooting device;

The electronic device is used to control the movement of the movable platform according to the shooting guidance, so that the shooting device shoots images and/or videos related to the missing lens.

In a fifth aspect, embodiments of the present application provide a computer-readable storage medium that stores executable instructions, and when executed by a processor, the executable instructions implement the first aspect or the second aspect. method described.

Description of the drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without exerting any creative effort.

Figure 1 is a schematic diagram of a flight system provided by an embodiment of the present application;

Figure 2 is a schematic flowchart of a shooting guidance method provided by an embodiment of the present application;

Figure 3 is a schematic diagram of an effect including shooting content provided by an embodiment of the present application;

Figure 4 is a schematic flowchart of another shooting guidance method provided by an embodiment of the present application;

FIG. 5 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of this application.

In related technologies, non-professional photographers with insufficient shooting experience may not be able to shoot videos of good quality when using shooting devices. For example, they may shoot randomly due to no shooting ideas, resulting in messy video content and lack of video content. Fluency, continuity, etc. For this type of video, users may delete it and re-shoot the video until the captured video meets expectations. The whole process is relatively cumbersome.

In response to the problems in related technologies, embodiments of the present application provide a shooting guidance method to fully utilize the videos that have been shot by the user, obtain the target video that has been shot by the user in advance, and then determine the target video based on the shooting information of the target video. The corresponding template is usually obtained based on excellent films with reasonable and coherent shooting levels. It can provide users with shooting ideas, guide the video shooting process, and then generate shooting guidelines for one or more missing shots in the target video based on the template. , allowing users to shoot images and/or videos related to missing lenses based on shooting guidelines, which helps improve shooting effects.

Moreover, users can only shoot images and/or videos related to the missing lens based on the missing lens shooting guide, without having to start the entire video shooting process from scratch, which will help reduce the user’s shooting steps and shooting time, and improve shooting efficiency; subsequent missing lenses Relevant images and/or videos can be synthesized with the target video (corresponding to the video shooting template and/or editing template) into a high-quality video to achieve "waste rescue" of the target video on the basis of making full use of the target video that the user has shot. ".

In some embodiments, the shooting guidance method provided by the embodiments of the present application can be executed by an electronic device. For example, the shooting guidance method may be a computer program product, and the computer program product may be installed in an electronic device. For example, the shooting guidance method may be an executable instruction, which may be stored in a memory of an electronic device and executed by a processor in the electronic device.

In a possible implementation, the electronic device is provided with a shooting device. For example, the electronic device can be a mobile phone, a tablet, a computer or a monitoring device equipped with a shooting device. The shooting device can take photos under the control of the electronic device. images or recorded videos; optionally, the electronic device can also have an image processing function, capable of processing images or videos captured by the shooting device; optionally, the electronic device can also have a visual display function (for example, the electronic device includes Display), capable of displaying images or videos captured by the shooting device.

In another possible implementation, the above-mentioned image processing function, image display function, and even the shooting control function can also be decoupled from the shooting device. For example, the electronic device and the shooting device are independent of each other and connected through communication, such as the electronic device. It may be a device with image processing function, image display function and shooting control function that is communicatively connected with the shooting device.

After obtaining the shooting guidance based on the shooting guidance method provided by the embodiment of the present application, the electronic device can control the shooting device to capture images or record videos according to the shooting guidance, and thereby obtain images and/or videos related to the missing lens.

In an exemplary application scenario, the electronic device may be a remote control device of a movable platform, the movable platform is equipped with a shooting device, and the remote control device has the functions of controlling the movement of the movable platform and controlling the shooting of the shooting device; Image processing function, optionally, the remote control device may also include a display (with image display function). The movable platform includes but is not limited to unmanned aerial vehicles, unmanned vehicles, cloud platforms, unmanned ships or mobile robots. Please refer to Figure 1. Taking the movable platform as an unmanned aerial vehicle as an example, Figure 1 shows a schematic diagram of a flight system. The flight system includes an unmanned aerial vehicle 20 and a remote control device 10. The unmanned aerial vehicle 20 can be an agricultural unmanned aerial vehicle. Human aerial vehicles or unmanned aerial vehicles for industrial applications require cyclic operations; the unmanned aerial vehicle 20 is equipped with a shooting device 21 , and the shooting device 21 can be installed on the unmanned aerial vehicle 20 through a pan/tilt, or can be directly fixed on the unmanned aerial vehicle 20 . The photographing device 21 at least includes a photosensitive element, such as a complementary metal oxide semiconductor (Complementary Metal Oxide Semiconductor, CMOS) sensor or a charge-coupled device (Charge-coupled Device, CCD) sensor.

The unmanned aerial vehicle 20 and the remote control device 10 are connected through communication. After obtaining the shooting guidance based on the shooting guidance method provided in the embodiment of the present application, the remote control device 10 can generate a control instruction sent to the unmanned aerial vehicle 20 according to the shooting guidance to control the unmanned aerial vehicle. The aircraft 20 flies to a desired position, so that the shooting device 21 can capture images or videos related to the missing lens at the desired position. Optionally, the shooting device 21 can send the captured image or video related to the missing lens to the remote control device 10, and the remote control device 10 can display the image or video related to the missing lens through the display. Optionally, the remote control device 10 can synthesize the image or video related to the missing lens and the target video corresponding to the missing lens to obtain a high-quality video and realize full utilization of the target video.

Next, an exemplary description of the shooting guidance method provided by the embodiment of the present application is provided: Please refer to Figure 2, which is a schematic flow chart of a shooting guidance method provided by the embodiment of the present application. The method is applied to electronic equipment, and the method includes:

In step S101, the target video is obtained.

In step S102, a template corresponding to the target video is determined based on the shooting information of the target video; the template is used to guide the video shooting process.

In step S103, the template is used to generate shooting guidelines for one or more missing shots in the target video.

In some embodiments, after acquiring the target video, the electronic device can intelligently analyze the target video, obtain the shooting information of the target video, and then determine the template corresponding to the target video based on the shooting information of the target video. The templates may include video shooting templates and/or editing templates. Video shooting templates and/or editing templates are usually obtained based on excellent films with reasonable and coherent shooting levels. The shooting templates and editing templates can provide users with shooting ideas. Guide the video shooting process; for example, the template indicates a series of shots with continuity. The electronic device can determine one or more missing shots in the target video based on the shot difference between the template and the target video, and then generate shooting guidelines for one or more missing shots in the target video, so that the user can based on Shooting Guidelines Capture images and/or videos related to the missing footage to help improve the results. Moreover, only shooting images and/or videos related to the missing lens can also help reduce shooting steps and improve shooting efficiency; then the images and/or videos related to the missing lens can be combined with the target video to make full use of the target that the user has shot. Based on the video, the "waste film rescue" of the target video is realized.

In some embodiments, the target video may be a video taken by the user at a historical time. For example, the target video can be manually selected by the user from the video collection based on actual shooting needs, and then the electronic device can obtain the target video based on the user's selection operation. For example, the target video may also be automatically selected by the electronic device based on relevant information.

In a possible implementation, it is considered that after the user reaches a place, he may want to take pictures of the surrounding environment. Then the electronic device can control the shooting device to obtain scene information from the surrounding environment. For example, taking the flight system shown in Figure 1 as an example, the remote control device can control the unmanned aerial vehicle to fly around and obtain the information of the shooting device during the flight of the unmanned aerial vehicle. The electronic device can then perform scene recognition on the captured image or video to obtain scene information of the surrounding environment. For example, the scene information includes but is not limited to beach and seaside scenes, grassland scenes, food scenes, architectural scenes, desert scenes or hillside scenes, etc. The electronic device may select the target video from a video collection based on scene information obtained by the shooting device from the surrounding environment; the video collection includes at least one video shot at a historical time. In this embodiment, the acquisition of the target video related to the surrounding environment is achieved, which facilitates the natural combination of the image (or video) related to the missing lens and the target video captured in the surrounding environment.

For example, the videos in the video collection all correspond to shooting scene information. The electronic device can compare the scene information of the surrounding environment with the shooting scene information of each video in the video collection. According to the scene information of the surrounding environment and the shooting scene information, The target video is selected based on the similarity of the shooting scene information of each video in the video collection; for example, a video with a similarity higher than a preset threshold can be selected as the target video; for another example, the selected target video satisfies the following conditions: The similarity between the shooting scene information corresponding to the target video and the scene information of the surrounding environment is higher than the similarity between the shooting scene information corresponding to other videos in the video collection and the scene information of the surrounding environment. . In this embodiment, obtaining a target video with a similar scene to the surrounding environment facilitates the natural combination of subsequent images (or videos) related to the missing lens captured in the surrounding environment and the target video.

In another possible implementation, the electronic device can also select the target video from a video collection based on the geographical location of the shooting device; the video collection includes at least one video shot at a historical time. . For example, the videos in the video collection all correspond to shooting location information, and the electronic device can select a target video based on the distance difference between the geographical location of the shooting device and the shooting location of each video in the video collection. For example, a video whose distance difference is less than a preset distance value is selected as the target video; for another example, the selected target video satisfies the following conditions: the distance between the shooting location corresponding to the target video and the geographical location where the shooting device is located , is less than the distance between the shooting locations corresponding to other videos in the video collection and the geographical location where the shooting device is located. In this embodiment, the shooting location of the target video is the same as or close to the current shooting location of the shooting device, which facilitates the natural combination of subsequent images (or videos) related to the missing lens and the target video that are captured at the current shooting location.

Optionally, the target video can also be selected from the video collection by combining the scene information obtained by the shooting device from the surrounding environment and the geographical location of the shooting device, thereby further improving the accuracy of the subsequent shooting corresponding to the missing lens. The correlation between the image (or video) and the target video improves the coherence and fluency of the combination of the two.

Of course, the user can also pre-specify the shooting scene and/or the geographical location of the shooting scene, so that the electronic device selects a target video with a similar scene based on the scene information of the specified shooting scene, and/or based on the specified geographical location of the shooting scene. Select a target video with the same or similar shooting location, so that a shooting guide related to the missing lens of the target video and the shooting scene can be generated, which is helpful for subsequent images (or videos) related to the missing lens that are shot in the shooting scene. ) is a natural combination with the target video.

In some embodiments, after acquiring the target video, the electronic device may analyze the target video to obtain shooting information of the target video. Exemplarily, the photographing information includes at least one of the following: photographing scene information, photographing location information, photographed object, posture of the photographed subject, scene type of the photographed subject, composition of the photographed subject, lens movement technique, or Aesthetic quality evaluation information.

In one example, the electronic device can perform scene recognition on the target video and obtain the shooting scene information of the target video. The shooting scene information includes but is not limited to beach and seaside scenes, grassland scenes, food scenes, people scenes, animal scenes, and architectural scenes. , desert scenes or hillside scenes, etc.

In one example, the electronic device can perform object recognition on the target video and obtain the subject in the target video. For example, the subject may be a person, a car, an animal, a static object (such as a sculpture), etc.

In one example, the electronic device can determine the subject in the target video, for example, determine the object that appears most often in the target video as the subject; and then determine the scene of the subject, for example, based on the subject. The difference between the proportion of the subject in each video frame is used to determine the scene of the subject; alternatively, the composition of the subject can be analyzed, for example, the subject can be determined based on the position of the subject in the video frame. The composition of the subject; or, you can also perform posture recognition on the subject to obtain the posture of the subject.

In one example, the electronic device can perform lens movement analysis on the target video, such as determining the lens movement technique of the target video by analyzing the position changes of the subject in the target video in different video frames.

In one example, the electronic device can input the target video into a pre-trained aesthetic quality evaluation model, and obtain the aesthetic quality evaluation information of the target video output by the aesthetic quality evaluation model. For example, the aesthetic quality evaluation model can be obtained through supervised learning based on labeled video samples.

In some embodiments, after obtaining the shooting information of the target video, the electronic device can determine a template corresponding to the target video according to the shooting information of the target video; wherein the template is used to guide the video shooting process; Templates can be obtained based on excellent videos with reasonable and coherent shooting levels, and can provide users with shooting ideas. Users can refer to the templates to shoot similar high-quality videos.

Wherein, the template includes a video shooting template and/or the editing template. The video shooting template indicates the video content that needs to be shot, and the editing template indicates the video content that needs to be edited. For example, the video shooting template and/or the editing template include a series of shot information abstracted from excellent finished films. The lens information includes but is not limited to: lens information indicating different scenes, lens information indicating different camera movements, lens information indicating different scenes of the subject, lens information indicating different postures of the subject, lens information indicating different postures of the subject, Lens information for different compositions or lens information indicating different objects. That is to say, the video shooting template or the editing template may include at least two different lens information; the lens information may be presented in the form of text, image or video.

In one example, the video shooting template is obtained from excellent films of food production. The video shooting template includes a coherent series of kitchen environment, food introduction, washing vegetables, cutting vegetables, stir-frying, and close-ups of food. Lens information, such as the above-mentioned lens information, can be presented as an image. In another example, for example, the editing template is obtained from excellent films in the camera movement category. The editing template includes a coherent series of lens information such as camera advancement → 360° camera movement to the left → camera zoom out. , for example, these three lens information can be presented in video.

Exemplarily, the electronic device has a template set pre-stored, and the template set includes several reference video shooting templates and/or several reference editing templates, and the reference video shooting templates and the reference editing templates all correspond to reference Shooting information.

In the process of determining the template corresponding to the target video, the electronic device may compare the shooting information of the target video with the reference shooting information of each template (reference video shooting template and/or reference editing template) in the template set. Yes, according to the similarity between the shooting information of the target video and each reference shooting information, a template corresponding to the target video is selected from a pre-stored template set. For example, a reference video shooting template and/or a reference editing template whose similarity is higher than a preset threshold can be selected as the template corresponding to the target video; for example, the selected template satisfies the following conditions: the reference shooting information of the template corresponding to the target video and The similarity between the shooting information of the target video is higher than the similarity between other reference shooting information in the template set and the shooting information of the target video. In this embodiment, selecting a reference shooting template and/or a reference editing template that is similar to the shooting information of the target video can guide the user to shoot high-quality images or videos, which is beneficial to subsequent shooting of images corresponding to the missing shots (or video) is coherently and smoothly combined with the target video.

In some embodiments, several reference video shooting templates and/or several reference editing templates included in the template set can be obtained based on excellent films with reasonable shooting levels and coherent and smooth shooting levels. For example, several videos in the preset video library can be analyzed, videos with coherence and/or aesthetic quality evaluation information that meet the preset requirements can be selected, and then reference video shooting templates and/or reference clips can be generated based on such videos. template. For example, the reference video shooting template and/or the reference editing template indicates a series of shot information abstracted from excellent finished films.

It can be understood that this embodiment does not place any restrictions on the number of videos in the preset video library, and specific settings can be made according to actual application scenarios. In one example, the videos in the preset video library may be videos with a high number of likes and/or comments on a public video sharing platform.

In one example, several videos in the preset video library can be input into a pre-trained aesthetic quality evaluation model, and the aesthetic quality evaluation information of each video can be obtained through the aesthetic quality evaluation model, and then the aesthetic quality evaluation information satisfies the predetermined requirements. Generate reference video shooting templates and/or reference editing templates for videos that meet certain requirements (for example, the aesthetic quality evaluation score is higher than a preset score).

Exemplarily, the reference video shooting template and/or the reference editing template are determined based on the video specified by the user selection instruction. For example, the user can select the video he wants to refer to according to his own aesthetics and needs, and then the electronic device can generate the reference video shooting template and/or the reference editing template according to the video specified by the user's selection instruction.

In some embodiments, after determining the template corresponding to the target video (the template includes the video shooting template and/or the editing template), the electronic device can use the video shooting template and/or the editing template, Generate shooting guidelines for one or more missing shots in the target video. In this embodiment, the generated shooting guide can provide the user with shooting ideas, guide the user to shoot high-quality images or videos, and only need to shoot images or videos related to the missing shots in the target video, without starting the video shooting from scratch. , which is conducive to reducing the user's shooting steps and improving shooting efficiency, and the captured images or videos related to the missing lens can be combined with the target video to achieve effective use of the target video.

In some embodiments, when the shooting effect of certain video frames in the target video may not be very good, re-shooting may be performed. The electronic device can analyze the target video, determine the aesthetic quality evaluation information of each video frame in the target video, and then eliminate the video frames in the target video whose aesthetic quality evaluation information does not meet the preset requirements. Furthermore, after determining the template corresponding to the target video, the electronic device can use the template to generate shooting guidelines for one or more missing shots in the target video after removing video frames. In this embodiment, some video frames in the target video with poor shooting results can be deleted and re-shot, thereby improving the effect of combining the target video with the image (or video) corresponding to the missing shot.

In some embodiments, the electronic device can use the template to determine one or more missing shots in the target video, and then generate shooting guidelines for the one or more missing shots. Wherein, the missing shots include but are not limited to at least one of the following: shots missing a specified scene, shots missing a specified camera technique, shots missing a specified scene of the subject, shots missing a specified pose of the subject, shots missing the subject. A shot in which the subject has a specified composition or a shot in which a specified object is missing. It can be understood that the shooting guide indicates how to obtain the image or video corresponding to the missing lens.

In the first example, for example, the template corresponding to the target video is a video shooting template of a food production scene. The video shooting template includes a coherent sequence of kitchen environment, food introduction, washing vegetables, cutting vegetables, stir-frying, and close-ups of food. According to the series of lens information, it is determined that the target video includes shots of the kitchen environment, food introduction, washing vegetables, and cooking according to the video shooting template. Then it can be determined that the missing shots in the target video include vegetable cutting shots and food close-ups. , can generate shooting guidelines for vegetable cutting shots and close-up shots of food. For example, the shooting guide for close-up shots of gourmet food instructs the forward-moving close-up shooting technique of food, or rotating the food from left to right (or from right to left).

In the second example, for example, the template corresponding to the target video is a scene-type editing template. The editing template indicates that there are coherent shots such as close-ups, close-ups, medium shots, long shots, and panoramic shots. According to the editing template, all the shots are determined. If the target video includes mid-range, close-up and panoramic shots, it can be determined that the missing shots in the target video include close-up shots and long-range shots, and the electronic device can generate shooting guidelines for close-up shots and long-range shots.

In the third example, for example, the template corresponding to the target video is a video shooting template of the character posture category. The video shooting template indicates a series of shots of characters dancing, jumping, rotating, and sideways. According to the video shooting template, all the shots are determined. If the target video includes shots of characters dancing, rotating, and side-stepping, it can be determined that the missing shots in the target video include shots of characters jumping, and the electronic device can generate shooting guidelines for the character jumping shots.

In some embodiments, it is considered that when a user goes to a certain place to take pictures, he or she most likely wants to take pictures from the surrounding environment. Therefore, the electronic device can obtain the environmental information of the environment where the shooting device is located, and then generate shooting guidelines for one or more missing shots in the target video based on the template and the environmental information; in this embodiment, the generated The shooting guideline is related to both the missing lens of the target video and the environment where the shooting device is located, which helps to capture images or videos related to the missing lens from the surrounding environment according to the shooting guideline.

For example, the electronic device can determine one or more missing shots in the target video according to the template; and, the electronic device can obtain information related to the environment where the shooting device is located based on the environmental information of the environment where the shooting device is located. Multiple reference shooting guides; and then the electronic device can obtain the shooting guides related to the one or more missing lenses from the multiple reference shooting guides, so that the shooting device can capture the missing lenses from the surrounding environment according to the shooting indexes. Related images or videos.

In one example, the template corresponding to the target video is a video shooting template of a seaside scene. The video shooting template includes a panoramic view of the sea, a medium shot of the beach, a close shot of the beach, a close-up of the waves, a picture of the waves hitting the rocks, and a picture of the waves receding from the beach. and a series of shot information, and based on the video shooting template, it is determined that the missing shots in the target video include mid-range shots of the beach, close-ups of waves, and scenes of waves hitting the rocks. Assuming that the current environment of the shooting device is also a seaside environment, multiple reference shooting guidelines related to the seaside environment include the following reference shooting guidelines corresponding to various images: sea panorama, sea mid-range, sea close-up, beach panorama, beach mid-range, Close-up of the beach, moving the camera upward from the beach to the magnificent sea, moving the camera from left to right to capture the waves, close-ups of the waves, waves hitting the rocks, waves receding from the beach, high tide, low tide, swimming, and sailboat driving. , close-up of people on the seaside, mid-shot of people on the seaside, close-up of people on the seaside, long-shot of seaside buildings or mid-shot of seaside buildings, etc. The electronic device can obtain the shooting guide for the mid-range shot of the beach, the shooting guide for the close-up of the waves, and the shooting guide for the scene of the waves hitting the rocks from multiple reference shooting guides related to the seaside environment. The shooting device can obtain the shooting guide from the seaside environment according to the shooting index. Images or videos related to the missing footage.

Here is an exemplary description of the acquisition process of multiple reference shooting guidelines related to the environment where the shooting device is located:

In a possible implementation, the environmental information of the environment where the shooting device is located includes visual semantic information; the electronic device can control the shooting device to shoot images and/or videos of the surrounding environment, such as flying as shown in Figure 1 Taking the system as an example, the remote control device can control the unmanned aerial vehicle to fly around and obtain the images or videos captured by the shooting device during the flight of the unmanned aerial vehicle. The electronic device can perform semantic segmentation processing or instance segmentation processing on the image or video of the surrounding environment to recognize visual semantic information from the image or video of the surrounding environment.

In an example, such as in a seaside scene, the visual semantic information recognized by the electronic device may include: beaches, coasts, seawater, reefs, seaside buildings, etc. In another example, in a kitchen scene, the visual semantic information recognized by the electronic device may include various kitchen supplies (such as cutting boards, rice cookers or woks, etc.), kitchen buildings (sinking tables, dining tables), spices and/or food etc.

The electronic device can pre-store the mapping relationships between different reference visual semantic information and reference shooting guidelines. Then, after acquiring the visual semantic information of the surrounding environment, the electronic device can, based on the visual semantic information, use the different pre-stored reference visual semantic information and reference shooting guidelines. A plurality of reference shooting guidelines related to the environment where the shooting device is located are determined from the mapping relationship. For example, the visual semantic information can be compared with the reference visual semantic information in the mapping relationship, and the reference shooting guideline corresponding to the reference visual semantic information with a similarity higher than a preset threshold is determined to be the same as the location where the shooting device is located. Environmentally relevant reference shooting guidelines.

In one example, if the visual semantic information includes mountains, then reference shooting guidelines about the mountains are matched, such as starting with an introduction to the mountain's stone tablet, or starting with the magnificent mountain scenery to explain the environmental information. In another example, for example, the visual semantic information includes a sculpture, which can be matched with a reference shooting guideline related to the sculpture. For example, the reference shooting guideline includes a shooting method that surrounds the sculpture, or a shooting method that gradually zooms in on the sculpture.

In another possible implementation, the environmental information includes the geographical location of the environment where the shooting device is located. In the process of obtaining multiple reference shooting guidelines related to the environment where the shooting device is located, the electronic device can determine the scenic spots around the geographical location based on the geographical location of the environment where the shooting device is located, and obtain the information related to the location. Describes multiple reference shooting guidelines related to scenic spots. For example, if the scenic spot around the geographical location is determined to be "Hangzhou West Lake", multiple reference shooting guidelines related to the "Hangzhou West Lake" scenic spot can be obtained.

For example, the electronic device can pre-store the mapping relationships between different reference scenic spots and reference shooting guidelines. Then, after determining the scenic spots around the geographical location, the electronic device can determine the scenic spots from the pre-stored mapping relations based on the scenic spots near the shooting device. Multiple reference shooting guidelines related to scenic spots.

In some embodiments, after generating shooting guidelines for one or more missing shots in the target video, the user can shoot images or videos related to the missing shots according to the shooting guidelines; or, the electronic device can shoot images or videos related to the missing shots according to the shooting guidelines. The shooting guide automatically controls the shooting device to shoot images or videos related to the missing lens. Only shooting images or videos related to the missing lens can also help reduce shooting steps and improve shooting efficiency.

In some embodiments, in a scenario where the user captures images or videos related to the missing lens according to the shooting guide, the shooting guide includes: shooting content and/or shooting techniques corresponding to the shooting content. That is to say, the shooting guide can tell the user what to shoot and how to shoot, so that the user can shoot high-quality images or videos. The shooting guidance may include one or more of text guidance, auditory information guidance, image guidance and video guidance. For example, in order to allow the user to intuitively experience the effect of the film, the shooting content can be presented in a schematic diagram or an example video clip containing the shooting content; for example, please refer to Figure 3, which shows a close-up view of a person on the beach in a seaside scene. Effect diagram. For example, in order to allow users to intuitively learn shooting techniques, the shooting techniques can also be presented through images or videos, and the implementation of relevant shooting techniques can be intuitively explained through images or videos, which is conducive to improving the user's learning efficiency, wherein, so The shooting method may indicate at least one of the following shooting guidance information: shooting parameters of the shooting device, a field of view direction of the shooting device, a reference distance between the shooting device and the subject, or a distance of the shooting device. Movement trajectories, etc.

For example, taking an example video clip in which the scene of a character changes from a close-up to a close-up, the relevant shooting technique can be to instruct the user in text by changing the focal length of the shooting device, or it can be intuitive in the form of images or videos. Instruct users to achieve this by changing the distance between themselves and the person being photographed.

In other embodiments, in a scenario where the electronic device automatically controls the shooting device to capture images or videos related to the missing lens according to the shooting guide, the shooting guide may include one or more shooting control instructions for the missing lens, so as to This allows the shooting device to capture images or videos related to the missing lens based on the shooting control instruction.

In some exemplary embodiments, taking the electronic device as a remote control device of a movable platform as an example, the movable platform is communicatively connected to the remote control device, and the movable platform is provided with a shooting device. The remote control device may control the movement of the movable platform according to the shooting guidance, so that the shooting device shoots images and/or videos related to the one or more missing shots. This embodiment realizes automatic control of the shooting device to shoot images or videos related to the missing lens based on the shooting guidance, which is beneficial to reducing the user's operating steps and improving the user experience.

Exemplarily, the shooting guide indicates the shooting parameters of the shooting device, the desired position of the movable platform, the orientation of the movable platform, the motion trajectory of the movable platform, and/or the Reference distance between the movable platform and the subject, etc. The remote control device can generate shooting control instructions for the one or more missing lenses according to the shooting guidance, and send them to the movable platform to control the movable platform to move (such as controlling the movable platform to change its orientation). , change one's own position, change the distance between oneself and the subject, or move according to the indicated movement trajectory, etc.) and use the shooting device to shoot images or videos related to the missing lens. For example, when the shooting control instruction carries the shooting parameters of the shooting device, the shooting parameters of the shooting device can also be changed according to the shooting control instruction.

In one example, the shooting guide includes an effect diagram; the remote control device can control the movement of the movable platform according to the difference between the effect diagram and the image currently captured by the remote control device until the image currently captured by the remote control device is consistent with the difference between the effect diagram and the image currently captured by the remote control device. The difference between the effect diagrams is less than the preset difference.

In another example, the shooting guide indicates a desired position where the movable platform is located; the remote control device can control the movable platform to move to the desired position according to the shooting guide, so that the shooting The device captures images and/or videos associated with the one or more missing shots at the desired location.

In another example, the shooting guide indicates the movement trajectory of the movable platform; the remote control device can control the movement of the movable platform according to the movement trajectory, so that the shooting device moves on the movable platform Images and/or videos related to the one or more missing shots are captured during the process. Taking the flight system shown in Figure 1 as an example, for example, the missing lens is a surround shooting lens, the shooting guide indicates the flight path of the unmanned aircraft flying around the subject, and the remote control device can be controlled according to the flight path. The unmanned aerial vehicle flies around the subject, so that the shooting device captures video corresponding to the surrounding shooting lens.

In some embodiments, the template includes a editing template; after the shooting device captures the images and/or videos related to the missing lens based on the shooting guide, the electronic device can obtain the image related to the missing lens. and/or video, and then use the editing template to synthesize the image and/or video and the target video, taking into account that the image and/or video shot based on the shooting guide is associated with the target video, thereby ensuring It ensures the natural coherence of the synthesized video and makes full use of the target video.

Correspondingly, please refer to Figure 4. This embodiment of the present application also provides a schematic flow chart of another shooting guidance method. The method is applied to electronic equipment, and the method includes:

In step S201, environmental information of the environment where the shooting device is located is obtained; and, a target video is obtained and one or more missing shots in the target video are determined.

In step S202, based on the environment information, a shooting guide for one or more missing shots in the target video is generated.

This embodiment realizes full utilization of the target video. After determining one or more missing shots in the target video, the shooting of one or more missing shots in the target video can be generated based on the environmental information of the environment where the shooting device is located. Guide, which helps to capture images or videos related to the missing lens from the surrounding environment according to the shooting guidelines, and then the target video can be combined with the images or videos related to the missing lens to obtain a coherent and smooth synthetic video, achieving full understanding of the target video. use. Moreover, users can only shoot images and/or videos related to the missing lens based on the shooting guidelines of the missing lens, without having to start the entire video shooting process from scratch, which helps reduce the user's shooting steps and shooting time, and improves shooting efficiency.

In some embodiments, determining one or more missing shots in the target video includes: determining a template corresponding to the target video based on the shooting information of the target video; the template is used to guide video shooting Process: using the template, determine one or more missing shots in the target video.

In some embodiments, generating shooting guidelines for one or more missing shots in the target video based on the environmental information includes: obtaining, based on the environmental information, information related to the environment where the shooting device is located. A plurality of reference shooting guidelines; and obtaining a shooting guide related to the one or more missing lenses from the plurality of reference shooting guidelines.

Exemplarily, the shooting device is used to collect images and/or videos of the surrounding environment; the environmental information includes visual semantic information recognized from the images or videos of the surrounding environment; the multiple reference shooting guidelines According to the visual semantic information, it is determined from the mapping relationship between different pre-stored reference visual semantic information and reference shooting guidelines.

Exemplarily, the environmental information includes the geographical location of the environment where the shooting device is located; and obtaining multiple reference shooting guidelines related to the environment where the shooting device is located based on the environmental information includes: according to the shooting device The geographical location of the environment is determined, the scenic spots around the geographical location are determined, and multiple reference shooting guidelines related to the scenic spots are obtained. For example, the plurality of reference shooting guidelines related to the scenic spot are determined from the pre-stored mapping relationships between different scenic spots and reference shooting guidelines based on the scenic spots around the geographical location.

In some embodiments, the shooting information includes at least one of the following: shooting scene information, shooting location information, the subject, the posture of the subject, the scene of the subject, the composition of the subject, the movement of the camera Technique or aesthetic quality evaluation information.

In some embodiments, the template corresponding to the target video is obtained from a pre-stored template set; the template set includes several reference video shooting templates and/or several reference editing templates, and the reference video shooting template and all The reference editing templates all correspond to reference shooting information; the similarity between the reference shooting information of the template corresponding to the target video and the shooting information of the target video is higher than that between other reference shooting information in the template set and The similarity between the shooting information of the target video.

In some embodiments, the missing shots include at least one of the following: shots missing a specified scene, shots missing a specified camera technique, shots missing a specified scene of the subject, shots missing a specified pose of the subject, Shots with a specified composition of the subject or shots with a specified object missing.

In some embodiments, obtaining the target video includes: selecting the target video from a video collection according to the scene information obtained by the shooting device from the surrounding environment and/or the geographical location of the shooting device; the video collection Include at least one video shot in historical time. Exemplarily, the similarity between the shooting scene information corresponding to the target video and the scene information is higher than the similarity between the shooting scene information corresponding to other videos in the video collection and the scene information. ; and/or the distance between the shooting location corresponding to the target video and the geographical location where the shooting device is located is less than the distance between the shooting locations corresponding to other videos in the video collection and the geographical location where the shooting device is located. distance between.

In some embodiments, the shooting guide includes: shooting content and/or shooting techniques corresponding to the shooting content. Exemplarily, the shooting content is presented in a schematic diagram or an example video clip containing the shooting content; the shooting method is presented in an image or video; the shooting method instruction includes at least one of the following shooting guidance information: the shooting device The shooting parameters, the field of view direction of the shooting device, the reference distance between the shooting device and the subject, or the motion trajectory of the shooting device.

In some embodiments, the method is applied to a remote control device of a movable platform; the movable platform is provided with a shooting device; the method further includes: controlling the movement of the movable platform according to the shooting guidance, so that The photographing device collects and captures images and/or videos related to the one or more missing lenses. Wherein, the shooting guide indicates the desired position of the movable platform, the orientation of the movable platform, the motion trajectory of the movable platform, and/or the distance between the movable platform and the subject. Reference distance.

In some embodiments, the method further includes: obtaining images and/or videos related to the missing lens shot based on the shooting guidelines; using the editing template to edit the images and/or videos, and the The target video is synthesized. The various technical features in the above embodiments can be combined arbitrarily, as long as there is no conflict or contradiction between the combinations of features. Therefore, any combination of the various technical features in the above embodiments also falls within the scope of the disclosure of this specification.

Correspondingly, please refer to Figure 5. This embodiment of the present application also provides an electronic device 10, including:

one or more processors 11;

Memory 12 for storing instructions executable by the processor 11;

The one or more processors 11 execute the executable instructions individually or jointly to perform any one of the above methods.

The processor 11 executes executable instructions included in the memory 12. The processor 11 may be a central processing unit (Central Processing Unit, CPU), or other general-purpose processor, digital signal processor (Digital Signal Processor). Processor, DSP), application specific integrated circuit (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

The memory 12 stores executable instructions for the shooting guidance method. The memory 12 may include at least one type of storage medium. The storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.) , Random Access Memory (RAM), Static Random Access Memory (SRAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Programmable Read Only Memory (PROM), Magnetic Memory, Disk, CDs etc. Furthermore, the device may cooperate with a network storage device that performs the storage functions of the memory through a network connection. The memory 12 may be an internal storage unit of the device 10 , such as a hard disk or memory of the device 10 . The memory 12 may also be an external storage device of the device 10, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SD) card, or a flash card (Flash Card) equipped on the device 10. wait. Further, the memory 12 may also include both an internal storage unit of the device 10 and an external storage device. Memory 12 is used to store executable instructions and other programs and data required by the device. The memory 12 may also be used to temporarily store data that has been output or is to be output.

In some embodiments, the electronic device is provided with a photographing device; or the electronic device is communicatively connected with the photographing device.

Exemplarily, the electronic device includes a remote control device of a movable platform, the movable platform is provided with a shooting device, and the remote control device is communicatively connected with the movable platform.

Exemplarily, the movable platform includes at least one of the following: unmanned aerial vehicle, unmanned vehicle, cloud platform, unmanned vessel or mobile robot.

Various implementations described herein may be implemented using computer-readable media, such as computer software, hardware, or any combination thereof. For hardware implementation, the embodiments described herein may be implemented using an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array ( FPGA), a processor, a controller, a microcontroller, a microprocessor, an electronic unit designed to perform the functions described herein. For software implementations, embodiments such as procedures or functions may be implemented with separate software modules that allow at least one function or operation to be performed. The software code may be implemented by a software application (or program) written in any suitable programming language, which may be stored in memory and executed by the controller.

The electronic device 10 may include, but is not limited to, a processor 11 and a memory 12 . Those skilled in the art can understand that FIG. 5 is only an example of the electronic device 10 and does not constitute a limitation of the electronic device 10. It may include more or fewer components than shown, or some components may be combined, or different components may be used. , for example, devices may also include input and output devices, network access devices, buses, etc.

The specific implementation process of the functions and roles of each unit in the above equipment can be found in the implementation process of the corresponding steps in the above method, and will not be described again here.

Correspondingly, please refer to Figure 1. An embodiment of the present application also provides a system, including the above-mentioned electronic device and a movable platform; the movable platform is provided with a shooting device. The electronic device is used to control the movement of the movable platform according to the shooting guidance, so that the shooting device shoots images and/or videos related to the missing lens.

In some embodiments, the movable platform includes at least one of the following: an unmanned aerial vehicle, an unmanned vehicle, a gimbal, an unmanned vessel, or a mobile robot.

In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions, such as a memory including instructions, which can be executed by a processor of the device to complete the above method is also provided. For example, non-transitory computer-readable storage media may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

A non-transitory computer-readable storage medium enables the terminal to perform the above method when instructions in the storage medium are executed by a processor of the terminal.

It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that these entities or operations are mutually exclusive. any such actual relationship or sequence exists between them. The terms "comprises," "comprises," or any other variation thereof are intended to cover a non-exclusive inclusion such that a process, method, article or apparatus including a list of elements includes not only those elements but also others not expressly listed elements, or elements inherent to such process, method, article or equipment. Without further limitation, an element defined by the statement "comprises a..." does not exclude the presence of additional identical elements in a process, method, article, or apparatus that includes the stated element.

The methods and devices provided by the embodiments of the present application have been introduced in detail above. Specific examples are used in this article to illustrate the principles and implementations of the present application. The description of the above embodiments is only used to help understand the methods and methods of the present application. The core idea; at the same time, for those of ordinary skill in the field, there will be changes in the specific implementation and application scope based on the ideas of this application. In summary, the content of this description should not be understood as a limitation of this application. .

Claims

A shooting guidance method, which is characterized by including:

Get the target video;

Determine a template corresponding to the target video according to the shooting information of the target video; the template is used to guide the video shooting process;

Using the template, shooting guidelines for one or more missing shots in the target video are generated.
The method according to claim 1, characterized in that the shooting information includes at least one of the following: shooting scene information, shooting location information, subject, posture of the subject, scene of the subject, subject The subject’s composition method, camera movement, or aesthetic quality evaluation information.
The method according to claim 1, characterized in that the template corresponding to the target video is obtained from a pre-stored template set;

The template set includes several reference video shooting templates and/or several reference editing templates, and the reference video shooting templates and the reference editing templates each correspond to reference shooting information;

The similarity between the reference shooting information of the template corresponding to the target video and the shooting information of the target video is higher than the similarity between other reference shooting information in the template set and the shooting information of the target video. Spend.
The method according to claim 3, characterized in that the reference video shooting template and/or the reference editing template are determined based on videos with coherence and/or aesthetic quality evaluation information that meet preset requirements; or the reference The video shooting template and/or the reference editing template are determined based on the video specified by the user selection instruction.
The method according to claim 1, characterized in that the missing shots include at least one of the following: shots missing a specified scene, shots missing a specified camera technique, shots missing a specified scene of the subject, shots missing the subject. Shots in which the subject poses in a specified manner, in which the specified composition of the subject is missing, or in which a specified object is missing.
The method according to claim 1, characterized in that said obtaining the target video includes:

The target video is selected from a video collection according to the scene information obtained by the shooting device from the surrounding environment and/or the geographical location of the shooting device; the video collection includes at least one video shot at a historical time.
The method according to claim 6, characterized in that the similarity between the shooting scene information corresponding to the target video and the scene information is higher than the shooting scene information corresponding to other videos in the video collection and the similarity between the shooting scene information corresponding to the target video and the scene information. The similarity between the scene information; and/or

The distance between the shooting location corresponding to the target video and the geographical location where the shooting device is located is smaller than the distance between the shooting location corresponding to other videos in the video collection and the geographical location where the shooting device is located.
The method according to any one of claims 1 to 7, characterized in that using the template to generate shooting guidelines for one or more missing shots in the target video includes:

Obtain environmental information of the environment where the shooting device is located;

According to the template and the environment information, shooting guidelines for one or more missing shots in the target video are generated.
The method of claim 8, wherein generating shooting guidelines for one or more missing shots in the target video based on the template and the environment information includes:

Determine one or more missing shots in the target video according to the template; and

According to the environmental information, obtain multiple reference shooting guidelines related to the environment where the shooting device is located;

A shooting guide related to the one or more missing shots is obtained from the plurality of reference shooting guides.
The method according to claim 9, characterized in that the shooting device is used to shoot images and/or videos of the surrounding environment; the environmental information includes visual semantic information recognized from the images or videos of the surrounding environment. ;

The plurality of reference shooting guidelines are determined from the mapping relationships between different pre-stored reference visual semantic information and reference shooting guidelines based on the visual semantic information.
The method according to claim 9, characterized in that the environmental information includes the geographical location of the environment where the shooting device is located;

Obtaining multiple reference shooting guidelines related to the environment where the shooting device is located based on the environmental information, including:

According to the geographical location of the environment where the shooting device is located, a scenic area around the geographical location is determined, and a plurality of reference shooting guidelines related to the scenic area are obtained.
The method according to claim 11, characterized in that the plurality of reference shooting guides related to the scenic spots are determined from the pre-stored mapping relationships between different scenic spots and the reference shooting guides based on the scenic spots around the geographical location.
The method according to any one of claims 1 to 12, wherein the shooting guide includes: shooting content and/or shooting techniques corresponding to the shooting content.
The method according to claim 13, characterized in that the photographed content is presented in the form of a schematic diagram or an example video clip containing the photographed content;

The shooting techniques are presented through images or videos;

The shooting technique instruction includes at least one of the following shooting guidance information: the shooting parameters of the shooting device, the field of view direction of the shooting device, the reference distance between the shooting device and the subject, or the shooting distance of the shooting device. Movement trajectory.
The method of claim 1, further comprising:

Eliminate video frames whose aesthetic quality evaluation information does not meet preset requirements in the target video;

The generation of shooting guidelines for one or more missing shots in the target video includes:

Generate shooting guidelines for one or more missing shots in the target video after removing the video frames.
The method according to claim 1, characterized in that the method is applied to a remote control device of a movable platform; the movable platform is provided with a shooting device;

The method also includes:

The movement of the movable platform is controlled according to the shooting guide, so that the shooting device shoots images and/or videos related to the one or more missing shots.
The method according to claim 16, wherein the shooting guide indicates shooting parameters of the shooting device, a desired position of the movable platform, an orientation of the movable platform, movement trajectory, and/or the reference distance between the movable platform and the subject.
The method according to claim 1 or 16, characterized in that the template includes a clipping template, and the method further includes:

Obtain images and/or videos related to the missing lens shot based on the shooting guidelines;

The image and/or video and the target video are synthesized using the editing template.
A shooting guidance method, which is characterized by including:

Obtain environmental information of the environment where the shooting device is located; and, obtain the target video and determine one or more missing shots in the target video;

According to the environmental information, shooting guidelines for one or more missing shots in the target video are generated.
The method of claim 19, wherein determining one or more missing shots in the target video includes:

Determine a template corresponding to the target video according to the shooting information of the target video; the template is used to guide the video shooting process;

Using the template, one or more missing shots in the target video are determined.
The method of claim 19, wherein generating shooting guidelines for one or more missing shots in the target video based on the environmental information includes:

According to the environmental information, obtain multiple reference shooting guidelines related to the environment where the shooting device is located;

A shooting guide related to the one or more missing shots is obtained from the plurality of reference shooting guides.
An electronic device, characterized by including:

one or more processors;

memory for storing instructions executable by the processor;

Wherein, the one or more processors execute the executable instructions individually or jointly to perform the method according to any one of claims 1 to 21.
The electronic device according to claim 22, characterized in that the electronic device is provided with a photographing device; or the electronic device is communicatively connected with the photographing device.
The electronic equipment according to claim 22, characterized in that the electronic equipment includes a remote control device of a movable platform, the movable platform is provided with a shooting device, and the remote control device is communicatively connected with the movable platform.
A system, characterized in that it includes the electronic device according to any one of claims 22 to 24 and a movable platform; the movable platform is provided with a shooting device;

The electronic device is used to control the movement of the movable platform according to the shooting guidance, so that the shooting device shoots images and/or videos related to the missing lens.
The system of claim 25, wherein the movable platform includes at least one of the following: an unmanned aerial vehicle, an unmanned vehicle, a cloud platform, an unmanned vessel or a mobile robot.
A computer-readable storage medium, characterized in that the computer-readable storage medium stores executable instructions, and when the executable instructions are executed by a processor, the method according to any one of claims 1 to 21 is implemented.