WO2023231616A9 - 一种拍摄方法和电子设备 - Google Patents

一种拍摄方法和电子设备 Download PDF

Info

Publication number
WO2023231616A9
WO2023231616A9 PCT/CN2023/088623 CN2023088623W WO2023231616A9 WO 2023231616 A9 WO2023231616 A9 WO 2023231616A9 CN 2023088623 W CN2023088623 W CN 2023088623W WO 2023231616 A9 WO2023231616 A9 WO 2023231616A9
Authority
WO
WIPO (PCT)
Prior art keywords
image
recording
window
module
video
Prior art date
Application number
PCT/CN2023/088623
Other languages
English (en)
French (fr)
Other versions
WO2023231616A1 (zh
Inventor
王枘
冯文瀚
Original Assignee
荣耀终端有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 荣耀终端有限公司 filed Critical 荣耀终端有限公司
Priority to EP23814802.7A priority Critical patent/EP4391570A1/en
Publication of WO2023231616A1 publication Critical patent/WO2023231616A1/zh
Publication of WO2023231616A9 publication Critical patent/WO2023231616A9/zh

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
    • H04N23/631—Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
    • H04N23/632—Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724—User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72403—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
    • H04M1/7243—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality with interactive means for internal management of messages
    • H04M1/72439—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality with interactive means for internal management of messages for image or video messaging
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/61—Control of cameras or camera modules based on recognised objects
    • H04N23/611—Control of cameras or camera modules based on recognised objects where the recognised objects include parts of the human body
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
    • H04N23/631—Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
    • H04N23/633—Control of cameras or camera modules by using electronic viewfinders for displaying additional information relating to control or operation of the camera
    • H04N23/635—Region indicators; Field of view indicators
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/67—Focus control based on electronic image sensor signals
    • H04N23/675—Focus control based on electronic image sensor signals comprising setting of focusing regions
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/76—Television signal recording
    • H04N5/765—Interface circuits between an apparatus for recording and another apparatus
    • H04N5/77—Interface circuits between an apparatus for recording and another apparatus between a recording apparatus and a television camera
    • H04N5/772—Interface circuits between an apparatus for recording and another apparatus between a recording apparatus and a television camera the recording apparatus and the television camera being placed in the same enclosure

Definitions

  • the present application relates to the field of terminals, and in particular, to a photographing method and electronic equipment.
  • terminal devices such as mobile phones and tablets are usually equipped with multiple cameras. Provides multiple shooting modes for photo and video functions. For example, front camera mode, rear camera mode, front and rear dual camera mode, etc.
  • users can use terminal devices to record videos of one or more characters.
  • the video can be manually edited.
  • the manual editing method is cumbersome and requires the shooting user to have certain video editing skills, resulting in a poor user experience.
  • this application provides a shooting method, which is applied to an electronic device with a camera and a screen.
  • the method includes: displaying a preview window on the screen, displaying an image collected by the camera in the preview window, and the image displayed in the preview window includes One or more first objects; selecting a second object among the one or more first objects; while displaying the preview window on the screen, the first window is also displayed, and the image displayed in the first window includes the second object; recording Preview the image in the window and save it as the first video, record the image in the first window and save it as the second video.
  • the electronic device can identify the objects in the images collected by the camera and select the focus object, that is, the protagonist. Then, the electronic device can display another window on the screen, that is, the first window, for displaying the protagonist image. The electronic device can record and save the image in the preview window and the image in the first window respectively. In this way, users can either take the original image displayed in the preview window or automatically obtain a close-up image focusing on the protagonist.
  • selecting a second object among one or more first objects includes: displaying a selection box on one or more first objects; responding to the first selection The user operation on the frame determines the first object corresponding to the first selection frame as the second object.
  • the electronic device can display a selection box corresponding to each object after recognizing the objects in the image. In this way, the user can determine each object recognized by the electronic device through the above selection box, and select any one of the above objects as the focus object through the above selection box.
  • the method further includes: no user operation acting on the screen is detected within the first period of time, and the selection box disappear.
  • the electronic device can detect interactive events on the screen after displaying the selection box. If no interactive events on the screen are detected within a preset time, such as within the first time period, the electronic device The device can hide the selection box. This can not only reduce the occlusion of the image displayed on the screen by the selection box, but also reduce the calculation cost required to draw the image and save power consumption.
  • the second object is determined by the electronic device.
  • the electronic device can also determine the focus object from among the recognized objects.
  • object evaluation rules may be preset in the electronic device. The electronic device can evaluate each object in the recognized image according to the above rules, and then determine the object with the highest evaluation as the focus object.
  • a recording control is also displayed on the screen. After the first window is displayed on the screen, the method further includes: receiving a user's recording operation corresponding to the recording control and starting recording at the same time. The first video and the second video; the first video and the second video start recording at the same time.
  • the user can click the recording control to control the electronic device to start recording after determining the focus object.
  • the screen includes a preview window and a first window.
  • the electronic device can start recording the image in the preview window and the image in the first window at the same time.
  • a recording control is also displayed on the screen. Before selecting the second object among one or more first objects, the method further includes: receiving the user's response corresponding to the recording control.
  • the recording operation the first video is started to be recorded; while the first window is displayed, the second video is started to be recorded; the start recording time of the second video is later than the start recording time of the first video.
  • the user can click the recording control to start recording before selecting the focus object.
  • the preview window is included in the screen but the first window is not displayed.
  • the electronic device first records the image in the preview window. After the user determines the focus object and displays the first window, the electronic device can automatically start recording the image in the first window.
  • the recording end time of the first video and the second video are the same; or, the end recording time of the second video is earlier than the end recording time of the first video.
  • the user can end recording the image of the first window first, or can end recording the images of the first window and the preview window at the same time.
  • the method further includes: changing a second object from one or more first objects; the image displayed in the first window includes the changed second object. Two objects.
  • the user can switch the focus object at any time during the recording process.
  • the electronic device can focus on the newly selected focus object and display a close-up image of the newly selected focus object in the first window.
  • the method further includes: when it is detected that the image collected by the camera does not include the second object and the duration is longer than the second duration, suspending the recording of the second object. An image in a window.
  • the electronic device when it is recognized that the image collected by the camera does not include the previously selected focus object, the electronic device can start a timer.
  • the image displayed in the first window can stay at the frame where the timer ends.
  • the electronic device can pause recording the image in the first window.
  • the method further includes: when it is detected that the image collected by the camera does not include the second object, suspending the recording of the image in the first window.
  • the image displayed in the first window may stay at the last frame including the focus object.
  • the electronic device can pause recording the image in the first window. That is, after losing focus, the electronic device can immediately pause recording the image in the first window. Because, at this time, the close-up image of the focus object is no longer updated in the first window. In this way, when the protagonist disappears from the viewing range, the electronic device can automatically pause the recording of the close-up image, thereby obtaining a purer close-up video that only includes the protagonist.
  • the method further includes: when it is detected that the image collected by the camera includes the second object, resume recording the image in the first window.
  • the electronic device after pausing to record the image in the first window due to out-of-focus, if it is re-identified that the image collected by the camera includes the previously selected focus object, the electronic device can continue to record the image in the first window. Displays a close-up image of the focus subject and restores the image in the first window of recording. In this way, when the protagonist reappears within the viewing range, the electronic device can automatically continue to focus on the protagonist without user operation.
  • the first window and the preview window are displayed in separate screens, or the first window is displayed above the preview window but does not completely cover the preview window.
  • the user can simultaneously browse the original image displayed in the preview window and the close-up image including the protagonist displayed in the first window during the video recording process. Furthermore, users can adjust the camera position at any time to capture original videos and close-up videos that better suit their personal needs.
  • the image including the second object displayed in the first window is cropped based on the image displayed in the preview window.
  • the electronic device includes a function module, an object detection module and a large window control module.
  • Displaying a selection box on one or more first objects includes: the object detection module collects data from a camera. One or more first objects are recognized in the image; the object detection module sends the characteristic information of one or more first objects to the functional module, and the characteristic information includes a center point indicating the position of the first object; one first object corresponds to a Group characteristic information.
  • the function module determines the display coordinates based on the center point in the feature information; the large window control module displays a selection box corresponding to one or more first objects on the screen based on the display coordinates.
  • no user operation on the screen is detected within the first period of time, and the selection box disappears, including: after displaying the selection box, the function module sets a first timer, The timing duration of a timer is the first duration; after the first timer ends and the function module does not detect the user operation on the screen, the large window control module clears the selection box.
  • the electronic device further includes a focus tracking module and a small window control module to display the first window, including: a functional module sending characteristic information of the second object to the focus tracking module; focus tracking The module determines whether the second object is included in the image to be displayed based on the characteristic information of the second object; when the second object is included, the focus tracking module sends a focus tracking success instruction to the function module, and the function module controls the small window control module according to the instruction.
  • the first window is displayed on the screen;
  • the method further includes: when a second object is included, the focus tracking module crops out the image displayed in the first window from the image to be displayed.
  • the electronic device further includes a start recording module, receives the user's recording operation corresponding to the recording control, and starts recording the first video and the second video at the same time, including: the start recording module receives to the user's recording operation corresponding to the recording control; in response to the recording operation, the start recording module turns on the first encoder and the second encoder, the first encoder starts encoding the image in the preview window, and the second encoder starts encoding the first window images in .
  • the electronic device further includes a pause recording module.
  • a pause recording module When it is detected that the image collected by the camera does not include the second object and the duration is longer than the second duration, the recording of the first window is paused.
  • the image in the image includes: when it is recognized that the image to be displayed does not include the second object, the focus tracking module sends an out-of-focus instruction to the function module, and the function module sets a second timer according to the out-of-focus instruction, and the second timer The timing duration is the second duration; when the second timer ends and the function module has not received the focus tracking success indication, the pause recording module controls the second encoder to pause encoding of the image displayed in the first window.
  • resuming recording the image in the first window includes: when it is recognized that the image to be displayed includes the second object.
  • the focus tracking module sends a focus tracking success indication to the function module.
  • the function module sends a recovery recording instruction to the start recording module based on the focus tracking success indication.
  • the start recording module reopens the second encoder based on the recovery recording instruction and continues to encode the first The image displayed in the window.
  • the method further includes: displaying a thumbnail of the first video and a thumbnail of the second video in the gallery, and the thumbnail of the second video contains the second object.
  • the thumbnail of the second video also contains the first object, and the image content of the first object in the thumbnail of the second video is consistent with the image content of the first object in the thumbnail of the first video.
  • the electronic device can display the first video obtained by recording the image in the preview window and the second video obtained by recording the first window in the gallery application. In this way, the user can arbitrarily choose to browse the first video or the second video.
  • the thumbnail of the first video is displayed adjacent to the thumbnail of the second video.
  • the first video and the second video recorded at the same time can be displayed adjacently. Users can clearly and intuitively obtain the first video and the second video recorded at the same time.
  • the camera includes a first camera and a second camera.
  • the image collected by the first camera displayed in the preview window is the second image collected by the second camera focusing on the second object. .
  • the electronic device can also focus on the protagonist through different cameras, thereby obtaining a close-up image that is different from the content of the protagonist image displayed in the preview window.
  • the present application provides an electronic device, which includes one or more processors and one or more memories; wherein one or more memories are coupled to one or more processors, and one or more
  • the memory is used to store computer program code.
  • the computer program code includes computer instructions.
  • embodiments of the present application provide a chip system, which is applied to an electronic device.
  • the chip system includes one or more processors, and the processor is used to call computer instructions to cause the electronic device to execute the first step. aspect and the method described in any possible implementation manner in the first aspect.
  • the present application provides a computer-readable storage medium, including instructions.
  • the above instructions When the above instructions are run on an electronic device, the above electronic device causes the above-mentioned electronic device to execute as described in the first aspect and any possible implementation manner of the first aspect. method.
  • the present application provides a computer program product containing instructions.
  • the computer program product When the computer program product is run on an electronic device, the electronic device causes the electronic device to execute as described in the first aspect and any possible implementation manner of the first aspect. method.
  • the electronic device provided by the second aspect the chip system provided by the third aspect, the computer storage medium provided by the fourth aspect, and the computer program product provided by the fifth aspect are all used to execute the method provided by this application. Therefore, the beneficial effects it can achieve can be referred to the beneficial effects in the corresponding methods, and will not be described again here.
  • 1A-1J are schematic user interface diagrams of a set of protagonist mode shooting methods provided by embodiments of the present application.
  • 2A-2I are schematic user interface diagrams of a set of protagonist mode shooting methods provided by embodiments of the present application.
  • Figures 3A-3C are schematic diagrams of a set of user interfaces for browsing captured images provided by embodiments of the present application.
  • Figure 4 is a schematic system structure diagram of a terminal 100 provided by an embodiment of the present application.
  • Figure 5 is an interaction flow chart of a software module of a terminal 100 provided by an embodiment of the present application.
  • Figure 6 is a schematic diagram of a terminal 100 determining object feature information of a frame of image provided by an embodiment of the present application
  • Figure 7 is an interaction flow chart of a software module of another terminal 100 provided by an embodiment of the present application.
  • Figure 8 is an interaction flow chart of a software module of another terminal 100 provided by an embodiment of the present application.
  • Figure 9 is an interaction flow chart of a software module of another terminal 100 provided by an embodiment of the present application.
  • Figure 10 is an interaction flow chart of a software module of another terminal 100 provided by an embodiment of the present application.
  • Figure 11 is an interaction flow chart of a software module of another terminal 100 provided by an embodiment of the present application.
  • Figure 12 is a schematic diagram of the hardware structure of the terminal 100 provided by the embodiment of the present application.
  • terminal devices such as mobile phones and tablets are usually equipped with multiple cameras. Provides multiple shooting modes for photo and video functions. For example, front camera mode, rear camera mode, front and rear dual camera mode, etc.
  • users can use terminal devices to record videos of one or more characters.
  • the video can be manually edited.
  • the manual editing method is cumbersome and requires the shooting user to have certain video editing skills, resulting in a poor user experience.
  • embodiments of the present application provide a shooting method, in which the terminal device can provide a shooting function in protagonist mode.
  • the terminal device can generate an additional video for the protagonist without manual editing, improving the user's shooting experience.
  • the terminal 100 can generate two videos at the same time, recorded as the original video and the close-up video.
  • the original video is composed of original images collected by the camera.
  • the close-up video is obtained by identifying the protagonist in the image based on the original image, and then cropping the protagonist image.
  • the terminal 100 can simultaneously display the original video and the close-up video for the user to preview.
  • users can also select multiple protagonists.
  • the terminal 100 can locate the multiple protagonists based on the original images collected by the camera, and then generate multiple close-up videos corresponding to each protagonist.
  • users can not only take close-up videos centered on the protagonist, but also obtain original videos composed of original images collected by the original camera, further meeting user personalized needs and improving user experience.
  • terminal 100 can be used uniformly to refer to the above-mentioned terminal equipment.
  • the terminal 100 can also be a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, or a personal digital assistant.
  • PDA personal digital assistant
  • AR augmented reality
  • VR virtual reality
  • AI artificial intelligence
  • FIG. 1A exemplarily shows the user interface of the terminal 100 enabling the camera to perform a shooting action.
  • the user interface may include a menu bar 111 , a shooting control 112 , a preview window 113 , and a review control 114 .
  • the menu bar 111 may display multiple shooting mode options, such as night scene, video, photo, portrait and other shooting modes.
  • Night scene mode can be used to take photos in dark scenes, such as taking photos at night.
  • Video mode can be used to record videos.
  • Photo mode can be used to take photos in daylight scenes.
  • Portrait mode can be used to take close-up photos of people.
  • the terminal 100 may first enable the video recording mode and prepare to start recording video. Of course, the terminal 100 can first enable other shooting modes such as taking photos and portraits. At this time, the terminal 100 can enter the video recording mode according to user operations.
  • the shooting control 112 may be used to receive a user's shooting operation.
  • a photographing scene including photographing mode, portrait mode, and night scene mode
  • the above-mentioned photographing operation is an operation of controlling photographing that acts on the photographing control 112 .
  • the scene of recording video video recording mode
  • the above-mentioned shooting operation includes an operation of starting recording on the shooting control 112 .
  • the preview window 113 can be used to display the image stream collected by the camera in real time. At any time, a frame of image displayed in the preview window 113 is a frame of original image.
  • the review control 114 can be used to view photos or videos taken previously. Generally, the review control 114 can display a thumbnail of a photo taken previously or a thumbnail of the first frame of a video taken previously.
  • the user interface shown in FIG. 1A may also include a function bar 115.
  • the function bar 115 may include multiple functional controls, such as flash 1151, filter 1152, settings 1153, protagonist mode 1154 and other functional controls.
  • the flash 1151 can be used to turn on or off the flash, thereby changing the brightness of the image captured by the camera.
  • the filter 1152 can be used to select a filter style, thereby adjusting the image color in the preview window 113 .
  • Setting 1153 can be used to provide more controls for adjusting camera shooting parameters or image optimization parameters, such as white balance controls, ISO controls, beauty controls, body beauty controls, etc., thereby providing users with richer shooting services.
  • Protagonist mode 1154 may be used to provide a function of shooting a close-up video centered on the protagonist.
  • the terminal 100 can select and change the protagonist according to user operations, shoot and save at least two videos: a close-up video centered on the protagonist, and a video collected based on the camera. Original video generated from original image. Among them, there can be multiple close-up videos centered on the protagonist.
  • the terminal 100 may determine one or more protagonists according to user operations. A protagonist can correspond to one or more close-up videos.
  • the terminal 100 may detect a user operation on the protagonist mode 1154 to enable the protagonist mode shooting function.
  • the above-mentioned user operation that acts on the protagonist mode 1154 is, for example, the operation of clicking the protagonist mode 1154 .
  • the terminal 100 can execute the algorithm corresponding to the protagonist mode and enter the protagonist mode shooting scene, see FIG. 1B .
  • FIG. 1B exemplarily shows a user interface for shooting after the terminal 100 enters the protagonist mode to shoot a scene.
  • the terminal 100 can perform image recognition on the image collected by the camera and identify objects included in the image (ie, object recognition).
  • objects include but are not limited to humans, animals, and plants.
  • the following description of the embodiments of this application will mainly take humans as an example.
  • the image collected by the camera of the terminal 100 includes Person 1 and Person 2.
  • the terminal 100 may first identify objects included in the image using a preset object recognition algorithm.
  • the above-mentioned object recognition algorithm may be a human body detection algorithm. It can be understood that the terminal 100 also supports identifying animals and plant-type objects.
  • the above-mentioned object recognition algorithm also includes a recognition algorithm for one or more animals, and a recognition algorithm for one or more plants, which are not limited in the embodiments of the present application.
  • the terminal 100 can recognize that the above image includes two objects, Person 1 and Person 2.
  • the terminal 100 may display the above image in the preview window 113 . Based on the objects included in the recognized image, while displaying the above image, the terminal 100 may also display a selection box corresponding to each of the above objects. For example, the terminal 100 may display the selection box 121 corresponding to the character 1 on the character 1, and display the selection box 122 corresponding to the character 2 on the character 2. At this time, on the one hand, the user can confirm through the above selection box that the terminal 100 has detected an object for the user to select; on the other hand, the user can click on any one of the above selection boxes to set the corresponding object as main character.
  • the terminal 100 can also display a prompt 125 in the preview window 113, such as "Please select the protagonist and enable automatic focus recording" to prompt the user to select the protagonist.
  • a prompt 125 in the preview window 113, such as "Please select the protagonist and enable automatic focus recording" to prompt the user to select the protagonist.
  • the preview window 113 may include a beauty control 127 .
  • the beauty control 127 can be used to adjust the face image of the person in the image.
  • the terminal 100 can perform beautification processing on the characters in the image, and display the beautified image in the preview window.
  • the user interface shown in FIG. 1B may also display other shooting controls, such as a focus control for adjusting the camera, etc., which are not listed here one by one.
  • the terminal 100 can detect a user operation on any selection box. In response to the above operation, the terminal 100 may determine that the object corresponding to the above selection box is the protagonist. For example, referring to the user interface shown in FIG. 1C , the terminal 100 may detect a user operation on the selection box 122 . In response to the above operation, the terminal 100 may determine that the character 2 corresponding to the selection box 122 is the protagonist.
  • the terminal 100 may display a small window in the preview window 113 in a picture-in-picture form (embedded), and display a close-up image of the character 2 in the small window.
  • the above close-up image refers to an image obtained by cropping the selected protagonist as the center based on the original image collected by the camera (the image displayed in the preview window).
  • the terminal 100 may also divide the display area corresponding to the original preview window 113 into two parallel parts: one part is used to display the image collected by the camera; the other part is used to display the close-up image based on the image collected by the camera.
  • FIG. 1D exemplarily shows a user interface in which the terminal 100 displays a small window and displays a close-up image of the character 2 in the small window.
  • the preview window 113 may include a small window 141 .
  • a close-up image of the character 2 can be displayed in the small window 141 .
  • the close-up image of the character 2 displayed in the small window 141 will also change accordingly.
  • the continuous close-up images centered on the character 2 displayed in the small window 141 constitute a close-up video of the character 2 .
  • the close-up image displayed in the small window 141 and the original image displayed in the preview window 113 may also come from different cameras.
  • the close-up image displayed in the small window 141 may be from an image collected by a telephoto camera
  • the original image displayed in the preview window 113 may be from an image collected by a wide-angle camera.
  • Regular cameras and wide-angle cameras can capture images at the same time.
  • the images collected by the telephoto camera and the wide-angle camera can be corresponding through timestamps.
  • the close-up image displayed in the small window 141 is cropped from the image collected by the ordinary camera, and the image collected by the ordinary camera corresponds to the image collected by the wide-angle camera displayed in the preview window 113 at the same time.
  • the user can browse a larger range of landscape in the preview window 113, and at the same time, a more detailed protagonist image is displayed in the small window 141.
  • the above-mentioned wide angle and telephoto are only examples.
  • the image displayed in the small window 141 can also come from any camera such as a wide-angle camera, an ultra-wide-angle camera, a black-and-white camera, a multi-spectral camera, etc.
  • the image displayed in the same preview window 113 can also come from any camera such as ultra-wide-angle camera, telephoto camera, black and white camera, multi-spectral camera, etc.
  • the selection box 122 corresponding to character 2 can be changed to the selection box 142 in Figure 1D.
  • the user can distinguish between protagonist and non-protagonist objects by checking box 142 . It is not limited to the check box 142 shown in the user interface.
  • the terminal 100 can also display other styles of icons or use different colors, which is not limited in the embodiment of the present application.
  • the user can also select multiple protagonists.
  • the terminal 100 can also detect the user operation on the selection box 121. In response to the above operation, the terminal 100 can determine that the character 1 corresponding to the selection box 121 is another protagonist. In some examples, the terminal 100 may display a small window corresponding to a close-up image of the character 1 . In other examples, in order to avoid causing a large obstruction to the preview window 113, when there are multiple protagonists, the terminal 100 may only display a small window showing a close-up image of the protagonist. Corresponding to the selected multiple protagonists, the terminal 100 can generate multiple close-up videos.
  • the small window 141 for displaying close-up images may also include a close control 143 and a transpose control 144.
  • the close control 143 can be used to close the small window 141.
  • the transpose control can be used to resize the small window 141.
  • the terminal 100 may detect a user operation to close the control 143 .
  • the terminal 100 may close the small window 141, refer to FIG. 1F.
  • the terminal 100 can cancel the previously selected protagonist (Character 2).
  • the check box 142 corresponding to the character 2 can be changed to the check box 122 .
  • the user can reselect any object recognized in the preview window 113 as the protagonist.
  • the terminal 100 may display the small window 141 in the preview window 113 again based on the redetermined protagonist. At this time, a close-up image obtained by processing the original image with the new protagonist as the center is displayed in the small window 141 .
  • the terminal 100 may detect a user operation on the transpose control 144 . In response to the above operation, the terminal 100 may adjust the vertical small window 141 in FIG. 1F to a horizontal orientation, refer to FIG. 1H.
  • the terminal 100 may first generate a small window (vertical window) with an aspect ratio of 9:16 for displaying a close-up image, refer to the small window 141 in FIG. 1D.
  • the above aspect ratio is an exemplary example, and the aspect ratio of vertical windows includes but is not limited to 9:16.
  • the terminal 100 may permanently display the above-mentioned small window 141 at the lower left (or lower right, upper left, or upper right) of the screen. After detecting a user operation on the transpose control 144, the terminal 100 can change the original vertical window into a small horizontal window (horizontal window) with an aspect ratio of 16:9.
  • the terminal 100 can also generate a horizontal window by default, and then adjust the horizontal window to a vertical window according to user operations.
  • This embodiment of the present application does not limit this. In this way, the user can adjust the video content and video format of the close-up video through the transpose control 144 to meet their own personalized needs.
  • the above-mentioned small window can also adjust its display position according to the position of the protagonist in the preview window 113 to avoid blocking the protagonist in the preview window 113 .
  • the terminal 100 can also adjust the position and size of the small window according to user operations.
  • the terminal 100 may also detect a long press operation and a drag operation acting on the small window 141. In response to the above operation, the terminal 100 may move the small window to the position where the user's drag operation last stopped.
  • the terminal 100 may also detect a double-click operation on the small window 141, and in response to the above operation, the terminal 100 may enlarge or reduce the small window 141.
  • the terminal 100 can also control and adjust the position and size of the small window through gesture recognition and voice recognition. For example, the terminal 100 can recognize that the user has made a fist gesture through the image collected by the camera, and in response to the fist gesture, the terminal 100 can reduce the small window 141 .
  • the terminal 100 can recognize that the user has made a hand-opening gesture through the image collected by the camera. In response to the hand-opening gesture, the terminal 100 can enlarge the small window 141 .
  • the terminal 100 may also set the above object as the protagonist by default. For example, referring to the user interface shown in FIG. 1B , within 5 seconds, the identified objects displayed in the preview window 113 that can be used as the protagonist are only character 2. At this time, the terminal 100 can set the character 2 by default. as the protagonist. In this way, in a typical single-person shooting scene, the terminal 100 can intelligently determine the protagonist, thereby saving user operations.
  • the terminal 100 may also select an optimal object from the plurality of objects as the protagonist according to preset evaluation rules.
  • the above evaluation rules such as the size of the character image, the image brightness of the face area, etc., are not listed here one by one.
  • the close-up image of the protagonist displayed in the small window 141 will be frozen at the last image before the loss. frame.
  • the image displayed in the preview window 113 may be the nth frame image collected by the camera.
  • the image displayed in the small window 141 is a close-up image centered on the protagonist (Character 2) obtained based on the n-th frame image.
  • the image displayed in the preview window 113 may be the n+1th frame image collected by the camera.
  • the n+1th frame image does not include the previously selected protagonist (Character 2), that is, the protagonist is lost.
  • the close-up image centered on the main character (Character 2) obtained based on the n-th frame image is kept displayed in the small window 141 .
  • the terminal 100 can also display a prompt 151 in the preview window 113, such as "The protagonist is lost, exit focus tracking after 5 seconds" to prompt the user to adjust the camera position or angle. , so that the terminal 100 can reacquire the original image including the protagonist.
  • the terminal 100 can close the small window 141, and cancel the previously selected protagonist (Character 2).
  • the above 5 seconds is preset, and the terminal 100 can also set other time lengths, such as 10 seconds, etc., which are not limited in the embodiments of the present application.
  • the terminal 100 may start recording the video.
  • the terminal 100 can generate an original video based on the original image displayed in the preview window 113.
  • the terminal 100 can also generate a close-up video based on the close-up image of the protagonist in the small window 141.
  • the terminal 100 may detect a user operation on the shooting control 112 .
  • the above operation may be called a user operation to start shooting (start recording).
  • the shooting control 112 is also called a recording control.
  • the terminal 100 may start recording videos (original video and close-up video), that is, encoding and saving the original image collected by the camera and the close-up image centered on the protagonist. At this time, the recording start moments of the original video and the close-up video are the same.
  • the above-mentioned user operation to start shooting may also be a preset user body movement or voice instruction.
  • the user's body movements include gesture movements (such as raising hands and making fists) and body movements (such as jumping and running).
  • the terminal 100 may display the control module 211 .
  • the control module 211 may include a pause control 2111 and a stop control 2112.
  • the pause control 2111 can be used to pause recording videos, including pausing the original video corresponding to the preview window 113 and pausing the close-up video corresponding to the small window 141.
  • the stop control 2112 can be used to stop recording video, including stopping recording original video, and stopping recording close-up video.
  • the timestamp can be displayed in both the preview window 113 and the small window 141 .
  • the timestamp "00:01” is displayed in the upper left corner of the preview window 113
  • the timestamp "00:01” is displayed in the upper left corner of the small window 141.
  • the timestamps in the preview window 113 and the small window 141 are the same.
  • the timestamps in the preview window 113 and the small window 141 may be different, which will not be described here.
  • the terminal 100 can also determine the protagonist after starting to record the video, generate the small window 141 and display the close-up video of the protagonist. At this time, the terminal 100 first detects the user operation of starting recording, and then detects the operation of clicking a certain selection box to select the corresponding object as the protagonist.
  • the video corresponding to the above-mentioned detected user operation to start recording is the recording start time of the original video.
  • the above-mentioned operation of clicking a certain selection box to select the corresponding object as the protagonist corresponds to the moment when the close-up video starts recording. Therefore, in the above recording scenario, the start recording moment of the close-up video is after the original video.
  • a stop control 212 may also be displayed in the small window 141. Stop control 212 can be used to stop recording the close-up video. After the detected user operation on the stop control 212 is detected, the terminal 100 can close the small window 141 and stop recording the close-up video corresponding to the small window 141 . After stopping recording the small window video, the terminal 100 may generate a close-up video based on the recorded image in the small window. At this time, the terminal 100 does not stop recording the original video.
  • the user can reselect the protagonist.
  • the terminal 100 can re-display the small window 141, display the new protagonist's close-up video in the small window 141, and record the new protagonist's close-up video.
  • the above-mentioned new protagonist can be the same as the previously selected protagonist, or it can be different.
  • the terminal 100 will save the recorded image in the small window as another close-up video.
  • a pause control 213 can also be displayed in the small window 141.
  • the terminal 100 may pause updating the image displayed in the small window 141 and also pause recording the image in the small window 141 (ie, pause recording the close-up video).
  • the terminal 100 may display a continue recording control (not shown in FIG. 2B) instead of the pause control 213.
  • the terminal 100 can resume updating the image displayed in the small window 141, resume recording the close-up video, and display the pause control 213 in the small window 141 to replace the above-mentioned continue recording control.
  • the terminal 100 can also monitor user operations on the screen. On the basis of displaying the user interface shown in Figure 2B, when no user operation, such as a click operation, on the preview window 113 on the screen is detected within 5 seconds, the terminal 100 can hide the selection box in the preview window 113; when 5 seconds When no user operation on the small window 141 on the screen is detected within seconds, the terminal 100 can hide the functional controls in the small window 141, such as the time stamp of the small window and the stop control 212.
  • the terminal 100 may also provide a service of switching protagonists.
  • the terminal 100 may detect the user operation on the selection box 121 .
  • the above operation can be called a user operation for switching the protagonist.
  • the terminal 100 may set the character 1 corresponding to the selection box 121 as the protagonist.
  • character 2 who was previously set as the protagonist, is no longer the protagonist.
  • the terminal 100 may display the close-up image of character 1 in the small window 141 and no longer display the close-up image of character 2 .
  • the terminal 100 can update the selection box 121 corresponding to the character 1 to the selection box 221, and at the same time, update the selection box 142 corresponding to the character 2 to the selection box 122.
  • the small window 141 can directly display the close-up image of the switched character 1, presenting a jumping display effect.
  • the small window 141 can also achieve a non-jumping protagonist switching display effect through a smoothing strategy. For example, after switching the protagonist to character 1, the terminal 100 can determine a set of smoothly moving image frames based on the path from character 2 to character 1 in the preview window 113, and then display the above image frames in the small window 141 to achieve non-stop operation. Jumping protagonist switching display.
  • the terminal 100 can also use a fixed transition effect to connect the close-up images of the protagonist before and after switching.
  • the above-mentioned fixed transition effects include overlay, vortex, translation, etc. commonly used in video editing. The embodiments of the present application do not limit this.
  • the protagonist mode can also be preset with an automatic protagonist selection algorithm.
  • the protagonist automatic selection algorithm can automatically determine the best-performing object as the protagonist from the multiple identified objects according to the above object evaluation rules.
  • the terminal 100 can also switch the protagonist according to changes in the scores of each object. For example, when recording a video at the beginning, character 2 has the highest evaluation. At this time, the terminal 100 can determine that character 2 is the protagonist. Subsequently, the terminal 100 can detect that character 1 has a higher evaluation than character 2. At this time, the terminal 100 The protagonist can be changed from character 2 to character 1.
  • the protagonist initially selected by the user may leave the viewing range of the camera of the terminal 100 .
  • the original image corresponding to the preview window 113 may also lose the main character.
  • the terminal 100 may also display a prompt that the protagonist is missing, and freeze the close-up image of the protagonist in the small window 141 .
  • the terminal 100 can detect that the original image (the original image collected by the camera) displayed in the preview window 113 includes person 2, but does not include person 1 ( Protagonist), that is, the protagonist is lost. At this time, the terminal 100 can display the prompt 231 ("The protagonist is lost, exit focus tracking after 5 seconds") in the preview window 113 to prompt the user to adjust the camera position or angle, so that the terminal 100 can reacquire the original image including the protagonist. . At the same time, the terminal 100 can keep displaying in the small window 141 the close-up image of the main character (Character 1) determined at the previous moment.
  • the prompt 231 The protagonist is lost, exit focus tracking after 5 seconds
  • the close-up image displayed in the small window 141 is paused and fixed on the close-up image of the protagonist (Character 1) determined at the previous moment.
  • the timestamp displayed in the upper left corner of the small window 141 is suspended.
  • the user can adjust the camera position so that the protagonist is within the viewing range of the camera, so that the camera can re-collect images including the protagonist.
  • the terminal 100 may display prompt 232 ("Protagonist focus has been paused"). At the same time, the terminal 100 can display a translucent gray mask on the layer where the small window 141 displays the close-up image to prompt the user that the focus recording has been paused.
  • the terminal 100 has stopped recording the close-up video in the small window 141.
  • the 5 seconds from the 10th second to the 15th second are the transition time set for the terminal 100 for user adjustment.
  • the terminal 100 can also crop the subsequent image in the preview window 113 to obtain a close-up based on the last frame before the protagonist is lost. image. At this time, the above image may not include the protagonist.
  • the terminal 100 If at some point after the recording of the close-up video is paused, the terminal 100 re-identifies the protagonist (Character 1), then the terminal 100 can display the newly acquired close-up image of the protagonist (Character 1) in the small window 141, and continue Record a close-up video.
  • the camera re-captured an image including person 1, that is, the image displayed in the preview window 113 included person 1 again.
  • the terminal 100 may determine a close-up image centered on the character 1 based on the original image including the character 1 , and then display the close-up image in the small window 141 .
  • the timestamp displayed in the upper left corner of the small window 141 resumes timing.
  • the terminal 100 continues to encode the above close-up image, that is, continues to record the close-up video.
  • the terminal 100 may detect a user operation to end the shooting.
  • a user operation to end the shooting Referring to the user interface shown in FIG. 2H, for example, at the 25th second after starting to record the video, the terminal 100 may detect a user operation on the stop control 2112. The above-mentioned user operation may be called a user operation to end shooting.
  • the terminal 100 may stop encoding images, and encapsulate the encoded images from the beginning of recording to the end of recording as videos and save them in the local storage.
  • the terminal 100 may stop encoding the original image corresponding to the preview window 113, and encapsulate the original image that has been encoded from the beginning of recording to the end of recording as an original video and save it in the local storage.
  • the terminal 100 can stop encoding the close-up image corresponding to the small window 141, and encapsulate the encoded close-up image from the beginning of recording to the end of recording as a close-up video and save it in the local memory.
  • the terminal 100 may display the user interface shown in Figure 2I. As shown in FIG. 2H , the terminal 100 may redisplay the shooting control 112 and the review control 114 . At this time, thumbnails indicating the above-mentioned recorded original video and close-up video may be displayed on the review control 114 .
  • the above identification may be a thumbnail of the first frame image of the above original video, or a thumbnail image of the first frame image of the above close-up video.
  • the terminal 100 may detect a user operation on the lookback control 114 . In response to the above operations, the terminal 100 may play the captured original video and/or close-up video. In this way, the user immediately views the above-mentioned original video and/or close-up video.
  • FIG. 3A exemplarily shows a user interface provided by a gallery application on the terminal 100 for displaying locally saved videos and/or pictures.
  • the gallery application (or photo album application) is an application on the terminal 100 that manages locally saved videos and/or pictures.
  • the user interface may display multiple thumbnail icons.
  • a thumbnail icon corresponds to a video or picture obtained by a shooting operation.
  • the plurality of thumbnail icons may include icon 313.
  • the icon 313 may correspond to the video generated by the aforementioned shooting operation shown in FIGS. 2A-2I.
  • the terminal 100 may detect user operations on the icon 313 . In response to the above operations, the terminal 100 may display the videos captured by the aforementioned shooting operations shown in FIGS. 2A-2I: original videos and close-up videos, refer to FIG. 3B.
  • the user interface may include window 322 and window 323.
  • the window 322 can be used to display the original video captured by the shooting operation shown in FIGS. 2A to 2I; the window 323 can be used to display the close-up video captured by the shooting operation shown in FIGS. 2A to 2I.
  • the terminal 100 when the terminal 100 displays the user interface shown in FIG. 3B, the original video and the close-up video can be played simultaneously. Among them, the video frames of the close-up video are aligned with the original video. This way, users can browse the original video and the close-up video at the same time. In some examples, the terminal 100 may also play the original video first and then play the close-up video, so that the user can browse each video individually.
  • the terminal 100 can detect user operations, such as click operations, that act on the original video or the close-up video. In response to the above operation, the terminal 100 may individually display the above video. Taking the original video as an example, after detecting a click operation on the original video, the terminal 100 can play the original video separately so that the user can browse the clearer original video with a higher resolution.
  • FIG. 3C exemplarily illustrates another user interface of the terminal 100 displaying locally saved videos and/or pictures.
  • the terminal 100 may display two thumbnail icons, such as icon 331 and icon 332 . These two thumbnail icons correspond to the original video and close-up video captured in protagonist mode.
  • icon 331 may correspond to the above-mentioned original video
  • icon 332 may correspond to the above-mentioned close-up video.
  • the terminal 100 may display the original video.
  • the terminal 100 may display the close-up video.
  • the icon 331 and the icon 332 may include the same image content, such as the protagonist 2 selected by the user. In this way, the user can intuitively and clearly determine that the video corresponding to the above two icons is a group of videos shot in the protagonist mode through the image content displayed in the image.
  • the terminal 100 can detect objects in the original images collected by the camera in real time.
  • the user can select any one of the one or more objects recognized by the terminal 100 as the protagonist at any time, and can also switch the protagonist at any time.
  • the terminal 100 Before starting recording, when the object that has been set as the protagonist is lost for more than a period of time, the terminal 100 can cancel the protagonist qualification of the above-mentioned object, and then instruct the user to re-select the protagonist. During the recording process, when the object that has been set as the protagonist is lost, the terminal 100 can pause the recording; when the protagonist is retrieved, the terminal 100 can continue recording. In this way, users can get a coherent close-up video centered on the protagonist, and the protagonist is not limited to one object.
  • the terminal 100 can save the original video and the close-up video at the same time.
  • the original video can retain all the image content collected by the camera during the recording process.
  • Close-up videos can focus on displaying the video content of the protagonist selected by the user. Users can browse or use the above-mentioned original videos or close-up videos according to different needs, thereby providing users with a richer user experience.
  • Figure 4 exemplarily shows a schematic diagram of the software structure of the terminal 100.
  • the software structure diagram shown in Figure 4 may include: camera application CameraApp and camera hardware abstraction CameraHAL.
  • the terminal 100 may adopt a layered architecture.
  • the layers communicate through software interfaces.
  • the layered architecture from top to bottom includes: application layer, application framework layer, Android runtime and system libraries, and kernel layer.
  • the camera application can be set up at the application layer.
  • the application layer can include a series of application packages, such as camera application, gallery application, phone application, etc. Among them, the embodiments of this application mainly relate to camera applications.
  • a camera application may include one or more functional modules.
  • a function module corresponds to a type of camera service provided by the camera application.
  • the video recording module is shown in Figure 4.
  • the video recording module can provide users with video recording services.
  • other functional modules can be used in the camera application.
  • the above-mentioned other functional modules also include a camera module (providing photography services), a portrait module (providing portrait photography services), a night scene module (providing nighttime photography services), and so on.
  • the video recording module may include a flash module, a filter module, a protagonist mode module, etc.
  • the flash module can be used to control the flash on or off.
  • the filter module can be used to control turning on or off filters and select filter types.
  • the protagonist mode module is used to control the shooting function of automatically focusing on the protagonist introduced in the aforementioned user interface.
  • the protagonist mode module may include a feature switch module, a function module, a large window control module, a small window control module, a start recording module, a pause recording module, and an end recording module.
  • the feature switch module records the processing logic for turning on the protagonist mode.
  • the functional module records the global processing logic of the protagonist mode shooting method.
  • the large window control module records the processing logic related to the preview window display in the protagonist mode.
  • the small window control module records the processing logic related to the small window display in the protagonist mode.
  • the start recording module records the processing logic for starting recording in protagonist mode.
  • the pause recording module records the processing logic for pausing recording in protagonist mode.
  • the end recording module records the processing logic for ending recording in protagonist mode.
  • the camera hardware abstraction can be set in the system library.
  • the camera hardware abstraction may include a frame stream module Stream, a frame preprocessing module framePreprocessing, an object detection module tackDetect, and a focus tracking module tracking.
  • the frame stream module can be used to cache the image frame stream collected by the camera.
  • the frame preprocessing module can be used to perform image processing algorithms such as beautification, body beautification, and filters on the acquired image frames.
  • the object detection module includes object recognition algorithms.
  • the object detection module executes an object recognition algorithm and can identify one or more objects included in any frame of image.
  • the focus tracking module includes a focus tracking algorithm.
  • the focus tracking module executes the focus tracking algorithm and can locate the protagonist in the subsequent image frame, so that the camera application can provide automatic focus shooting services for the protagonist.
  • the functional modules in the camera application can also be set at the application framework layer or system library, and the camera hardware abstraction can also be set at the kernel layer, etc.
  • the functional modules in the camera application are based on the camera hardware abstraction. It can be understood that the software structure shown in Figure 4 is only an optional example and should not constitute a limitation on this application.
  • the camera installed in the terminal 100 will enter the working state, collect light signals and generate images represented by corresponding electrical signals.
  • the generated images are first stored in the camera's buffer (RAW-buffer).
  • the camera can then report the currently generated image frames to CameraHAL in real time.
  • CameraHAL can process the received image frames and then send them to the corresponding area of the screen preview window for display.
  • the image sent by CameraHAL to the corresponding area of the screen preview window for display is the original image described in the embodiment of this application.
  • CameraHAL can also perform image recognition and focus tracking on the above-mentioned original image, thereby realizing Figures 1A-1J and 2A- Figure 2I.
  • FIG. 5 exemplarily shows a flow chart for selecting the protagonist and displaying a close-up image of the protagonist during the preview process shown in FIGS. 1A-1D.
  • the feature switch module detects a user operation to turn on the protagonist mode.
  • the user can instruct the terminal 100 to turn on the protagonist mode by operating on the protagonist mode 1154, and then use the protagonist mode to record a close-up video centered on the protagonist and the original video .
  • the feature switch module in CameraApp can detect the above-mentioned user operation of turning on the protagonist mode that acts on the protagonist mode 1154.
  • the feature switch module After detecting the user operation to turn on the protagonist mode, S502.
  • the feature switch module sends control information instructing to load the protagonist mode program block to the function module.
  • the function module loads the protagonist mode program block.
  • the function module loads the protagonist mode program blocks stored in the memory into the memory.
  • the processor can read the program codes in the above program blocks from the memory and execute them sequentially, thereby running the protagonist mode shooting method provided by the embodiment of the present application.
  • the function module calls the object detection interface for object recognition.
  • the function module can call the object detection interface (object detection function) to perform object recognition.
  • object detection interface can be used to call the object detection module in CameraHAL to execute the object recognition algorithm and identify objects included in the images collected by the camera.
  • the object detection module obtains a frame of original image that is about to be displayed, executes the object recognition algorithm, and determines the objects included in the image and the corresponding feature information.
  • the object detection module can obtain the current frame (the latest frame of image generated by the camera) from the RAW-buffer of the camera. Then, the object detection module can directly input the above-mentioned image frame into the object recognition algorithm, identify and output the object and object feature information in the image frame.
  • the camera can continuously collect and generate images (original image stream), and the object detection module can continuously identify objects included in each image in the original image stream.
  • the above object characteristic information includes the number of objects and characteristic information of each object.
  • the characteristic information of an object includes but is not limited to the object identifier (objectId), facial area (face), body area (body), center point (center), and tracking label (isTracking) of the object.
  • the object identifier is used to uniquely identify an object.
  • the facial area refers to the image area of an object's facial image in the original image.
  • the human body area refers to the image area of an object's facial image in the original image.
  • the center point represents the coordinates of the center position of an object image.
  • the above-mentioned center point may be the center point of the human body area or the center point of the facial area, or may be a point coordinate determined based on the center point of the human body area and the center point of the facial area.
  • the tracking tag is used to indicate whether the object is a protagonist.
  • the object recognition algorithm is executed, and the object detection module can determine the object feature information of S1.
  • the object detection module can determine the object feature information of S1. Refer to Table 1:
  • the object recognition algorithm is mainly a recognition algorithm for objects such as people, that is, a human body detection algorithm.
  • the terminal 100 may also support focusing on objects such as plants, animals, and buildings.
  • the above-mentioned object recognition algorithm also includes recognition algorithms for identifying specific plants, animals, and buildings, which will not be described again here.
  • the terminal 100 may utilize a human detection algorithm to determine one or more people included in the image. In some embodiments, the terminal 100 may also utilize a human detection algorithm and a face detection algorithm to determine one or more people included in the image. When a human body image output by the human body detection algorithm matches a human face image output by the human face detection algorithm, the terminal 100 may determine a person. In multi-person scenes, the above method can reduce the interference of overlapping character images and improve the accuracy of character recognition.
  • the function module when calling the object detection interface, can also determine the address of the RAW-buffer, and then when calling the object detection module for identification, the above address can be sent to the object detection module.
  • the function module can also obtain the image collected by the camera from the RAW-buffer after the address of the RAW-buffer, and then send the above image to the object detection module. In this way, the object detection module can directly obtain the original image from the functional module, or obtain the original image according to the address transmitted by the functional module.
  • the object detection module returns object feature information (object identification, facial area, human body area, center point, focus tracking label) to the function module.
  • the object detection module can return the above feature information to the function module.
  • the object detection module may return the object characteristic information shown in Table 1 above to the function module.
  • the function module can determine whether the latest frame of image reported by the camera (the frame of image to be displayed) includes one or more objects that can serve as protagonists based on the received object characteristic information.
  • the function module can determine that the image to be displayed includes one or more objects, and then the function module can determine according to the characteristic information. At the center point, a mark (selection box) corresponding to each object is generated. On the contrary, if the received object characteristic information indicates that there is no object in the frame of image to be displayed, the functional module can determine that there is no object in the image to be displayed, and then the functional module can determine that there is no need to generate a mark, and then the user can You cannot select any object as the protagonist.
  • the function module determines the display coordinates of the selection box of each object based on the center point in the object feature information.
  • the function module can determine the selection boxes that match the recognized objects one by one according to the center point in the object characteristic information, and determine the display coordinates of each selection box.
  • the center point of the selection box is the center point of each object in the above object characteristic information.
  • the terminal 100 when the terminal 100 displays the original image collected by the camera, it will also display selection boxes corresponding to each object in the original image, such as selection boxes 121 and 122 .
  • the center point of the above selection frame is the center point of character 1 and character 2 in the preview window 113 in Figure 1B.
  • the function module can determine the center point C1 (x1, y1) of person 1. Then, the function module may generate a selection box 121 corresponding to Person 1. In the above process, the function module can determine the display coordinate C_screen (X1, Y1) of the selection box 121 through the center point C1 (x1, y1). In the same way, the function module can determine the display coordinates of the selection boxes of other objects: the display coordinates of the selection box 122 C_screen (X2, Y2).
  • the function module sends an instruction to display the selection box to the large window control module.
  • the function module can send control information for displaying the selection box to the large window control module.
  • the above instructions can carry the display coordinates of each selection box, such as C_screen(X1,Y1), C_screen(X2,Y2), so that the large window control module can determine the display position of each selection box on the screen.
  • the large window control module draws the selection box and controls the display screen to display the selection box.
  • the large window control module may determine to draw and display the selection box. Among them, through the display coordinates carried in the above control information, the large window control module can draw a selection box corresponding to each object in the original image. Then, the large window control module can send the display data of the drawn selection box to the display, thereby displaying each of the above selection boxes on the screen.
  • the terminal 100 can display the corresponding selection box on each object in the image while displaying the image, thereby providing the user with the function of selecting the protagonist.
  • the large window control module detects the click operation of the selected user.
  • the large window control module can monitor the user's click operations. At some point after the selection box is displayed, the large window control module can detect the user's click on the screen. The large window control module can determine the coordinates (click coordinates) of the above-mentioned click operation on the screen, such as click_screen (X0, Y0).
  • the terminal 100 may detect a user operation acting on the selection box 122 . It is understandable that when the large window control module detects a click operation, it does not determine whether the click operation is an operation of clicking a selection box. The large window control module only needs to determine the click coordinates of the click operation on the screen.
  • the large window control module returns the click coordinates to the function module.
  • the function module determines the focus object based on the click coordinates.
  • the function module After receiving the click coordinates returned by the large window control module, the function module can first determine the image coordinates corresponding to the click coordinates, that is, determine the position of the click operation in the image. Then, the function module can determine the object clicked by the user based on the above-mentioned image coordinates and the center point of each object, that is, determine the focus object (protagonist).
  • the function module can determine the image coordinate C(x0,y0) corresponding to the user's click operation based on the above click_screen(X0,Y0). Then, optionally, the function module can calculate the distance between the above-mentioned image coordinates C (x0, y0) and the center points of the objects in the image: C1 (x1, y1), C2 (x2, y2). The function module can determine that the object corresponding to the center point with the smallest distance is the focus object.
  • the function module sends the focus object information (object identification, facial area, human body area, center point, focus label, and timestamp) to the focus tracking module.
  • focus object information object identification, facial area, human body area, center point, focus label, and timestamp
  • the function module can call the focus interface.
  • the focus tracking interface can be used to call the focus tracking module in CameraHAL to execute the focus tracking algorithm and identify whether the user-selected focus tracking object (protagonist) is included in the subsequent original image stream.
  • the function module can send the characteristic information of the focus tracking object selected by the user to the focus tracking module.
  • the above feature information is used by the focus tracking module to locate character 2 in the next image collected by the camera, that is, to locate the protagonist selected by the user.
  • the function module will also send a timestamp timestamp to the focus tracking module.
  • the above timestamp is the timestamp when the function module calls the focus tracking interface.
  • the above timestamp can be used to determine which frame of images collected by the camera starts to execute the focus tracking algorithm.
  • the large window control module can detect multiple user operations of clicking on the selection box.
  • the function module can determine multiple focus tracking objects based on the above multiple click operations, and then send the information to the focus tracking module.
  • the above-mentioned multiple focus objects can be detected using the above-mentioned multiple focus objects.
  • the focus tracking module obtains the object-detected original image from the object detection module.
  • the focus tracking module can first obtain the image to be focused, that is, a frame of original image collected by the camera and about to be displayed.
  • the focus tracking module can obtain the above-mentioned image to be focused from the object detection module.
  • only one frame of the original image that has undergone object recognition is stored in the cache of the object detection module.
  • the focus tracking module can directly obtain the image frame currently stored in the cache from the above cache.
  • the above currently stored image frame is the image to be focused.
  • the cache of the object detection module may store multiple frames of original images that have undergone object recognition. Among them, the last frame in the above cache is the image to be focused. At this time, if the timestamp of the moment of the focus object is not issued, the focus module may obtain the image frame that has been sent for display, so that the original image displayed in the preview window and the close-up image of the protagonist displayed in the small window Mismatch. In view of this, the functional module can also send the current timestamp when issuing the characteristic information of the focus object. The focus module can obtain the image to be focused from the cache according to the timestamp of the functional module, thereby avoiding the problem of image mismatch.
  • the focus tracking module can also obtain object feature information corresponding to the image for subsequent determination of the focus pursuit object in the image.
  • the focus tracking module executes the focus tracking algorithm and determines the focus tracking status (success, failure, out of focus) and the close-up image of the focus tracking object.
  • the focus module can execute the focus algorithm. Through the above focusing algorithm, the focusing module can confirm whether the image to be focused includes the focusing object indicated by the above characteristic information. If the image to be focused includes the focus object indicated by the above characteristic information, the focus tracking module may determine the focus tracking status to be successful; otherwise, the focus tracking module may determine the focus tracking status to be failure. When successful, the focus tracking module can further generate a close-up image centered on the above focus tracking object.
  • the focus-tracking module can obtain the i-th frame image and its corresponding object feature information. Then, the focus tracking module can calculate the similarity between each object in the i-th frame image and the focus tracking object sent in S513. The object in the i-th frame image that is most similar to the issued focus-chasing object and is higher than the threshold value can be determined as the focus-chasing object in the i-th frame image.
  • the focus tracking module when the focus tracking module obtains the i-th frame image from the object detection module, the focus tracking module can also obtain the characteristic information of each object in the above-mentioned i-th frame image, refer to Table 2:
  • the focus module may determine that the tracking status of the current frame is "successful”; otherwise, the focus module may determine that the tracking status of the current frame is "failed”.
  • the focus tracking module can continue to determine the focus tracking status of the i+1th frame and return the focus tracking status of the i+1th frame to the function module state.
  • the focus tracking module can determine that the tracking status of the i+1-th frame image is "successful"; if the i+1-th frame image matches the focus tracking object, If one frame of image does not match the focus object, the focus tracking module can determine that the tracking status of the i+1th frame image is "out of focus”.
  • the focus tracking module can also determine a close-up image based on the above focus tracking object, so that the camera APP can subsequently obtain and display the close-up image.
  • the specific process for the focus tracking module to calculate the similarity between each object in the i-th frame image and the focus tracking object issued above is as follows: First, the focus tracking module can calculate the similarity of each object according to the number of objects output by the object detection module and the number of each object. of the image area (such as the human body frame body1 in Figure 6), determine whether the characters in the i-th frame image overlap.
  • the terminal 100 can determine the focus object in the i-th frame image by using the intersection-over-union ratio IoU of the image area of each object and the image area of the issued focus object.
  • the image area of an object can be represented by the human body area in the object feature information, that is, the focus tracking module can focus on the protagonist through the IoU of the human body area in the i-th frame image.
  • the object in the i-th frame image that has the largest IoU with the delivered focus object can be determined as the focus object in the i-th frame image.
  • the terminal 100 can determine the focus object in the i-th frame image using the intersection-over-union ratio IoU of the image areas of each object and the similarity of the image content.
  • the object in the i-th frame image that has the largest sum of IoU and image content similarity with the issued focus object can be determined as the focus object in the i-th frame image.
  • the focus tracking module returns the focus tracking status (success, failure, out of focus) to the function module.
  • the focus tracking module may first return the focus tracking state to the function module.
  • the focus tracking status of a frame of image can be any one of success, failure, and out-of-focus status. Each time the focus tracking status of an image frame is determined, the focus tracking module can return the focus tracking status of the frame image to the function module.
  • the successful focus tracking status received by the function module can be called a focus tracking success indication; the out-of-focus focus tracking status can be called an out-of-focus indication.
  • the function module determines the focus tracking state. After receiving the focus tracking status returned by the focus tracking module in CameraHAL, the function module can confirm whether the above focus tracking status is successful, failed, or out of focus.
  • the function module sends an instruction to open the small window to the small window control module.
  • the small window control module draws a small window and controls the display to display the small window.
  • the small window control module may draw a small window, such as the small window 141 shown in FIG. 1D. Then, the small window control module can send the drawn image of the small window to the display. After receiving the image of the small window, the display displays the small window at the corresponding position of the display.
  • the small window control module obtains the close-up image of the focus tracking object from the focus tracking module and displays it in the small window.
  • the focus tracking module can further generate a close-up image centered on the focus tracking object.
  • the small window control module can obtain the close-up image through a session for reading the close-up image provided by the focus tracking module, thereby displaying the close-up image of the focus tracking object in the small window.
  • the above channel is a storage space in the focus tracking module used to store close-up images.
  • CameraAPP can receive the i-th frame image reported by CameraHAL and its corresponding close-up image of the protagonist.
  • the large window control module in CameraAPP can display the i-th frame image in the preview window
  • the small window control module in CameraAPP can display the above close-up image of the protagonist in the drawn small window.
  • the function module sends an instruction to display the selection box to the large window control module.
  • the function module can also send an instruction to display the object mark (selection box and/or check box) to the large window control module.
  • the focus object the protagonist
  • the mark of the non-protagonist is a selection box.
  • the above control information may carry the display coordinates of each object mark. It is understandable that S518 and S521 are not in sequence, they are both operations performed by the function module after S517.
  • the large window control module draws the selection box and controls the display to display the selection box.
  • the large window control module may determine to draw and select the box. Among them, according to the above display coordinates, the large window control module can determine the specific display position of each selection box on the screen. At this time, the large window control module can also confirm whether there is a focus object in the image based on the object feature information (isTracking) of the i-th frame image, and which specific focus object it is. The large window control module may then determine the type of the selection box: a selection box that represents an unchecked selection box (selection box 122), or a selection box that represents a selected selection box (selected box 142).
  • the large window control module can indicate that person 2 corresponds to the selected selection box (checked box), and person 1 corresponds to the unselected selection box. .
  • the large window control module can draw different types of selection boxes, so that the user can easily distinguish the protagonist from the non-protagonist.
  • the function module will not perform any action.
  • the terminal 100 will continue to display the images collected by the camera, identify the objects in the images and display the selection boxes corresponding to each object according to the method introduced above, and wait for the user's operation of selecting the protagonist.
  • the function module can also instruct the large window control module to display a focus tracking failure prompt in the preview window, instructing the user to reselect the protagonist.
  • FIG. 7 exemplarily shows a flow chart of defocusing during the preview process shown in FIGS. 1I-1J.
  • the small window 141 can immediately pause the small window, or it can wait for a period of time before pausing the small window.
  • the image cropped according to the position of the last close-up image in the original image can be displayed in the small window, or the original image that has been scaled can also be directly displayed.
  • the function module can start the timer.
  • the function module can set the timing duration of the timer. Combined with the introduction of the user interface shown in Figure 1I, the timing length of the above timer can be 5 seconds.
  • the focus tracking module can continue to crop the subsequent frames of original images according to the cropping method of the last frame before losing it, and obtain the image that is sent to the small window.
  • the small window control module can continue to obtain the image of the small window from the session provided by the focus tracking module, and then display it. It can be understood that at this time, the image content of the small window does not include the protagonist set above.
  • the function module sends the control information to pause the display to the small window control module.
  • the function module can send control information to close the small window to the small window control module.
  • the focus tracking module can store the close-up image of the focus tracking object in the session.
  • the small window control module can obtain the above close-up image from the session. At this time, the user can see the new close-up image in the small window.
  • the function module can close the timer that has not yet ended.
  • the function module can send control information to pause recording to the small window control module, and The small window will not be closed directly, and will be introduced in detail in subsequent embodiments.
  • the small window control module controls the closing of the small window.
  • the small window control module can close the small window and stop obtaining close-up images from the session. Referring to the user interface shown in FIG. 1J , the small window 141 is no longer displayed in the preview window 113 .
  • the function module sends control information to pause the display to the small window control module.
  • the function module can immediately send the closing control information to the small window control module without setting a timer and waiting. After the timer expires, the control information for closing the small window is sent.
  • the small window control module controls the closing of the small window. Same as S525, no details will be given here.
  • FIG. 8 exemplarily shows a flowchart in the process of starting recording shown in FIG. 2A-FIG. 2B.
  • the recording start module detects the user operation of starting recording.
  • the user can start video recording through the shooting control.
  • the terminal 100 may detect a user operation on the shooting control 112 , and in response to the above operation, the terminal 100 may start recording a video.
  • the recording start module in CameraAPP can detect the above user operation of starting recording.
  • the recording start module controls the large window encoder and the small window encoder to start encoding.
  • the recording start module can turn on the large window encoder and the small window encoder, and control the large window encoder and the small window encoder to start encoding and displaying images.
  • the large window encoder is used to encode the image stream in a large window (preview window 113).
  • the small window encoder is used to encode the image stream in a small window (small window 141).
  • the recording start module sends an instruction to start recording to the large window control module.
  • the large window control module starts the large window timer and displays the timing result.
  • the preview window view during recording may include a time stamp recording the duration of the preview window recording.
  • the large window control module can initialize the large window timer and start timing.
  • the large window control module can display the timing results of the above timer in the preview window.
  • the timing result displayed in the large window is the time stamp of the recording duration of the above-mentioned record preview window. Referring to the user interface shown in Figure 2B, the recording duration "00:01" is displayed in the upper left corner of the preview window 113.
  • the above recording duration "00:01" is the timing result of the current large window timer.
  • the recording start module sends instruction information to start recording to the small window control module. Similarly, after turning on the small window encoder, the recording start module can also send instruction information to start recording to the small window control module. It is understandable that there is no timing relationship between S603-S604 and S605-S606.
  • the small window control module starts the small window timer and displays the timing result.
  • the widget view during recording may include a time stamp recording the duration of the widget recording.
  • the small window control module can initialize the small window timer and start timing.
  • the small window control module can display the timing result of the above timer in the small window.
  • the timing result displayed in the small window is the time stamp of the recording duration of the small window. Referring to the user interface shown in Figure 2B, the recording duration "00:01" is displayed in the upper left corner of the small window 141.
  • the above recording duration "00:01" is the timing result of the current small window timer.
  • the function module monitors interactive events acting on the screen.
  • the function module can detect and monitor interactive control events acting on the screen in real time, such as click operations, sliding operations and other interactive events.
  • the function module can send instruction information to hide the selection box to the large window control module.
  • the above 5 seconds is an exemplary listening time.
  • the function module can also set other listening durations.
  • the above monitoring duration can be called the first duration.
  • the functional module can time the first duration through a first timer.
  • the large window control module hides the selection box.
  • the function module can send instruction information to hide the function control to the small window control module.
  • the above functional controls include the recording duration displayed in the small window, stop controls, etc., to reduce occlusion of the image displayed in the small window and improve the user browsing experience.
  • the small window control module hides function controls.
  • the small window control module After receiving the instruction information to hide the function controls, the small window control module can stop displaying the recording duration, stop control and other function controls. Similarly, there is no timing relationship between S608-S609 and S610-S611.
  • the function module monitors the interactive events acting on the screen.
  • the function module can monitor the interactive events acting on the screen, such as user operations of clicking anywhere on the screen.
  • the interaction time acting on the screen includes interactive events acting on the preview window and interactive events acting on the small window.
  • the function module After monitoring the interaction event acting on the screen, S613.
  • the function module sends an instruction to display the selection box to the large window control module.
  • S614. The large window control module may display the selection box. In this way, the user can select or switch the focus object through the above display frame.
  • the function module After monitoring the interaction event acting on the screen, S615.
  • the function module sends instruction information for displaying the function control to the small window control module.
  • S616 In response to the above instructions, S616.
  • the small window control module displays function controls.
  • the above functional controls include recording duration, pause control, stop control, etc. in the small window 141 . In this way, users can control the pause or stop of the recording window through the above functional modules.
  • the function module can respectively monitor interactive events acting on the large window area and small windows on the screen.
  • the function module can instruct the large window control module to display the selection box.
  • the function module may instruct the small window control module to display the function control.
  • FIG. 9 exemplarily shows a flowchart in the process of switching protagonists shown in FIGS. 2C to 2D.
  • the large window control module detects the user operation of switching the focus object.
  • the user can switch the focus object at any time.
  • the terminal 100 can detect the user operation of switching the focus object to the character 1 on the selection box 121 .
  • the large window control module in CameraAPP can detect the above-mentioned user operation of switching the focus object.
  • the large window control module returns the click coordinates to the function module.
  • the large window control module when detecting the user operation of clicking the selection box to switch the focus object, the large window control module also first determines the coordinates (click coordinates) of the click operation on the screen. Then, the large window control module can return the above-mentioned click coordinates to the function module.
  • the function module updates the focus object based on the click coordinates.
  • the function module may first determine the image coordinates corresponding to the click coordinates. In the process of switching the focus object, the function module can determine the object clicked by the user based on the above-mentioned image coordinates and the center point of each object, and then determine the new focus object (protagonist). Combined with the user interface shown in FIG. 2C , at this time, the terminal 100 can set character 1 as the new focus object (protagonist).
  • the function module sends the updated focus object information (face, body, center, isTracking, objectId, timestamp) to the focus module.
  • the functional module will also send the current timestamp timestamp to the focus tracking module, so that the focus tracking module can start positioning the new focus tracking object from which frame in the original image stream.
  • the focus tracking module obtains the original image that has been subject to object recognition from the object detection module, executes the focus tracking algorithm on the above image, and determines the focus tracking status (successful, out of focus).
  • the focus tracking module can determine the first frame image to start positioning the new focus tracking object based on the above timestamp timestamp.
  • the above timestamp may indicate that focusing on the new protagonist character 1 starts from the k-th frame image.
  • the focus tracking module can obtain the k-th image frame that has been subject to object recognition from the object detection module, and then determine the focus tracking status of the image frame: successful or out of focus.
  • the focus tracking module when obtaining the k-th frame image from the object detection module, can also obtain the characteristic information of each object in the k-th frame image. Referring to step S515, the focus tracking module can then locate the focus object of the k-th frame image based on the issued feature information of the focus-tracking object and the feature information of each object in the k-th frame image, which will not be described again here. Similarly, after locating the focus object of the k-th frame image, the focus module can also determine a close-up image based on the focus object, so that the CameraAPP can subsequently obtain and display the close-up image.
  • the focus tracking module returns the focus tracking status (successful, out of focus) to the function module.
  • the focus tracking module may first return the focus tracking state to the function module.
  • the focus-tracking status of the k-th frame image returned by the focus tracking module is "successful".
  • the focus-tracking status of the k-th frame image returned by the focus tracking module is "out of focus”.
  • the small window control module will periodically obtain the image of the small window from the session provided by the focus tracking module.
  • the focus tracking status is "successful"
  • the function module does not need to make any changes.
  • the close-up image in the focus tracking module session has been replaced with the close-up image of the new focus tracking object according to the focus tracking algorithm.
  • the image obtained by the small window control module from the session is correspondingly a close-up image of the new focus object.
  • the function module can start the timer.
  • the function module can set the timer duration when starting the timer.
  • the timing duration of the above timer is, for example, 5 seconds.
  • the above-mentioned timing duration may be called the second duration.
  • the above timer may be called a second timer.
  • the function module sends an instruction to pause recording to the pause recording module. While setting the timer, or after setting the timer, the function module can also send an instruction to pause recording to the pause recording module. S709.
  • the pause recording module controls the small window encoder to pause encoding. In response to the above instructions, the small window encoder in the pause recording module may pause encoding.
  • the pause recording module sends an instruction to pause recording to the small window control module. After pausing the small window encoder, the pause recording module may also send an instruction to pause the display to the small window control module. S711.
  • the small window control module pauses the small window timer. After receiving the instruction to pause the display, the small window control module may stop timing of the small window timer. At this time, the timestamp in the small window can stop at the last timing result before losing focus, such as "00:05".
  • the small window control module can pause obtaining and displaying new close-up images from the session. At this time, the close-up image displayed in the small window can stop at the currently displayed image.
  • the function module After pausing the recording, the function module receives the successful focus tracking status.
  • the function module can receive the focus tracking status of a certain frame returned from the focus tracking module as successful.
  • the small window control module can obtain the new close-up image of the focus object from the session, so the new close-up image can continue to be displayed in the small window.
  • the S713. function module can send an instruction to continue recording the small window to the start recording module.
  • S714. The small window encoder resumes encoding. The small window encoder may continue to encode the image stream in the small window (small window 141).
  • the start recording module can send instructions to resume recording to the small window control module.
  • S716 The small window control module resumes the small window timer and continues to display the close-up image.
  • the small window control module can restart the small window timer and redisplay the current timing of the small window timer in the small window.
  • the small window control module can continue to obtain and display the close-up image of the focus object from the session.
  • FIG. 10 exemplarily shows a flow chart in the process of pausing recording.
  • the recording pause module detects a user operation to pause recording.
  • pause control 2111 may be used to pause video recording.
  • the terminal 100 may confirm that the user operation of pausing recording is detected.
  • the S802. pause recording module can control the large window encoder and the small window encoder to stop encoding. At the same time, S803. Pause recording module can send an instruction to pause recording to the large window control module. In response to the above instructions, S804.
  • the large window control module may pause the large window timer. At this time, the timestamp in the large window can be stopped at the current timing moment. The user can confirm that the large window recording has been paused by the above timing time no longer increasing.
  • Pause recording module can also send instructions to pause recording to the small window control module.
  • the small window control module may pause the small window timer. At this time, the timestamp in the small window can be stopped at the current timing time. The user can determine that the small window recording has been paused by the above timing time no longer increasing. There is no timing relationship between S803-S804 and S805-S808. Moreover, from the user's perspective, the above-mentioned large window timer and small window timer stop timing at the same time.
  • the focus tracking module returns the focus tracking status to the function module.
  • the camera can still collect and generate images, and after generating the images, the generated image frames are reported to CameraHAL in turn.
  • the focus tracking module executes the focus tracking algorithm on the frame, determines whether each image frame includes a focus tracking object, and then determines the focus tracking status, and transmits the above focus tracking status to Return to the function module.
  • the function module can send an indication that the focus tracking object is lost to the small window control module.
  • S809 The small window control module controls the small window to display the mask layer.
  • S810 When recording is paused in both large and small windows, when the returned focus status is "out of focus", the function module can also send an instruction to the large window control module indicating that the focus object is lost. In response to the above instructions, S811. The large window control module may control the large window to display prompt information indicating that the focus tracking object is lost. Similarly, there is no timing relationship between S808 and S810.
  • the focus tracking module can determine that the focus tracking status of the frame image is successful, and transmit the above focus tracking status back to the function module .
  • the function module sends an instruction to retrieve the focus object to the small window control module.
  • the small window control module eliminates the small window mask.
  • the function module will also send an instruction to retrieve the focus tracking object to the large window control module.
  • S815. The large window control module displays prompt information instructing the retrieval of the focus object in the large window.
  • the functional module can control to pause the recording of the small window image.
  • the function module will also send an instruction that the focus object is lost to the small window control module, so that the small window displays the mask layer accordingly.
  • the function module can restore the recording of small window images.
  • the function module will also send instructions for retrieving the focus object to the small window control module to eliminate the masking layer displayed on the small window.
  • FIG. 11 exemplarily shows a flowchart in the process of ending recording shown in FIGS. 2H-2I.
  • the user can choose to close the small window first and end recording the close-up video.
  • the user can also close the large window and end recording the original video and close-up video at the same time.
  • the end recording module detects the user operation of ending the small window recording.
  • the terminal 100 can detect user operations acting on the above-mentioned control.
  • the above user operation is the user operation that ends the small window recording.
  • the ending recording module can control to close the small window encoder, stop the small window encoder from encoding and display the close-up image of the small window.
  • the small window encoder can encapsulate the encoded close-up image into a video, that is, a close-up video.
  • the ending recording module may also send an instruction to close the small window to the small window control module.
  • the small window control module can close the small window.
  • the end recording module detects a user operation to end recording.
  • the terminal 100 may also detect user operations acting on the control 2112.
  • the above user operation ends the recorded user operation.
  • the above operation of ending recording corresponds to ending large window recording and small window recording at the same time.
  • the ending recording module can control to close the large window encoder and the small window encoder, stop encoding and display the images of the large window and the small window.
  • the small window encoder can encapsulate the encoded close-up image into a video, that is, a close-up video.
  • the large window encoder can encapsulate the encoded original image into a video, that is, the original video.
  • the end recording module After detecting the user operation to end the recording, S907.
  • the end recording module sends an instruction to close the large window to the large window control module.
  • the large window control module displays the preview interface before starting to record the video.
  • the large window control module can display the preview interface.
  • the end recording module sends an instruction to close the small window to the small window control module. After receiving the instruction to close the small window, S910.
  • the small window control module closes the small window.
  • the terminal 100 can display the user interface shown in FIG. 1A: preview interface without small window. Similarly, there is no timing relationship between S907 and S909.
  • the object recognized by the object detection module from the image collected by the camera may be called the first object, such as Person 1 and Person 2 shown in Figure 1B.
  • One frame of image collected by the camera may include one or more first objects.
  • the object selected as the focus object (protagonist) from the objects recognized by the object detection module may be called a second object, for example, the character 2 determined by the user as the protagonist shown in FIG. 1D.
  • the window used to display the close-up image of the focus object may be called the first window, such as the small window 141 showing the character 2 (the focus object) shown in FIG. 1D.
  • the original video obtained by recording the original image in the preview window 113 is the first video.
  • the close-up video obtained by recording the close-up image in the small window 141 is the second video.
  • the large window encoder enabled by the start recording module may be called the first encoder, and the small window encoder may be called the second encoder.
  • Figure 12 is a schematic diagram of the hardware structure of the terminal 100 provided by the embodiment of the present application.
  • the terminal 100 may include a processor 110, an external memory interface 120, an internal memory 129, an audio module 130, Speaker 130A, receiver 130B, microphone 130C, headphone interface 130D, display screen 140, camera 150, touch sensor 160.
  • the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the terminal 100.
  • the terminal 100 may include more or fewer components than shown in the figures, or some components may be combined, or some components may be separated, or may be arranged differently.
  • the components illustrated may be implemented in hardware, software, or a combination of software and hardware.
  • the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic explanation and does not constitute a structural limitation of the terminal 100.
  • the processor 110 may include one or more processing units.
  • the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (GPU), and an image signal processor. (image signal processor, ISP), controller, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural network processor (neural-network processing unit, NPU), etc.
  • the processor 110 may also be provided with a memory for storing instructions and data.
  • the terminal 100 implements the display function through the GPU, the display screen 140, and the application processor.
  • the GPU is an image processing microprocessor and is connected to the display screen 140 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering.
  • Processor 110 may include one or more GPUs that execute program instructions to generate or alter display information.
  • the display screen 140 is used to display images, videos, etc.
  • the electronic device may include 1 or N display screens 140, where N is a positive integer greater than 1.
  • the terminal 100 displays the original image collected by the camera, the close-up image of the protagonist determined through focus tracking, and the user interface shown in Figures 1A to 1J, 2A to 2I, and 3A to 3C. capabilities, depending on the display functions provided by the aforementioned GPU, display 140, and application processor.
  • the terminal 100 can implement the shooting function through the ISP, camera 150, video codec, GPU, display screen 140 and application processor.
  • the terminal 100 is used to capture still images or video.
  • the ISP is used to process the data fed back by the camera 150 .
  • Light is transmitted to the camera sensor through the lens, and the optical signal is converted into an electrical signal.
  • the camera sensor passes the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye.
  • the terminal 100 may include 1 or N cameras 150, where N is a positive integer greater than 1.
  • Video codecs are used to compress or decompress digital video. Terminal 100 may support one or more video codecs. In this way, the terminal 100 can play or record videos in multiple encoding formats, such as moving picture experts group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.
  • MPEG moving picture experts group
  • the terminal 100 implements the shooting method provided by the embodiment of the present application, first relying on the ISP and the images collected by the camera 150, and secondly relying on the video codec and the image calculation and processing capabilities provided by the GPU.
  • Internal memory 129 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).
  • the random access memory can be directly read and written by the processor 110, can be used to store executable programs (such as machine instructions) of the operating system or other running programs, and can also be used to store user and application data, etc.
  • the non-volatile memory can also store executable programs and user and application program data, etc., and can be loaded into the random access memory in advance for direct reading and writing by the processor 110.
  • the code for implementing the photographing method described in the embodiment of the present application may be stored in a non-volatile memory.
  • the terminal 100 may load the executable code stored in the non-volatile memory into the random access memory.
  • the external memory interface 120 can be used to connect an external non-volatile memory to expand the storage capability of the terminal 100 .
  • the terminal 100 can implement audio functions through the audio module 130, the speaker 130A, the receiver 130B, the microphone 130C, the headphone interface 130D, and the application processor.
  • the audio module 130 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals.
  • Speaker 130A also called “speaker”
  • Receiver 130B also called “earpiece”
  • Microphone 130C also called “microphone” or “microphone”
  • the headphone interface 130D is used to connect wired headphones.
  • the terminal 100 can simultaneously enable the microphone 130C to collect sound signals, and convert the sound signals into electrical signals to store them. In this way, users can get videos with sound.
  • Touch sensor 160 is also called a "touch device”.
  • the touch sensor 160 can be disposed on the display screen 140, and the touch sensor 160 and the display screen 140 form a touch screen, which is also called a "touch screen”.
  • the touch sensor 160 is used to detect touch operations on or near the touch sensor 160 .
  • the touch sensor can pass the detected touch operation to the application processor to determine the touch event type.
  • Visual output related to the touch operation may be provided through the display screen 140 .
  • the touch sensor 160 may also be disposed on the surface of the terminal 100 in a position different from that of the display screen 140 .
  • the terminal 100 can use the touch sensor 160 to detect the user's click, slide and other operations on the display screen 140 to implement the operations shown in FIGS. 1A-1J, 2A-2I, and 3A-3C. shooting method.
  • UI user interface
  • the term "user interface (UI)” in the description, claims and drawings of this application is a media interface for interaction and information exchange between an application or operating system and a user, which implements the internal form of information. Conversion to and from a user-acceptable form.
  • the user interface of an application is source code written in specific computer languages such as Java and extensible markup language (XML).
  • XML Java and extensible markup language
  • the interface source code is parsed and rendered on the terminal device, and finally presented as content that the user can recognize.
  • Control also called widget, is the basic element of user interface. Typical controls include toolbar, menu bar, text box, button, and scroll bar. (scrollbar), images and text.
  • the properties and contents of controls in the interface are defined through tags or nodes.
  • XML specifies the controls contained in the interface through nodes such as ⁇ Textview>, ⁇ ImgView>, and ⁇ VideoView>.
  • a node corresponds to a control or property in the interface. After parsing and rendering, the node is rendered into user-visible content.
  • applications such as hybrid applications, often include web pages in their interfaces.
  • a web page also known as a page, can be understood as a special control embedded in an application interface.
  • a web page is source code written in a specific computer language, such as hypertext markup language (GTML), cascading styles Tables (cascading style sheets, CSS), java scripts (JavaScript, JS), etc.
  • web page source code can be loaded and displayed as user-recognizable content by a browser or a web page display component with functions similar to the browser.
  • the specific content contained in the web page is also defined through tags or nodes in the web page source code.
  • GTML defines the elements and attributes of the web page through ⁇ p>, ⁇ img>, ⁇ video>, and ⁇ canvas>.
  • GUI graphical user interface
  • Controls can include icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc.
  • Visual interface elements can include icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc.
  • the computer program product includes one or more computer instructions.
  • the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.
  • the computer instructions may be stored in or transmitted from one computer-readable storage medium to another, e.g., the computer instructions may be transferred from a website, computer, server, or data center Transmission to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) means.
  • the computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains one or more available media integrated.
  • the available media may be magnetic media (eg, floppy disk, hard disk, tape), optical media (eg, DVD), or semiconductor media (eg, solid state drive), etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Human Computer Interaction (AREA)
  • Business, Economics & Management (AREA)
  • General Business, Economics & Management (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Studio Devices (AREA)
  • Processing Or Creating Images (AREA)

Abstract

本申请提供了一种拍摄方法。实施该方法，手机、平板电脑等终端设备可以在开始拍摄或拍摄过程中，根据用户操作确定拍摄主角。在拍摄的过程中，终端设备可以在预览窗中显示摄像头采集的图像流，同时，终端设备还可生成一个专门显示主角的小窗。在结束录制之后，用户可以得到两个视频：基于预览窗中的图像流生成的原始视频，和基于小窗中主角图像流生成的特写视频。这样，用户可以选择使用原始视频和/或特写视频，以满足自身不同时刻不同场景下的个性化需求。

Description

一种拍摄方法和电子设备
本申请要求于2022年05月30日提交中国专利局、申请号为202210603605.4、申请名称为“一种拍摄方法和电子设备”的中国专利申请的优先权，和2022年07月29日提交中国专利局、申请号为202210908497.1、申请名称为“一种拍摄方法和电子设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及终端领域，尤其涉及一种拍摄方法和电子设备。
背景技术
为提高用户体验，手机、平板电脑等终端设备通常配置多个摄像头。为拍照与录像功能提供多种拍摄模式。例如，前摄模式、后摄模式、前后双摄模式等。实现中，用户可使用终端设备录制一个或多个人物的视频。在用户想得到其中一个目标人物的视频时，可对视频进行人工剪辑处理。但人工剪辑的处理方法操作繁琐，且需要拍摄用户具备一定的视频剪辑技术，用户体验较差。
发明内容
第一方面，本申请提供了一种拍摄方法，应用于具有摄像头和屏幕的电子设备，该方法包括：在屏幕中显示预览窗，预览窗中显示摄像头采集的图像，预览窗中显示的图像包括一个或多个第一对象；在一个或多个第一对象中选择第二对象；在屏幕中显示预览窗的同时，还显示第一窗口，第一窗口中显示的图像包括第二对象；录制预览窗中的图像并保存为第一视频，录制第一窗口中的图像并保存为第二视频。
实施第一方面提供的方法，电子设备可以识别摄像头采集的图像中的对象，并从中选择追焦对象，即主角。然后，电子设备可在屏幕中再显示一个窗口，即第一窗口，用于展示主角图像。电子设备可分别录制并保存预览窗中的图像和第一窗口中的图像。这样，用户既可以拍摄预览窗中展示的原始图像，也可以自动地获得聚焦主角的特写图像。
实施第一方面提供的方法，在一些实施例中，在一个或多个第一对象中选择第二对象，包括：在一个或多个第一对象上显示选择框；响应于作用于第一选择框上的用户操作，将第一选择框对应的第一对象确定为第二对象。
实施上述实施例提供的方法，电子设备可以在识别到图像中的对象之后，显示与各个对象对应的选择框。这样，用户可通过上述选择框确定电子设备识别到的各个对象，并通过上述选择框选择上述各个对象中的任意一个为追焦对象。
实施第一方面提供的方法，在一些实施例中，在一个或多个第一对象上显示选择框之后，该方法还包括：在第一时长内没有检测到作用于屏幕的用户操作，选择框消失。
实施上述实施例提供的方法，电子设备可以在显示选择框之后检测屏幕上的交互事件，若在预设时间内，例如第一时长内，未检测到任何作用于屏幕上的交互事件，则电子设备可以隐藏选择框。这样既可以减少选择框对屏幕中显示的图像的遮挡，也可以减少绘制图像所需的计算成本，节省功耗。
实施第一方面提供的方法，在一些实施例中，第二对象是电子设备确定的。
实施上述实施例提供的方法，电子设备也可自行从识别到的对象中确定追焦对象。例如，电子设备中可预设有对象评价规则。电子设备可依据上述规则对识别到的图像中的各个对象进行评价，然后确定评价最高的对象为追焦对象。
实施第一方面提供的方法，在一些实施例中，屏幕中还显示有录制控件，在屏幕中显示第一窗口之后，该方法还包括：接收到用户对应于录制控件的录制操作，同时开始录制第一视频和第二视频；第一视频和第二视频的开始录制时刻相同。
实施上述实施例提供的方法，用户可以在确定追焦对象之后，点击录制控件，控制电子设备开始录制。此时，屏幕中包括预览窗和第一窗口。电子设备可以同时开始录制预览窗中的图像和第一窗口中的图像。
实施第一方面提供的方法，在一些实施例中，屏幕中还显示有录制控件，在一个或多个第一对象中选择第二对象之前，该方法还包括：接收到用户对应于录制控件的录制操作，开始录制第一视频；在显示第一窗口的同时，开始录制第二视频；第二视频的开始录制时刻晚于第一视频的开始录制时刻。
实施上述实施例提供的方法，用户可以在选定追焦对象之前先点击录制控件开始录制。此时，屏幕中包括预览窗而未显示第一窗口。电子设备先录制预览窗中的图像。在用户确定追焦对象，并显示第一窗口之后，电子设备可以自动地开始录制第一窗口中的图像。
实施第一方面提供的方法，在一些实施例中，第一视频和第二视频的结束录制时刻是相同的；或者，第二视频的结束录制时刻早于第一视频的结束录制时刻。
实施上述实施例提供的方法，用户可以先结束录制第一窗口的图像，也可以同时结束录制第一窗口和预览窗的图像。
实施第一方面提供的方法，在一些实施例中，在录制过程中，该方法还包括：从一个或多个第一对象中变更第二对象；第一窗口中显示的图像包括变更后的第二对象。
实施上述实施例提供的方法，用户可以在录制过程中随时切换追焦对象。在切换追焦对象之后，电子设备可以追焦新选定的追焦对象，并在第一窗口中显示上述新选定的追焦对象的特写图像。
实施第一方面提供的方法，在一些实施例中，在录制过程中，该方法还包括：当监测到摄像头采集的图像不包括第二对象时，且持续时间长于第二时长后，暂停录制第一窗口中的图像。
实施上述实施例提供的方法，当识别到摄像头采集的图像不包括之前选定的追焦对象时，电子设备可以启动计时器。当上述计时器计时结束，且仍未从摄像头采集的图像中识别到主角时，第一窗口中显示的图像可以停留在计时器结束的一帧。同时，电子设备可以暂停录制第一窗口中的图像。
实施第一方面提供的方法，在一些实施例中，在录制过程中，该方法还包括：当监测到摄像头采集的图像不包括第二对象时，暂停录制第一窗口中的图像。
实施上述实施例提供的方法，当识别到摄像头采集的图像不包括之前选定的追焦对象时，第一窗口中显示的图像可以停留在包括追焦对象的最后一帧。同时，电子设备可以暂停录制第一窗口中的图像。即，在失焦之后，电子设备可以立即暂停录制第一窗口中的图像。因为，此时第一窗口中不再更新追焦对象的特写图像。这样，当主角从取景范围内消失时，电子设备可以自动的暂停录制特写图像，从而得到更纯净的仅包括主角的特写视频。
实施第一方面提供的方法，在一些实施例中，该方法还包括：当监测到摄像头采集的图像包括第二对象时，恢复录制第一窗口中的图像。
实施上述实施例提供的方法，在因为失焦暂停录制第一窗口中的图像之后，若重新识别到摄像头采集的图像包括之前选定的追焦对象，则电子设备还可以在第一窗口中继续显示追焦对象的特写图像，并恢复录制第一窗口中的图像。这样，当主角重新出现在取景范围内时，电子设备可以自动的继续追焦主角，且无需用户操作。
实施第一方面提供的方法，在一些实施例中，第一窗口与预览窗分屏显示，或者，第一窗口显示在预览窗之上，但不完全覆盖预览窗。
实施上述实施例提供的方法，用户在录制视频的过程中，可以同时浏览预览窗中展示的原始图像和第一窗口中展示的包括主角的特写图像。进而，用户可以随时调整摄像头位置，以便于拍摄更符合个人需求的原始视频和特写视频。
实施第一方面提供的方法，在一些实施例中，第一窗口中显示的包括第二对象的图像是基于预览窗中显示的图像裁剪得到的。
实施第一方面提供的方法，在一些实施例中，电子设备包括功能模块、对象检测模块和大窗控制模块，在一个或多个第一对象上显示选择框，包括：对象检测模块从摄像头采集的图像中识别到一个或多个第一对象；对象检测模块将一个或多个第一对象的特征信息发送给功能模块，特征信息包括指示第一对象位置的中心点；一个第一对象对应一组特征信息。功能模块依据特征信息中的中心点确定显示坐标；大窗控制模块依据显示坐标在屏幕上显示与一个或多个第一对象对应的选择框。
实施第一方面提供的方法，在一些实施例中，在第一时长内没有检测到作用于屏幕的用户操作，选择框消失，包括：在显示选择框之后，功能模块设置第一计时器，第一计时器的计时时长为第一时长；在第一计时器计时结束后，且功能模块未检测到作用于屏幕的用户操作，大窗控制模块消除选择框。
实施第一方面提供的方法，在一些实施例中，电子设备还包括追焦模块、小窗控制模块，显示第一窗口，包括：功能模块向追焦模块发送第二对象的特征信息；追焦模块依据第二对象的特征信息确定将要送显的图像中是否包括第二对象；当包括第二对象时，追焦模块向功能模块发送追焦成功指示，功能模块依据指示控制小窗控制模块在屏幕中显示第一窗口；
实施第一方面提供的方法，在一些实施例中，该方法还包括：当包括第二对象时，追焦模块从将要送显的图像中裁剪出第一窗口中显示的图像。
实施第一方面提供的方法，在一些实施例中，电子设备还包括开始录制模块，接收到用户对应于录制控件的录制操作，同时开始录制第一视频和第二视频，包括：开始录制模块接收到用户对应于录制控件的录制操作；响应于录制操作，开始录制模块开启第一编码器和第二编码器，第一编码器开始编码预览窗中的图像，第二编码器开始编码第一窗口中的图像。
实施第一方面提供的方法，在一些实施例中，电子设备还包括暂停录制模块，当监测到摄像头采集的图像不包括第二对象时，且持续时间长于第二时长后，暂停录制第一窗口中的图像，包括：当识别到将要送显的图像中不包括第二对象时，追焦模块向功能模块发送失焦指示，功能模块依据失焦指示设置第二计时器，第二计时器的计时时长为第二时长；在第二计时器计时结束，且功能模块仍未接收到追焦成功指示时，暂停录制模块控制第二编码器暂停编码第一窗口中显示的图像。
实施第一方面提供的方法，在一些实施例中，当监测到摄像头采集的图像包括第二对象时，恢复录制第一窗口中的图像，包括：当识别到将要送显的图像中包括第二对象时，追焦模块向功能模块发送追焦成功指示，功能模块依据追焦成功指示，向开始录制模块发送恢复录制指示；开始录制模块依据恢复录制指示重新开启第二编码器，继续编码第一窗口中显示的图像。
实施第一方面提供的方法，在一些实施例中，该方法还包括：在图库中显示第一视频的缩略图、第二视频的缩略图，第二视频的缩略图中包含第二对象，第二视频的缩略图也包含第一对象，第一对象在第二视频的缩略图中的图像内容与第一对象在第一视频的缩略图中的图像内容一致。
实施上述实施例提供的方法，电子设备可以在图库应用中分别显示录制预览窗中图像得到的第一视频，和录制第一窗口得到的第二视频。这样，用户可以任意的选择浏览第一视频或第二视频。
实施第一方面提供的方法，在一些实施例中，第一视频的缩略图与第二视频的缩略图相邻陈列。
实施上述实施例提供的方法，同时录制的第一视频和第二视频可以相邻的展示。用户可以清晰直观的获取同时录制的第一视频和第二视频。
实施第一方面提供的方法，在一些实施例中，摄像头包括第一摄像头和第二摄像头，预览窗中显示的第一摄像头采集的图像，第二图像为第二摄像头对焦第二对象采集得到的。
实施上述实施例提供的方法，电子设备也可以通过不同的摄像头追焦主角，从而得到与预览窗中显示的主角图像内容不同的特写图像。
第二方面，本申请提供了一种电子设备，该电子设备包括一个或多个处理器和一个或多个存储器；其中，一个或多个存储器与一个或多个处理器耦合，一个或多个存储器用于存储计算机程序代码，计算机程序代码包括计算机指令，当一个或多个处理器执行计算机指令时，使得电子设备执行如第一方面以及第一方面中任一可能的实现方式描述的方法。
第三方面，本申请实施例提供了一种芯片系统，该芯片系统应用于电子设备，该芯片系统包括一个或多个处理器，该处理器用于调用计算机指令以使得该电子设备执行如第一方面以及第一方面中任一可能的实现方式描述的方法。
第四方面，本申请提供一种计算机可读存储介质，包括指令，当上述指令在电子设备上运行时，使得上述电子设备执行如第一方面以及第一方面中任一可能的实现方式描述的方法。
第五方面，本申请提供一种包含指令的计算机程序产品，当上述计算机程序产品在电子设备上运行时，使得上述电子设备执行如第一方面以及第一方面中任一可能的实现方式描述的方法。
可以理解地，上述第二方面提供的电子设备、第三方面提供的芯片系统、第四方面提供的计算机存储介质、第五方面提供的计算机程序产品均用于执行本申请所提供的方法。因此，其所能达到的有益效果可参考对应方法中的有益效果，此处不再赘述。
附图说明
图1A-图1J是本申请实施例提供的一组主角模式拍摄方法的用户界面示意图；
图2A-图2I是本申请实施例提供的一组主角模式拍摄方法的用户界面示意图；
图3A-图3C是本申请实施例提供的一组浏览已拍摄图像的用户界面示意图；
图4是本申请实施例提供的一种终端100的系统结构示意图；
图5是本申请实施例提供的一种终端100的软件模块的交互流程图；
图6是本申请实施例提供的一种终端100的确定一帧图像的对象特征信息的示意图；
图7是本申请实施例提供的另一种终端100的软件模块的交互流程图；
图8是本申请实施例提供的另一种终端100的软件模块的交互流程图；
图9是本申请实施例提供的另一种终端100的软件模块的交互流程图；
图10是本申请实施例提供的另一种终端100的软件模块的交互流程图；
图11是本申请实施例提供的另一种终端100的软件模块的交互流程图；
图12是本申请实施例提供的终端100的硬件结构示意图。
具体实施方式
本申请以下实施例中所使用的术语只是为了描述特定实施例的目的，而并非旨在作为对本申请的限制。
为提高用户体验，手机、平板电脑等终端设备通常配置多个摄像头。为拍照与录像功能提供多种拍摄模式。例如，前摄模式、后摄模式、前后双摄模式等。可能的实现中，用户可使用终端设备录制一个或多个人物的视频。在用户想得到其中一个目标人物的视频时，可对视频进行人工剪辑处理。但人工剪辑的处理方法操作繁琐，且需要拍摄用户具备一定的视频剪辑技术，用户体验较差。
有鉴于此，本申请实施例提供了一种拍摄方法，其中，终端设备可提供主角模式的拍摄功能。在主角模式下，终端设备可额外生成一路针对主角的视频，无需人工剪辑处理，提升用户的拍摄体验。
具体的，在主角模式下，终端100在确定拍摄主角之后，可同时生成两个视频，记为原始视频和特写视频。其中，原始视频是由摄像头采集的原始图像组成的。特写视频是在原始图像的基础上，识别图像中的主角，然后裁剪主角图像得到的。在录制视频的过程中，终端100可同时显示上述原始视频和特写视频，以供用户预览。
不限于生成一个特写视频，用户也可以选择多个主角。终端100可基于摄像头采集的原始图像定位上述多个主角，进而生成多个对应各个主角的特写视频。
这样，用户既可以拍得以主角为中心的特写视频，又可以同时得到由原始的摄像头采集的原始图像组成的原始视频，进一步满足用户个性化需求，提升用户使用体验。
上述方法可应用于手机、平板电脑等终端设备上。后续统一使用终端100指代上述终端设备。不限于手机、平板电脑，终端100还可以是桌面型计算机、膝上型计算机、手持计算机、笔记本电脑、超级移动个人计算机(ultra-mobile personal computer，UMPC)、上网本，以及蜂窝电话、个人数字助理(personal digital assistant，PDA)、增强现实(augmented reality，AR)设备、虚拟现实(virtual reality，VR)设备、人工智能(artificial intelligence,AI)设备、可穿戴式设备、车载设备、智能家居设备和/或智慧城市设备，本申请实施例对该终端的具体类型不作特殊限制。
下面具体介绍终端100实施本申请实施例提供的拍摄方法的用户示意图。
首先，图1A示例性示出了终端100启用摄像头执行拍摄动作的用户界面。
如图1A所示，该用户界面可包括菜单栏111、拍摄控件112、预览窗113、回看控件114。
菜单栏111中可显示有多个拍摄模式选项，例如夜景、录像、拍照、人像等拍摄模式。夜景模式可用于光线较暗的场景下拍摄照片，例如夜晚拍摄照片。录像模式可用于录制视频。拍照模式可用于日光场景下拍摄照片。人像模式可用于拍摄人物特写照片。
在启用摄像头执行拍摄动作时，如图1A所示，终端100可首先启用录像模式，准备开始录制视频。当然，终端100可以首先启用拍照、人像等其他拍摄模式，这时，终端100可根据用户操作进入录像模式。
拍摄控件112可用于接收用户的拍摄操作。在拍照场景下(包括拍照模式、人像模式、夜景模式)，上述拍摄操作即作用于拍摄控件112的控制拍照的操作。在录制视频的场景下(录像模式)，上述拍摄操作包括作用于拍摄控件112的开始录制的操作。
预览窗113可用于实时地显示摄像头采集的图像流。任意时刻，预览窗113中显示的一帧图像为一帧原始图像。
回看控件114可用于查看前一次拍摄的照片或视频。一般的，回看控件114可显示前一次拍摄的照片的缩略图或前一次拍摄的视频的首帧图像的缩略图。
在录像模式下，图1A所示的用户界面还可包括功能栏115。功能栏115中可包括多个功能控件，例如闪光灯1151、滤镜1152、设置1153、主角模式1154等功能控件。闪光灯1151可用于开启或关闭闪光灯，从而改变摄像头采集到的图像的画面亮度。滤镜1152可用于选择滤镜风格，进而调整预览窗113中的图像色彩。设置1153可用于提供更多的用于调整摄像头拍摄参数或图像优化参数的控件，例如白平衡控件、ISO控件、美颜控件、美体控件等等，从而为用户提供更丰富的拍摄服务。
主角模式1154可用于提供拍摄以主角为中心的特写视频的功能。特别的，在本申请实施例提供的主角模式拍摄方法中，终端100可根据用户操作选择和变更主角，拍摄并保存至少两路视频：分别为以主角为中心的特写视频，和基于摄像头采集的原始图像生成的原始视频。其中，以主角为中心的特写视频可以有多个。终端100可以根据用户操作确定一个或多个主角。一个主角可对应一个或多个特写视频。
在显示图1A所示的用户界面的过程中，终端100可检测到作用于主角模式1154的用户操作，以开启主角模式拍摄功能。上述作用于主角模式1154的用户操作例如点击主角模式1154的操作。响应于上述操作，终端100可执行主角模式对应的算法，进入主角模式拍摄场景中，参考图1B。
图1B示例性示出了终端100进入主角模式拍摄场景后进行拍摄的用户界面。
在进入主角模式之后，终端100可对摄像头采集的图像进行图像识别，识别该图像中包括的对象(即对象识别)。上述对象包括但不限于人、动物、植物。本申请实施例后续主要以人为例进行说明。
参考图1B中预览窗113中展示的图像，某一时刻，终端100的摄像头采集的图像中包括人物1、人物2。在接收到上述图像之后，终端100可首先利用预设的对象识别算法识别图像中包括的对象。这里，上述对象识别算法可以为人体检测算法。可以理解的，当终端100还支持识别动物、植物类型的对象。相应的，上述对象识别算法还包括针对一种或多种动物的识别算法，以及针对一种或多种植物的识别算法，本申请实施例对此不作限制。这时，经过对象识别算法的处理，终端100可识别到上述图像中包括人物1、人物2这2个对象。
在接收到上述图像之后，终端100可在预览窗113中显示上述图像。基于已识别到的图像中包括的对象，在显示上述图像的同时，终端100还可显示与上述各个对象上对应的选择框。例如，终端100可在人物1上显示对应人物1的选择框121，在人物2上显示对应人物2的选择框122。这时，一方面，用户可以通过上述选择框确认终端100已检测的到可供用户选择的对象；另一方面，用户可以通过点击上述选择框中的任意一个，设定与之对应的对象为主角。
可选的，终端100还可在预览窗113中显示提示语125，例如“请选择主角人物，开启自动追焦录像”，用于提示用户选定主角。
在图1B所示的用户界面中，预览窗113中可包括美颜控件127。美颜控件127可用于调整图像中的人物的人脸图像。当检测作用于美颜控件127的用户操作之后，终端100可对图像中的人物进行美颜处理，并在预览窗中显示美颜处理后的图像。图1B所示的用户界面还可显示有其他拍摄控件，例如用于调整摄像头的焦距控件等，这里不再一一例举。
在显示图1B所示的用户界面时，终端100可检测到作用于任一选择框的用户操作。响应于上述操作，终端100可确定上述选择框对应的对象为主角。例如，参考图1C所示的用户界面，终端100可检测到作用于选择框122的用户操作。响应于上述操作，终端100可确定选择框122对应的人物2为主角。
随后，终端100可在预览窗113中以画中画的形式(嵌入)显示一个小窗，并在该小窗中显示人物2的特写图像。上述特写图像是指在摄像头采集的原始图像(预览窗中显示的图像)的基础上，以选定的主角为中心进行裁剪得到的图像。可选的，终端100可也将原来的预览窗113对应的显示区域划分为并列的两部分：一部分用于显示摄像头采集的图像；一部分用于显示基于上述摄像头采集的图像得到的特写图像。
图1D示例性示出了终端100显示小窗并在小窗中显示人物2的特写图像的用户界面。
如图1D所示，预览窗113中可包括小窗141。此时，小窗141中可显示人物2的特写图像。随着预览窗113中显示的原始图像中人物2的变化，小窗141中显示的人物2的特写图像也会相应的变化。这样，小窗141中显示的连续的以人物2为中心的特写图像即构成了人物2的特写视频。
可选的，小窗141中显示的特写图像与预览窗113中显示的原始图像还可来自不同的摄像头。例如，小窗141中显示的特写图像可以来自于长焦摄像头采集的图像，预览窗113中显示的原始图像可以来自于广角摄像头采集的图像。普通摄像头和广角摄像头可以同时采集图像。长焦摄像头和广角摄像头采集的图像可以通过时间戳进行对应。这时，小窗141中显示的特写图像是在普通摄像头采集的图像上裁剪得到的，而上述普通摄像头采集的图像对应同一时刻预览窗113中显示的广角摄像头采集的图像。这样，用户可以在预览窗113浏览更大范围的景观，同时，在小窗141中显示细节更多的主角图像。上述的广角和长焦仅是举例，小窗141中的显示图像还可以来自于广角摄像头、超广角摄像头、黑白摄像头、多光谱摄像头等任意摄像头。同样的预览窗113中显示的图像也可以自于超广角摄像头、长焦摄像头、黑白摄像头、多光谱摄像头等任意摄像头。
在确定人物2为拍摄主角之后，人物2对应的选择框122可变成图1D中选中框142。用户可通过选中框142区分主角和非主角的对象。不限于用户界面中所示的选中框142，终端100还可显示其他样式的图标，或使用不同的颜色，本申请实施例对此不作限制。
在一些示例中，用户还可以选定多个主角。例如，在图1D所示的用户界面上，终端100还可以检测到作用于选择框121的用户操作，响应于上述操作，终端100可确定选择框121对应的人物1为另一主角。在一些示例中，终端100可显示对应显示人物1的特写图像的小窗。在另一些示例中，为了避免对预览窗113造成较大遮挡，当有多个主角时，终端100也可只显示一个展示主角特写图像的小窗。对应选定的多个主角，终端100可生成多个特写视频。
可选的，用于展示特写图像的小窗141还可包括关闭控件143和转置控件144。关闭控件143可用于关闭小窗141。转置控件可用于调整小窗141的尺寸。
参考图1E，终端100可以检测到作用于关闭控件143的用户操作。响应于上述操作，终端100可关闭小窗141，参考图1F。如图1F所示，在关闭小窗141时，终端100可取消之前选定的主角(人物2)。相应地，对应人物2的选中框142可变更为选择框122。这时，用户可以重新选择预览窗113中已识别到的任一对象为主角。终端100可基于重新确定的主角，再次在预览窗113中显示小窗141。此时，小窗141中显示以新主角为中心对原始图像进行处理得到的特写图像。
参考图1G，终端100可以检测到作用于转置控件144的用户操作。响应于上述操作，终端100可将图1F中竖向的小窗141调整为横向，参考图1H。
可选的，在确定主角之后，终端100可首先生成宽高比9:16的用于展示特写图像的小窗(竖窗)，参考图1D中的小窗141。上述宽高比为示例性例举，竖窗的宽高比包括但不限于9:16这一类。其中，可选的，终端100可固定地在屏幕的左下方(或右下方、左上方、右上方)显示上述小窗141。当检测到作用于转置控件144的用户操作之后，终端100可将原来的竖窗变更为宽高比16:9的横向小窗(横窗)。当然，终端100也可默认生成横窗，然后，根据用户操作将横窗调整为竖窗，本申请实施例对此不作限制。这样，用户可用过转置控件144调整特写视频的视频内容和视频格式，以满足自身个性化需求。
在一些示例中，上述小窗还可根据预览窗中的主角的位置，调整显示位置，以避免对预览窗113中主角的造成遮挡。进一步的，终端100还可根据用户操作调整小窗的位置和大小。在一些示例中，终端100还可检测到作用于小窗141的长按操作和拖动操作。响应于上述操作，终端100可将小窗移动到用户拖动操作最后停下的位置。
在另一些示例中，终端100还可检测到作用于小窗141的双击操作，响应于上述操作，终端100可将放大或缩小小窗141。不限于上述介绍的长按操作、拖动操作以及双击操作，终端100还可通过手势识别和语音识别来控制调整小窗的位置和大小。例如，终端100可通过摄像头采集的图像识别到用户做出了握拳手势，响应于上述握拳手势，终端100可缩小小窗141。终端100可通过摄像头采集的图像识别到用户做出了张手手势，响应于上述张手手势，终端100可放大小窗141。
可选的，在进入主角模式之后，在预设时间内仅识别到第一个对象，终端100也可默认地将上述对象设定为主角。例如，参考图1B所示的用户界面，在5秒内，预览窗113中显示的已识别的可作为主角的对象均只有人物2一个，此时，终端100可默认地将上述人物2设定为主角。这样，在典型的单人拍摄场景中，终端100可以智能地确定主角，从而节省用户操作。
可选的，在进入主角模式之后，在识别到多个对象时，终端100也可根据预设的评价规则，从上述多个对象中的选择出一个最佳对象，作为主角。上述评价规则例如人物图像大小，人脸区域的图像亮度等等，这里不再一一例举。
在开始录制视频之前，在确定主角之后，若选定的主角丢失(预览窗113中显示的图像中不包括主角)，则小窗141中显示的主角的特写图像会定格在丢失前的最后一帧。
参考图1D，预览窗113中显示图像可以为摄像头采集的第n帧图像。此时，小窗141中显示的图像为基于上述第n帧图像得到的以主角(人物2)为中心的特写图像。参考图1I，预览窗113中显示图像可以为摄像头采集的第n+1帧图像。此时，第n+1帧图像中不包括之前选定的主角(人物2)，即主角丢失。这时，小窗141中保持显示基于上述第n帧图像得到的以主角(人物2)为中心的特写图像。
如图1I所示，在检测到主角丢失之后，终端100还可显示在预览窗113中显示提示语151,例如“主角丢失，5秒后退出追焦”，用于提示用户调整摄像头位置或角度，使得终端100可以重新获取到包括主角的原始图像。
如图1J所示，如果在5秒后，终端100仍未找回主角(人物2)，即摄像头采集的图像中仍然不包括之前选定的主角(人物2)，则终端100可关闭小窗141，并取消之前选定的主角(人物2)。上述5秒是预设的，终端100还可设定其他时间长度，例如10秒等等，本申请实施例对此不作限制。
在预览过程中，在确定主角之后，终端100可开始录制视频。在本申请实施例提供的主角模式下，终端100可基于预览窗113中显示的原始图像生成原始视频，同时，终端100还可基于小窗141中的主角的特写图像生成特写视频。
如图2A所示，终端100可检测到作用于拍摄控件112的用户操作。上述操作可称为开始拍摄(开始录制)的用户操作。拍摄控件112也称录制控件。响应于上述操作，终端100可开始录制视频(原始视频和特写视频)，即编码并保存摄像头采集的原始图像，和以主角为中心的特写图像。这时，原始视频和特写视频的开始录制时刻是相同的。在一些示例中，上述开始拍摄的用户操作还可以是预设的用户肢体动作或语音指令。其中，用户肢体动作包括手势动作(例如举手、握拳)、体势动作(例如跳跃、奔跑)。
在开始录制视频之后，如图2A所示的用户界面可变更为图2B所示的样子。如图2B所示，在开始录制视频之后，终端100可显示控制模组211。控制模组211中可包括暂停控件2111和停止控件2112。暂停控件2111可用于暂停录制视频，包括暂停录制预览窗113对应的原始视频，和暂停录制小窗141对应的特写视频。停止控件2112可用于停止录制视频，包括停止录制原始视频，和停止录制特写视频。
在开始录制视频之后，预览窗113和小窗141中均可显示时间戳。例如预览窗113左上角显示的时间戳“00:01”，小窗141左上角显示的时间戳“00:01”。初始时，预览窗113和小窗141中的时间戳是相同的。后续，根据主角在预览窗113中的出镜情况，预览窗113和小窗141中的时间戳可能会不同，这里先不展开描述。
可选的，终端100也可在开始录制视频之后，确定主角，生成小窗141并显示主角的特写视频。这时，终端100先检测到开始录制的用户操作，然后再检测到点击某一选择框选定对应的对象为主角的操作。上述检测到开始录制的用户操作对应的视频为原始视频的开始录制时刻。上述点击某一选择框选定对应的对象为主角的操作对应的时刻为特写视频开始录制的视频。因此，在上述录制场景中，特写视频的开始录制时刻在原始视频之后。
可选的，在开始录制视频之后，小窗141中还可显示停止控件212。停止控件212可用于停止录制特写视频。在检测到的作用于停止控件212的用户操作之后，终端100可关闭小窗141，并停止录制小窗141对应的特写视频。在停止录制小窗视频之后，终端100可基于已录制的小窗中的图像生成一个特写视频。这时，终端100不停止录制原始视频。
此后，可选的，用户可以重新选择主角。在选定新的主角之后，终端100可重新显示小窗141，并在小窗141中显示新主角的特写视频，录制新主角的特写视频。上述新的主角可以与之前选定的主角相同，也可以不同。这时，终端100会将录制的小窗中的图像保存为另一个特写视频。
可选的，小窗141中还可显示暂停控件213。在检测到的作用于暂停控件213的用户操作之后，终端100可暂停更新小窗141中显示的图像，同时也暂停录制小窗141中的图像(即暂停录制特写视频)。在检测到的作用于暂停控件213的用户操作之后，终端100可显示继续录制控件(图2B中未体现)替代暂停控件213。在检测到的作用于继续录制控件的用户操作之后，终端100可恢复更新小窗141中显示的图像，同时恢复录制特写视频，并在小窗141中显示暂停控件213替换上述继续录制控件。
终端100还可监听作用于屏幕的用户操作。在显示图2B所示的用户界面的基础上，当5秒内未监听到作用于屏幕上预览窗113的用户操作时，例如点击操作，终端100可以隐藏预览窗113中的选择框；当5秒内未监听到作用于屏幕上小窗141的用户操作时，终端100可以隐藏小窗141中的功能控件，例如小窗的时间戳、停止控件212。
在开始录制视频之后，终端100还可提供切换主角的服务。参考图2C所示的用户界面，在开始录制视频后的第5秒时，终端100可检测到作用于选择框121的用户操作。上述操作可称为切换主角的用户操作。响应于上述操作，终端100可将选择框121对应的人物1设定为主角。此时，之前设定为主角的人物2不再为主角。
参考图2D所示的用户界面，在将人物1设定为主角之后，终端100可在小窗141中显示人物1的特写图像，不再显示人物2的特写图像。适应性的，终端100可将人物1对应的选择框121更新为选中框221，同时，将人物2对应的选中框142更新为选择框122。
其中，在切换主角的过程中，小窗141可以直接显示切换后的人物1的特写图像，呈现跳跃式的显示效果。可选的，小窗141还可通过平滑策略实现非跳跃式的主角切换显示效果。例如，在将主角切换为人物1之后，终端100可根据预览窗113中人物2到人物1的路径，确定一组平滑移动的图像帧，然后在小窗141中显示上述图像帧，以实现非跳跃式的主角切换显示。例如，终端100还可使用固定的过渡效果，连接切换前后的主角的特写图像。上述固定的过渡效果例如视频编辑中常用的叠加、旋涡、平移等等。本申请实施例对此不作限定。
可选的，主角模式中还可预置有主角自动选定算法。该算法中预设有对象评价规则。主角自动选定算法可根据上述对象评价规则从识别到的多个对象中自动地确定表现最好的对象为主角。在根据上述主角自动选定算法确定主角的场景下，终端100也可根据各个对象的评分的变化切换主角。例如，在刚开始录制视频的过程中，人物2的评价最高，此时终端100可确定人物2为主角，后续，终端100可检测到人物1的评价高于上述人物2，此时，终端100可将主角由人物2更换为人物1。
在录制视频的过程中，用户初始选定的主角可能离开终端100的摄像头的取景范围。这时，预览窗113对应的原始图像也会出现主角丢失的情况。同样的，在识别到主角丢失之后，终端100也可显示主角丢失的提示，并定格小窗141中的主角的特写图像。
参考图2E所示的用户界面，在开始录制视频后的第10秒时，终端100可检测到预览窗113中显示的原始图像(摄像头采集的原始图像)包括人物2，但不包括人物1(主角)，即主角丢失。这时，终端100可在预览窗113中显示提示语231(“主角丢失，5秒后退出追焦”)，以提示用户调整摄像头位置或角度，使得终端100可以重新获取到包括主角的原始图像。同时，终端100可小窗141中保持显示前一时刻确定的主角(人物1)的特写图像。从用户角度，小窗141中显示的特写图像暂停了，定格在了前一时刻确定的主角(人物1)的特写图像上。相应的，小窗141中左上角显示的时间戳暂停计时。
在看到上述提示语231之后，用户可调整摄像头位置以使得主角在摄像头的取景范围内，从而使得摄像头可以重新采集到包括主角的图像。
如图2F所示，如果在5秒后(开始录制视频后的第15秒00:15)，终端100仍未找回主角(人物1)，即预览窗113中的图像中仍然不包括之前选定的主角(人物1)，则终端100可显示提示语232(“主角追焦已暂停”)。同时，终端100可在小窗141显示特写图像的图层之上显示一个半透明的灰色蒙版，以提示用户追焦录制已暂停。
可以理解的，在图2E所示的第10秒时，终端100就已经暂停录制小窗141中的特写视频了。其中，第10秒～第15秒这5秒的时间，是给终端100设置的用于用户调整的过渡时间。可选的，在第10秒～第15秒这5秒的时间内，终端100也可基于按主角丢失前的最后一帧的裁剪特写图像的方式，裁剪后续的预览窗113中的图像得到特写图像。此时，上述图像可不包括主角。
如果在暂停录制特写视频之后的某一时刻，终端100重新识别到了上述主角(人物1)，这时，终端100可在小窗141中显示新获取的主角(人物1)的特写图像，并继续录制特写视频。
例如，参考图2G所示的用户界面，在开始录制视频后的第18秒时，摄像头重新采集到了包括人物1的图像，即预览窗113中显示的图像再次包括人物1。这时，终端100可基于上述包括人物1的原始图像确定以人物1为中心的特写图像，然后，在小窗141中显示上述特写图像。相应的，小窗141中左上角显示的时间戳恢复计时。同时，终端100继续编码上述特写图像，即继续录制特写视频。
在录制视频一段时间之后，终端100可检测到结束拍摄的用户操作。参考图2H所示的用户界面，例如，在开始录制视频之后的第25秒时，终端100可检测到作用于停止控件2112的用户操作。上述用户操作可称为结束拍摄的用户操作。响应于上述结束拍摄的用户操作，终端100可停止编码图像，并将开始录制至结束录制期间已编码的图像封装为视频保存到本地存储器中。
其中，响应于上述结束拍摄的用户操作，终端100可停止编码预览窗113对应的原始图像，并将开始录制至结束录制期间已编码的原始图像封装为原始视频保存到本地存储器中。同时，终端100可停止编码小窗141对应的特写图像，并将开始录制至结束录制期间已编码的特写图像封装为特写视频保存到本地存储器中。
保存完成之后，终端100可显示图2I所示的用户界面。如图2H所示，终端100可重新显示拍摄控件112和回看控件114。此时，回看控件114上可显示用于指示上述已录制的原始视频和特写视频的缩略图。可选的，上述标识可以为上述原始视频的第一帧图像的缩略图，或者上述特写视频的第一帧图像的缩略图。
在显示图2I所示的用户界面的过程中，终端100可检测到作用于回看控件114的用户操作。响应于上述操作，终端100可播放已拍摄原始视频和/或特写视频。这样，用户立刻查看上述原始视频和/或特写视频。
图3A示例性示出了终端100上图库应用提供的一个用于展示本地保存的视频和/或图片用户界面。图库应用(或称相册应用)是终端100上管理本地保存的视频和/或图片的应用。
如图3A所示，该用户界面可显示多个缩略图图标。一个缩略图图标对应一次拍摄操作得到的视频或图片。示例性的，上述多个缩略图图标可包括图标313。图标313可对应前述图2A-图2I所示的拍摄操作生成的视频。
终端100可检测到作用于图标313的用户操作。响应于上述操作，终端100可展示前述图2A-图2I所示的拍摄操作拍摄的视频：原始视频和特写视频，参考图3B。
如图3B所示，该用户界面可包括窗口322和窗口323。其中，窗口322可用于展示图2A-图2I所示的拍摄操作拍摄的原始视频；窗口323可用于展示图2A-图2I所示的拍摄操作拍摄的特写视频。
在一些示例中，终端100在显示图3B所示的用户界面时，可同时播放原始视频和特写视频。其中，特写视频的视频帧与原始视频对齐。这样，用户可以同时浏览原始视频和特写视频。在一些示例中，终端100也可先播放原始视频后播放特写视频，以便用户单独浏览各个视频。
进一步的，在图3B所示的用户界面的基础上，终端100可检测到作用于原始视频或特写视频的用户操作，例如点击操作。响应于上述操作，终端100可单独地展示上述视频。以原始视频为例，在检测到作用于原始视频的点击操作之后，终端100可单独地播放原始视频，以便于用户可以浏览分辨率更高的更清晰的原始视频。
图3C示例性示出了另一种终端100展示本地保存的视频和/或图片的用户界面。
在图3C所示的用户界面中，终端100可以显示两个缩略图图标，例如图标331，图标332。这两个缩略图图标分别对应主角模式下拍摄得到的原始视频和特写视频。例如，图标331可对应上述原始视频，图标332可对应上述特写视频。在检测到作用于图标331的用户操作之后，终端100可展示原始视频。在检测到作用于图标332的用户操作之后，终端100可展示特写视频。
其中，图标331，图标332可包括相同的图像内容，例如用户选定的主角人物2。这样，用户可以通过图像中显示的图像内容，直观清晰地确定上述两个图标对应的视频是在主角模式下拍摄的一组视频。
实施图1A-图1J、图2A-图2I所示的自动追焦拍摄方法(主角模式)，终端100可以实时地检测摄像头采集的原始图像中的对象。用户可以随时选择终端100已识别到的一个或多个对象中的任一对象为主角，也可以随时切换主角。
在开始录制之前，当已设定为主角的对象丢失时超过一段时间，终端100可取消上述对象的主角资格，然后，指示用户重新选定主角。在录制过程中，当已设定为主角的对象丢失时，终端100可暂停录制；当找回主角时，终端100可以继续录制。这样，用户可以得到连贯的以主角为中心的特写视频，并且上述主角不限定为一个对象。
在录制结束后，终端100可同时保存原始视频和特写视频。其中，原始视频可以保留录制过程中摄像头采集到的全部图像内容。特写视频可以集中地展示用户选定的主角的视频内容。用户可根据不同的需求，浏览或使用上述原始视频或特写视频，从而为用户提供更丰富的使用体验。
图4示例性示出了终端100的软件结构示意图。
图4所示的软件结构示意图可包括：相机应用CameraApp、相机硬件抽象CameraHAL。
终端100可采用分层架构。层与层之间通过软件接口通信。以安卓Android系统为例，分层架构从上至下分别包括：应用程序层，应用程序框架层，安卓运行时(Android runtime)和系统库，以及内核层。
相机应用可设置在应用程序层。应用程序层可以包括一系列应用程序包，例如相机应用，图库应用，电话应用等。其中，本申请实施例主要涉及相机应用。
相机应用中可包括一个或多个功能模块。一个功能模块对应相机应用提供的一类相机服务。图4中示出了录像模块。录像模块可为用户提供录像服务。此外，相机应用中还可其他功能模块。结合图1A所示的菜单栏111，上述其他功能模块还包括拍照模块(提供拍照服务)、人像模块(提供拍摄人像照的服务)、夜景模块(提供针对夜间的拍摄服务)等等。
参考图1A所示的闪光灯1151、滤镜1152、主角模式1154等控件，录像模块可包括闪光灯模块、滤镜模块以及主角模式模块等。闪光灯模块可用于控制开启或关闭闪光灯。滤镜模块可用于控制开启或关闭滤镜以及选择滤镜种类。主角模式模块用于控制启用前述用户界面介绍的自动追焦主角的拍摄功能。
进一步的，主角模式模块可包括特性开关模块、功能模块、大窗控制模块、小窗控制模块、开始录制模块、暂停录制模块、结束录制模块。
特性开关模块中记录了开启主角模式的处理逻辑。功能模块中记录了全局的主角模式拍摄方法的处理逻辑。大窗控制模块中记录了主角模式中与预览窗显示相关的处理逻辑。小窗控制模块中记录了主角模式中与小窗显示相关的处理逻辑。开始录制模块中记录了主角模式中开始录像的处理逻辑。暂停录制模块中记录了主角模式中暂停录像的处理逻辑。结束录制模块中记录了主角模式中结束录像的处理逻辑。
相机硬件抽象可设置在系统库。在本申请实施例中，相机硬件抽象可包括帧流模块Stream、帧预处理模块framePreprocessing、对象检测模块tackDetect、追焦模块tracking。
帧流模块可用于缓存摄像头采集的图像帧流。帧预处理模块可用于对已获取到的图像帧执行美颜、美体、滤镜等图像处理算法。对象检测模块中包括对象识别算法。对象检测模块执行对象识别算法，可识别任意一帧图像中包括的一个或多个对象。追焦模块中包括追焦算法。追焦模块执行追焦算法，可定位在后图像帧中的主角，从而使得相机应用可以提供针对主角进行自动追焦拍摄的服务。
可选的，相机应用的中的功能模块也可设置在应用程序框架层或系统库，相机硬件抽象也可设置在内核层等。相机应用的中的功能模块在相机硬件抽象之上。可以理解的，图4所示的软件结构仅为一种可选的示例，不应构成对本申请的限制。
在开启相机应用之后，终端100中的安装的摄像头会进入工作状态，采集光信号并生成对应的电信号表示的图像。生成的图像首先存储在摄像头的缓存(RAW-buffer)中。然后，摄像头可实时地将当前生成的图像帧上报到CameraHAL。CameraHAL可对接收到的图像帧进行处理然后发送到屏幕预览窗对应的区域进行显示。CameraHAL发送到屏幕预览窗对应的区域进行显示的图像即本申请实施例所述的原始图像。
在本申请实施例提供的主角模式的拍摄方法中，在CameraHAL送显原始图像的过程中，CameraHAL还可对上述原始图像进行图像识别和追焦，进而实现图1A-图1J、图2A-图2I、图3A-图3C所示的主角模式的拍摄功能。
首先，图5示例性示出了图1A-图1D所示的预览过程中选中主角并显示主角特写图像的流程图。
S501.特性开关模块检测到开启主角模式的用户操作。
结合图1A所示的用户界面，在调用摄像头录像的场景下，用户可以通过作用于主角模式1154的操作，指示终端100开启主角模式，进而使用主角模式录制以主角为中心的特写视频以及原始视频。在上述过程中，CameraApp中的特性开关模块可检测到上述作用于主角模式1154的开启主角模式的用户操作。
在检测到开启主角模式的用户操作之后，S502.特性开关模块向可向功能模块发送指示加载主角模式程序块的控制信息。响应于上述控制信息，S503.功能模块加载主角模式程序块。功能模块可将存储器中存储的主角模式程序块加载到内存中。处理器可从内存中读取上述程序块中的程序代码，并依次执行，从而运行本申请实施例提供的主角模式的拍摄方法。
S504.功能模块调用对象检测接口进行对象识别。
在完成加载主角模式程序块的操作之后，在执行主角模式的程序代码的过程中，首先，功能模块可调用对象检测接口(对象检测函数)进行对象识别。对象检测接口可用于调用CameraHAL中的对象检测模块执行对象识别算法，识别摄像头采集的图像中包括的对象。
S505.对象检测模块获取即将送显的一帧原始图像，执行对象识别算法，确定该图像中包括的对象及对应的特征信息。
响应于S504所示的调用动作，对象检测模块可从摄像头的RAW-buffer中获取当前帧(摄像头最新生成的一帧图像)。然后，对象检测模块可直接将上述图像帧输入到对象识别算法中，识别并输出该帧图像中的对象及对象特征信息。摄像头可连续地采集并生成图像(原始图像流)，对象检测模块可连续地识别原始图像流中各图像中的包括的对象。
上述对象特征信息包括对象数量和各个对象的特征信息。一个对象的特征信息包括但不限于该对象的对象标识(objectId)、面部区域(face)、人体区域(body)、中心点(center)，以及追踪标签(isTracking)。
对象标识用于唯一标识一个对象。面部区域是指一个对象的面部图像在原始图像中的图像区域。人体区域是指一个对象的面部图像在原始图像中的图像区域。中心点是表示一个对象图像中心位置坐标。可选的，上述中心点可以是人体区域的中心点、或面部区域的中心点，还可以是基于人体区域的中心点和面部区域的中心点确定的一个点坐标。追踪标签用于指示该对象是否为主角。例如，isTracking＝1可表示该对象被设定为追踪对象(即主角)；isTracking＝0可表示该对象未被设定为追踪对象。初始的，追踪标签均为指示未被追踪的标签(isTracking＝0)。
示例性的，以图6所示图像S1为例，执行对象识别算法，对象检测模块可确定S1的对象特征信息，参考表1：
表1
以objectId＝1为例，objectId＝1可表示人物1。Face1可表示人物1的面部区域。Body1可表示人物1的人体区域。C1可表示人物1的图像中心点。其中(x1,y1)为上述中心点的坐标。isTracking＝0可表示人物1未被设定为追焦对象。
在本申请实施例中，对象识别算法主要是针对人物这一类对象的识别算法，即人体检测算法。在其他实施中，终端100还可支持追焦植物、动物、建筑等类型的对象。这时，上述对象识别算法还包括识别特定植物、动物、建筑的识别算法，这里不再赘述。
在一些实施例中，终端100可以利用人体检测算法确定图像中包括的一个或多个人物。在一些实施例中，终端100还可以利用人体检测算法和人脸检测算法确定图像中包括的一个或多个人物。当人体检测算法输出的一个人体图像和人脸检测算法输出的一个人脸图像匹配时，终端100可确定一个人物。在多人场景中，上述方法可以减少人物图像重叠的干扰，提升人物识别的准确性。
可选的，在调用对象检测接口时，功能模块还可确定RAW-buffer的地址，然后在调用对象检测模块进行识别时，可将上述地址发送给对象检测模块。或者，功能模块也可在RAW-buffer的地址之后，从RAW-buffer中获取摄像头采集的图像，然后将上述图像发送给对象检测模块。这样，对象检测模块可直接从功能模块获取原始图像，或根据功能模块传输的地址获取原始图像。
S506.对象检测模块向功能模块返回对象特征信息(对象标识，面部区域，人体区域，中心点，追焦标签)。
在对象检测模块通过对象识别算法确定原始图像中的对象特征信息之后，对象检测模块可将上述特征信息返回给功能模块。例如，对象检测模块可将上述表1所示的对象特征信息返回给功能模块。这样，功能模块即可基于接收到的对象特征信息确定摄像头上报的最新一帧图像(即将送显的一帧图像)中是否包括一个或多个可作为主角的对象。
如果接收到的对象特征信息指示上述即将送显的一帧图像中包括一个或多个对象，则功能模块可确定将要送显的图像中包括一个或多个对象，进而功能模块可依据上述特征信息中的中心点，生成对应各个对象的标记(选择框)。反之，如果接收到的对象特征信息指示上述即将送显的一帧图像中无对象，则功能模块可确定将要送显的图像中无对象，进而功能模块可确定无需生成标记，随之，用户也不能选择任意对象作为主角。
S507.当接收到的对象特征信息指示图像中包括一个或多个对象时，功能模块基于对象特征信息中的中心点确定各个对象的选择框的显示坐标。
在接收到上述对象特征信息之后，功能模块可依据对象特征信息中的中心点确定与已识别到的对象一一匹配的选择框，并确定各个选择框的显示坐标。选择框的中心点即上述对象特征信息中各个对象的中心点。
结合图1B所示的用户界面，终端100在显示摄像头采集的原始图像时，还会显示与上述原始图像中各个对象对应的选择框，例如选择框121、122。上述选择框的中心点即图1B预览窗113中人物1、人物2的中心点。
以人物1(objectId＝1)为例，在接收到表1所示的人物1的对象特征信息之后，功能模块可确定人物1的中心点C1(x1,y1)。然后，功能模块可生成与人物1对应的选择框121。在上述过程中，功能模块可通过中心点C1(x1,y1)确定选择框121的显示坐标C_screen(X1,Y1)。同理，功能模块可确定其他对象的选择框的显示坐标：选择框122的显示坐标C_screen(X2,Y2)。
S508.功能模块向大窗控制模块发送显示选择框的指示。
在确定图像中各个对象对应的选择框的显示坐标之后，功能模块可向大窗控制模块发送显示选择框的控制信息。上述指示中可携带各个选择框的显示坐标，例如C_screen(X1,Y1)、C_screen(X2,Y2)，以供大窗控制模块确定各选择框在屏幕上的显示位置。
S509.大窗控制模块绘制选择框并控制显示屏显示选择框。
在接收到上述显示选择框的指示之后，大窗控制模块可确定绘制并显示选择框。其中，通过上述控制信息中携带的显示坐标，大窗控制模块可绘制与原始图像中各个对象对应的选择框。然后，大窗控制模块可将已绘制的选择框的显示数据发送到显示器，从而在屏幕上显示上述各个选择框。
参考图1B所示的用户界面，此时，终端100可在显示图像的同时，在图像中各个对象之上显示与之对应的选择框，从而为用户提供选择主角的功能。
S510.大窗控制模块检测到选择用户的点击操作。
在大窗控制模块可监测用户的点击操作。在显示选择框之后的某一时刻，在大窗控制模块可检测到用户点击屏幕的操作。大窗控制模块可确定上述点击操作作用在屏幕上的坐标(点击坐标)，例如click_screen(X0,Y0)。
结合图1C所示的用户界面，在显示选择框121、122之后，终端100可检测到作用于选择框122的用户操作。可以理解的，大窗控制模块在检测到点击操作时，并不确定该点击操作是否是点击选择框的操作。大窗控制模块只需确定该点击的操作在屏幕上的点击坐标。
S511.大窗控制模块向功能模块返回点击坐标。
S512.功能模块依据点击坐标确定追焦对象。
在接收到大窗控制模块返回的点击坐标之后，功能模块可首先确定上述点击坐标对应的图像坐标，即确定点击操作作用在图像中的位置。然后，功能模块可依据上述图像坐标和各个对象的中心点，确定用户点击的对象，即确定追焦对象(主角)。
例如，在接收到click_screen(X0,Y0)之后，功能模块可依据上述click_screen(X0,Y0)确定用户点击操对应的图像坐标C(x0,y0)。然后，可选的，功能模块可计算上述图像坐标C(x0,y0)与图像中个对象的中心点：C1(x1,y1)、C2(x2,y2)的距离。功能模块可确定距离最小的中心点对应的对象为追焦对象。
例如，结合图1C所示的用户界面，功能模块可确定C2(x2,y2)与C(x0,y0)之间的距离dis[C,C2]最小，于是，终端100可确定C2(x2,y2)对应的人物2(objectId＝2)为追焦对象，S510所示的点击操作为点击人物2的选择框122的操作。此时，人物2可称为追焦对象或主角。随后，功能模块可更新人物2的追踪标签，例如将原来的isTracking＝0更新为isTracking＝1。
S513.功能模块向追焦模块下发追焦对象信息(对象标识，面部区域，人体区域，中心点，追焦标签，时间戳)。
在确定追焦对象之后，功能模块可调用追焦接口。追焦接口可用于调用CameraHAL中的追焦模块执行追焦算法，识别后续原始图像流中是否包括的用户选定的追焦对象(主角)。
在调用追焦接口时，功能模块可向追焦模块发送用户选定的追焦对象的特征信息。以上述追焦对象为人物2为例，上述功能模块发送的人物2的特征信息可以为：例如face＝Face2,body＝Face2,center＝C2,isTracking＝1,objectId＝2。上述特征信息用于追焦模块在接下来摄像头采集的图像中定位人物2，即定位用户选定的主角。
此外，功能模块还会向追焦模块发送时间戳timestamp。上述时间戳为功能模块调用追焦接口的时间戳。上述时间戳可用于确定从摄像头采集的哪一帧图像开始执行追焦算法。
在支持选定多个主角的场景下，大窗控制模块可检测到多个点击选择框的用户操作，功能模块可基于上述多个点击操作确定多个追焦对象，然后向追焦模块下发上述多个追焦对象。
S514.追焦模块从对象检测模块中获取经过对象检测的原始图像。
在接收到追焦对象信息之后，追焦模块可首先获取待追焦图像，即摄像头采集的即将送显的一帧原始图像。追焦模块可从对象检测模块中获取上述待追焦图像。
在一些实施例中，对象检测模块的缓存中仅存储一帧经过对象识别的原始图像。这时，追焦模块可直接从上述缓存中获取该缓存中当前存储的图像帧。上述当前存储的图像帧即待追焦图像。
在另一些实施例中，对象检测模块的缓存中可存储多帧经过对象识别的原始图像。其中，上述缓存中的最后一帧为待追焦图像。这时，若没有下发追焦对象时刻的时间戳，则追焦模块可能会获取到已经送显过的图像帧，从而使得预览窗中显示的原始图像和小窗中显示的主角的特写图像不匹配。鉴于此，功能模块在下发追焦对象的特征信息时还可发送当前时间戳，追焦模块可根据功能模块上述时间戳从上述缓存中获取待追焦图像，从而避免图像不匹配的问题。
在从对象检测模块中获取经过对象检测的原始图像时，除了获取图像数据，追焦模块还可获取该图像对应的对象特征信息，以供后续确定该图像中的追焦对象。
S515.追焦模块执行追焦算法，确定追焦状态(成功、失败、失焦)及追焦对象的特写图像。
在接收到追焦对象的特征信息，且确定待追焦图像之后，追焦模块可执行追焦算法。通过上述追焦算法，追焦模块可确认待追焦图像中是否包括上述特征信息指示的追焦对象。如果待追焦图像中包括上述特征信息指示的追焦对象，则追焦模块可确定追焦状态为成功；反之，追焦模块可确定追焦状态为失败。在成功时，追焦模块可进一步生成以上述追焦对象为中心的特写图像。
以摄像头采集的第i帧图像为待追焦图像为例：经过S514，追焦模块可获取到第i帧图像及其对应的对象特征信息。然后，追焦模块可计算第i帧图像中各个对象与S513中下发的追焦对象的相似度。第i帧图像中与下发的追焦对象相似度最高且高于门限值的对象，可确定为第i帧图像中的追焦对象。
例如，追焦模块在从对象检测模块获取第i帧图像时，追焦模块还可获取到上述第i帧图像中各个对象的特征信息，参考表2：
表2
然后，追焦模块可分别计算追焦对象(表1:objectId＝2)与第i帧图像中各个对象(objectId＝i1,objectId＝i2)的相似度。示例性的，追焦模块可确定objectId＝i2与objectId＝2的相似度最高且高于预设的门限值。这时，追焦模块可确定objectId＝i2对应的对象为第i帧图像中的追焦对象。
当第i帧图像匹配到追焦对象时，追焦模块可确定当前帧的追踪状态为“成功”，反之，追焦模块可确定当前帧的追踪状态为“失败”。同时，由于视频流是实时生成的，在确定第i帧图像的追踪状态之后，追焦模块可继续确定第i+1帧的追焦状态，并向功能模块返回第i+1帧的追焦状态。其中，在第i帧图像状态为“成功”时，若第i+1帧图像匹配到追焦对象，追焦模块可确定第i+1帧图像的追踪状态为“成功”；若第i+1帧图像未匹配到追焦对象，追焦模块可确定第i+1帧图像的追踪状态为“失焦”。
此外，在确定objectId＝i2对应的对象为第i帧图像中的追焦对象后，追焦模块可将上述objectId＝i2对应的isTracking设定为1，表示该对象为追焦对象。同时，追焦模块还可基于上述追焦对象确定特写图像，以供后续CameraAPP获取该特写图像并显示。
在一些实施例中，追焦模块计算第i帧图像中各个对象与上述下发的追焦对象的相似度的具体过程如下：首先，追焦模块可根据对象检测模块输出的对象数量和各个对象的图像区域(例如图6中的人体框body1)，确定第i帧图像中的人物是否重叠。
在不重叠的场景下，终端100可以利用各个对象的图像区域与下发的追焦对象的图像区域的交并比IoU，来确定第i帧图像中的追焦对象。其中，一个对象的图像区域可通过对象特征信息中的人体区域表示，即追焦模块可通过第i帧图像中的人体区域的IoU进行主角追焦。第i帧图像中与下发的追焦对象的IoU最大的对象可被确定为第i帧图像中的追焦对象。
在重叠的场景下，终端100可以利用各个对象的图像区域的交并比IoU以及图像内容的相似度确定第i帧图像中的追焦对象。其中，第i帧图像中与下发的追焦对象的IoU和图像内容相似度的和最大的对象可被确定为第i帧图像中的追焦对象。
S516.追焦模块向功能模块返回追焦状态(成功、失败、失焦)。
在确定追焦状态之后，追焦模块可首先将上述追焦状态返回到功能模块。一帧图像的追焦状态可以为成功、失败、失焦中任意一类状态。每确定一帧图像的追焦状态，追焦模块可向功能模块返回该帧图像的追焦状态。功能模块接收到的成功的追焦状态可称为追焦成功指示；失焦的追焦状态可称为失焦指示。
S517.功能模块确定追焦状态。在接收到CameraHAL中追焦模块返回的追焦状态后，功能模块可确认上述追焦状态为成功、失败或失焦。
S518.当追焦状态为追焦成功时，功能模块向小窗控制模块发送开启小窗的指示。
S519.小窗控制模块绘制小窗并控制显示器显示小窗。
响应于上述开启小窗的指示，小窗控制模块可绘制小窗，例如图1D所示的小窗141。然后，小窗控制模块可将绘制好的小窗的图像发送到显示器。显示器在接收到小窗的图像之后在显示器对应的位置上显示小窗。
S520.小窗控制模块从追焦模块中获取追焦对象的特写图像，并在小窗中显示。
参考S515介绍的，在成功时，追焦模块可进一步生成以追焦对象为中心的特写图像。 具体的，小窗控制模块可通过追焦模块提供的读取特写图像的通道(session)，获取特写图像，从而在小窗中显示追焦对象的特写图像。上述通道为追焦模块中用于存放特写图像的一段存储空间。
例如，在送显周期到来时，CameraAPP可接收到CameraHAL上报的第i帧图像及其对应的主角特写图像。这时，CameraAPP中的大窗控制模块可在预览窗中显示第i帧图像，CameraAPP中的小窗控制模块可在已绘制的小窗中显示上述主角特写图像。
S521.功能模块向大窗控制模块发送显示选择框的指示。
当追焦状态为追焦成功时，功能模块还可向大窗控制模块发送显示对象标记(选择框和/或选中框)的指示。其中，追焦对象(主角)的标记为选中框，非主角的标记为选择框。上述控制信息中可携带各个对象标记的显示坐标。可以理解的，S518和S521无先后顺序，均为S517之后，功能模块执行的操作。
S522.大窗控制模块绘制选择框并控制显示器显示选择框。
在接收到上述显示选择框的指示之后，大窗控制模块可确定绘制并选择框。其中，根据上述显示坐标，大窗控制模块可确定各个选择框在屏幕中的具体显示位置。此时，大窗控制模块还可根据第i帧图像的对象特征信息(isTracking)，确认该图像中是否有追焦对象，以及追焦对象具体为哪一个。然后，大窗控制模块可确定选择框的类型：表示未选中的选择框(选择框122)，或者表示已选中的选择框(选中框142)。
例如，根据表2所示的第i帧图像的对象特征信息(isTracking)，大窗控制模块可人物2对应的表示已选中的选择框(选中框)，而人物1对应表示未选中的选择框。这时，大窗控制模块可绘制不同类型的选择框，从而使得用户可以方便地区分主角和非主角。
在S517中，如果功能模块接收到的追焦状态为失败时，则功能模块不会执行任何动作。终端100会按照前述介绍的方法继续显示摄像头采集的图像、识别图像中的对象并显示与各个对象对应的选择框，等待用户的选择主角的操作。可选的，如果功能模块接收到的追焦状态为失败，则功能模块也可指示大窗控制模块在预览窗中显示追焦失败的提示，指示用户重新选择主角。
在追焦成功的基础上，如果某一帧中追焦模块未识别到追焦对象，则追焦模块可向功能模块返回失焦的追焦状态，并控制小窗暂停显示特写图像甚至关闭小窗。图7示例性示出了图1I-图1J所示的预览过程中失焦的流程图。
在预览场景(未开始录制)中，S523.当追焦状态为追焦失焦时，功能模块启动计时器。
参考图1I-图1J以及后续录像过程中图2E-图2F的介绍，刚开始检测到追焦对象丢失时，小窗141可以立即暂停小窗，也可以等待一段时间再暂停小窗。其中，在等待的过程中，小窗中可以显示根据最后一帧特写图像在原始图像中的位置裁剪得到的图像，也可以直接显示经过等比例缩放的原始图像。
以等待一段时间再暂停小窗的方法为例，当功能模块接收到的追焦状态为“失焦”时，功能模块可启动计时器。功能模块在启动计时器是可设定计时器的计时时长。结合图1I所示的用户界面的介绍，上述计时器的计时长度可以为5秒。
在等待一段时间再暂停小窗的方法中，在确定追焦状态为失焦之后，追焦模块可按照丢失之前最后一帧的裁剪方式继续裁剪后续几帧原始图像，得到送显小窗的图像。相应地，小窗控制模块可以继续从追焦模块提供的session中获取送显小窗的图像，然后显示。可以理解的，此时，送显小窗的图像内容不包括前述设定的主角。
S524.当计时器计时结束时，功能模块向小窗控制模块发送暂停显示的控制信息。
以上述计时时长5秒为例，当计时器计时结束且cameraHAL返回的追焦状态仍然是失焦时，功能模块可向小窗控制模块发送关闭小窗的控制信息。
其间，若cameraHAL返回的追焦状态为成功时，追焦模块可在session中存入追焦对象的特写图像。对应的，小窗控制模块可从session中获取到上述特写图像。这时，用户可以在小窗中看到新的特写图像。在cameraHAL返回的追焦状态为成功时，功能模块可关闭计时还未结束的计时器。
参考图2E-图2F的介绍，在开始录制的场景下，当计时器计时结束且cameraHAL返回的追焦状态仍然是失焦时，功能模块可向小窗控制模块发送暂停录制的控制信息，而不会直接关闭小窗，后续实施例再详细介绍。
S525.小窗控制模块控制关闭小窗。
响应于上述关闭小窗的控制信息，小窗控制模块可关闭小窗，并停止从session中获取特写图像。参考图1J所示的用户界面，预览窗113中不再显示小窗141。
可选的，在预览场景中，S526.当追焦状态为追焦失焦时，功能模块向小窗控制模块发送暂停显示的控制信息。
以立即暂停小窗的方法为例，当功能模块接收到的追焦状态为“失焦”时，功能模块可立即向小窗控制模块发送关闭的控制信息，而不再设定计时器、等待计时器计时结束后再发送关闭小窗的控制信息。
S527.小窗控制模块控制关闭小窗。同S525，这里不再赘述。
图8示例性示出了图2A-图2B所示的开始录制的过程中的流程图。
S601.开始录制模块检测到开始录制的用户操作。
在预览过程中，在确定追焦对象后，用户可通过拍摄控件开启视频录制。参考图2A所示的用户界面，终端100可检测到作用于拍摄控件112的用户操作，响应于上述操作，终端100可开始录制视频。具体的，CameraAPP中的开始录制模块可检测到上述开始录制的用户操作。
S602.开始录制模块控制大窗编码器和小窗编码器开始编码。
在检测到开始录制的用户操作之后，开始录制模块可开启大窗编码器和小窗编码器，并控制大窗编码器和小窗编码器可开始编码送显图像。大窗编码器用于编码大窗(预览窗113)中的图像流。小窗编码器用于编码小窗(小窗141)中的图像流。
S603.开始录制模块向大窗控制模块发送开始录制的指示。响应于上述指示，S604.大窗控制模块开启大窗计时器并显示计时结果。
录制过程中的预览窗视图可包括记录预览窗录制时长的时间标记。在接收到上述开始录制的指示信息之后，大窗控制模块可初始化大窗计时器，并开始计时。同时，大窗控制模块可在预览窗中显示上述计时器的计时结果。大窗中显示的计时结果即上述记录预览窗录制时长的时间标记。参考图2B所示的用户界面，预览窗113的左上角显示有录制时长“00:01”。上述录制时长“00:01”即当前大窗计时器的计时结果。
S605.开始录制模块向小窗控制模块发送开始录制的指示信息。同样的，在开启小窗编码器之后，开始录制模块还可向小窗控制模块发送开始录制的指示信息。可以理解的，S603-S604与S605-S606无时序上的先后关系。
响应于上述指示，S606.小窗控制模块开启小窗计时器并显示计时结果。
录制过程中的小窗视图可包括记录小窗录制时长的时间标记。在接收到上述开始录制的指示信息之后，小窗控制模块可初始化小窗计时器，并开始计时。同时，小窗控制模块可在小窗中显示上述计时器的计时结果。小窗中显示的计时结果即上述记录小窗录制时长的时间标记。参考图2B所示的用户界面，小窗141中的左上角显示有录制时长“00:01”。上述录制时长“00:01”即当前小窗计时器的计时结果。
S607.功能模块监听作用于屏幕的交互事件。
在检测到开始录制的用户操作之后，功能模块可实时地检监听作用于屏幕的交互控制事件，例如点击操作、滑动操作等交互事件。
S608.当预设时间内未监听到作用于屏幕中大窗区域的交互事件，功能模块向大窗控制模块发送隐藏选择框的指示信息。
以预设时间为5秒为例，当连续5秒内未监听到作用于屏幕中大窗区域的交互事件，则功能模块可向大窗控制模块发送隐藏选择框的指示信息。上述5秒为示例性例举的监听时长。功能模块也可设置其他时长的监听时长。上述监听时长可称为第一时长。功能模块可通过第一计时器计时所述第一时长。
在接收到隐藏选择框的指示信息之后，S609.大窗控制模块隐藏选择框。
S610.预设时间内未监听到作用于屏幕中小窗区域的交互事件，功能模块向小窗控制模块下发隐藏功能控件的指示信息。
同样以5秒为例，当连续5秒内未监听到作用于屏幕中小窗区域的交互事件，则功能模块可向小窗控制模块发送隐藏功能控件的指示信息。上述功能控件包括小窗中显示的录制时长、停止控件等，以减少对小窗中显示的图像的遮挡，提升用户浏览体验。
S611.小窗控制模块隐藏功能控件。
在接收到隐藏功能控件的指示信息之后，小窗控制模块可停止显示录制时长、停止控件等功能控件。同样的，S608-S609与S610-S611无时序上的先后关系。
S612.功能模块监听到作用于屏幕的交互事件。
在隐藏选择框之后的某一时刻，功能模块可监听到作用于屏幕的交互事件，例如点击屏幕中任意位置的用户操作。这里，作用于屏幕的交互时间包括作用于预览窗的交互事件和作用于小窗的交互事件。
在监听到作用于屏幕的交互事件之后，S613.功能模块向大窗控制模块发送显示选择框的指示。响应于上述指示，S614.大窗控制模块可显示选择框。这样，用户可以通过上述显示框选择或切换追焦对象。
在监听到作用于屏幕的交互事件之后，S615.功能模块向小窗控制模块发送显示功能控件的指示信息。响应于上述指示，S616.小窗控制模块显示功能控件。上述功能控件例如小窗141中的录制时长、暂停控件、停止控件等。这样，用户可以通过上述功能模块控制暂停或停止录制小窗。
可选的，功能模块可分别监听作用于屏幕中大窗区域的交互事件和小窗的交互事件。当监听到作用于屏幕中大窗区域的交互事件时，功能模块可指示大窗控制模块显示选择框。当监听到作用于屏幕中小窗区域的交互事件时，功能模块可指示小窗控制模块显示功能控件。
图9示例性示出了图2C-图2D所示的切换主角的过程中的流程图。
S701.大窗控制模块检测到切换追焦对象的用户操作。
在录制的过程中，用户可以随时切换追焦对象。例如，参考图2C所示的用户界面，终端100可检测到作用于选择框121的切换追焦对象为人物1的用户操作。具体的，CameraAPP中的大窗控制模块可检测到上述切换追焦对象的用户操作。
S702.大窗控制模块向功能模块返回点击坐标。
参考步骤S511，在检测到用户点击选择框以切换追焦对象的用户操作时，大窗控制模块也是首先确定上述点击操作作用在屏幕上的坐标(点击坐标)。然后，大窗控制模块可将上述点击坐标返回给功能模块。
S703.功能模块依据点击坐标更新追焦对象。
参考步骤S512，在接收到大窗控制模块返回的点击坐标之后，功能模块可首先确定上述点击坐标对应的图像坐标。在切换追焦对象的过程中，功能模块可依据上述图像坐标和各个对象的中心点，确定用户点击的对象，进而确定新的追焦对象(主角)。结合图2C所示的用户界面，这时，终端100可将人物1设定为新的追焦对象(主角)。
在更新追焦对象后，适应性地，功能模块可修改对应图像的对象特征信息。例如，功能模块可将之前的追焦对象人物2的追踪标签修改为0(isTracking＝0)，同时将新的追焦对象人物1的追踪标签修改为1(isTracking＝1)。
S704.功能模块向追焦模块下发更新后的追焦对象信息(face,body,center,isTracking,objectId，timestamp)。
在确定新的追焦对象之后，功能模块可调用追焦接口向追焦模块输入新的追焦对象的特征信息，例如人物1的特征信息：face＝Face1,body＝Face1,center＝C1,isTracking＝1,objectId＝1，以供追焦模块在原始图像流中定位新的追焦对象，并生成新的追焦对象的特写图像。参考步骤S513，同样的，功能模块还会向追焦模块发送当前时间戳timestamp，以供追焦模块从原始图像流中的哪一帧开始定位新的追焦对象。
S705.追焦模块从对象检测模块中获取经过对象识别的原始图像，对上述图像执行追焦算法，确定追焦状态(成功、失焦)。
参考S514-S515的介绍，追焦模块可根据上述时间戳timestamp确定开始定位新的追焦对象的第一帧图像。例如，上述timestamp可指示从第k帧图像开始追焦新的主角人物1。这时，追焦模块可从对象检测模块中获取经过对象识别的第k帧图像，然后确定该图像帧的追焦状态：成功或失焦。
以第k帧图像为例，从对象检测模块获取第k帧图像时，追焦模块还可获取到上述第k帧图像中各个对象的特征信息。参考步骤S515，然后，追焦模块可根据下发的追焦对象的特征信息，和第k帧图像中各个对象的特征信息，定位第k帧图像的追焦对象，这里不再赘述。同样的，在定位第k帧图像的追焦对象之后，追焦模块还可基于上述追焦对象确定特写图像，以供后续CameraAPP获取该特写图像并显示。
S706.追焦模块向功能模块返回追焦状态(成功、失焦)。
在确定追焦状态之后，追焦模块可首先将上述追焦状态返回到功能模块。
当追焦模块在第k帧图像中定位到了新的追焦对象时，追焦模块返回的第k帧图像的追焦状态为“成功”。当追焦模块在第k帧图像中未定位到新的追焦对象时，追焦模块返回的第k帧图像的追焦状态为“失焦”。
小窗控制模块会定时的从追焦模块提供的session中获取送显小窗的图像。当追焦状态为“成功”时，功能模块无需做任何变化。此时，追焦模块session中的特写图像已经根据追焦算法更换为新的追焦对象的特写图像。随之，小窗控制模块从session中获取的图像也相应地为新的追焦对象的特写图像。
S707.当追焦状态为失焦时，功能模块启动计时器。
当追焦状态为“失焦”时，功能模块可启动计时器。参考S523的介绍，功能模块在启动计时器是可设定计时器的计时时长。上述计时器的计时时长例如5秒。上述计时时长可称为第二时长。上述计时器可称为第二计时器。
S708.在计时结束后，功能模块向暂停录制模块发送暂停录制的指示。在设置计时器的同时，或设置计时器之后，功能模块还可向暂停录制模块发送暂停录制的指示。S709.暂停录制模块控制小窗编码器暂停编码。响应于上述指示，暂停录制模块中的小窗编码器可暂停编码。
S710.暂停录制模块向小窗控制模块发送暂停录制的指示。在暂停小窗编码器的之后，暂停录制模块还可向小窗控制模块发送暂停显示的指示。S711.小窗控制模块暂停小窗计时器。在接收到暂停显示的指示之后，小窗控制模块可停止小窗计时器的计时。这时，小窗中的时间戳可停止在失焦前的最后计时结果，例如“00:05”。同时，在接收到暂停显示的指示之后，小窗控制模块可暂停从session中获取并显示新的特写图像。这时，小窗中展示的特写图像可停止在当前显示的图像。
S712.在暂停录制之后，功能模块接收到成功的追焦状态。
在暂停录制之后，功能模块可接收到追焦模块传回的某一帧的追焦状态为成功。这时，小窗控制模块可从session中获取到新的追焦对象的特写图像，因此，小窗中可继续显示上述新的特写图像。相应的，S713.功能模块可向开始录制模块发送继续录制小窗的指示。响应于上述指示，S714.小窗编码器恢复编码。小窗编码器可继续编码小窗(小窗141)中的图像流。
同时，S715.开始录制模块可向小窗控制模块发送恢复录制的指示。响应于上述指示，S716.小窗控制模块恢复小窗计时器并继续显示特写图像。小窗控制模块可重新启动小窗计时器，并在小窗中重新显示小窗计时器当前计时。同时，小窗控制模块可继续从session中获取并显示追焦对象的特写图像。
图10示例性示出了暂停录制的过程中的流程图。
S801.暂停录制模块检测到暂停录制的用户操作。
在开始录制视频之后，用户可以通过拍摄控件暂停视频录制。例如，参考图2B所示的用户界面，暂停控件2111可用于暂停视频录制。当检测到作用于暂停控件2111的用户操作时，终端100可确认检测到暂停录制的用户操作。
响应于上述用户操作，S802.暂停录制模块可控制大窗编码器和小窗编码器停止编码。同时，S803.暂停录制模块可向大窗控制模块发送暂停录制的指示。响应于上述指示，S804.大窗控制模块可暂停大窗计时器。这时，大窗中的时间戳可停止在当前的计时时刻。用户可以通过上述计时时刻不再增长确定已经暂停大窗录制。
同时，S805.暂停录制模块还可向小窗控制模块发送暂停录制的指示。响应于上述指示，S806.小窗控制模块可暂停小窗计时器。这时，小窗中的时间戳可停止在当前的计时时刻。用户可以通过上述计时时刻不再增长确定已经暂停小窗录制。S803-S804与S805-S808无时序上的先后关系。且，从用户看来，上述大窗计时器和小窗计时器是同时停止计时。
S807.追焦模块向功能模块返回追焦状态。
在暂停录制视频的过程中，摄像头仍然可以采集并生成图像，并在生成图像之后依次将生成的图像帧上报给CameraHAL。相应的，追焦模块在每接收到摄像头上报的一帧之后，就对该帧执行追焦算法，确定各图像帧中是否包括追焦对象，进而确定追焦状态，并将上述追焦状态传回功能模块。
在录制过程中，S808.在大窗小窗均暂停录制的情况下，当返回的追焦状态为失焦时，功能模块可向小窗控制模块发送追焦对象丢失的指示。响应于上述指示，S809.小窗控制模块控制小窗显示蒙层。
在录制过程中，S810.在大窗小窗均暂停录制的情况下，当返回的追焦状态为“失焦”时，功能模块还可向大窗控制模块发送追焦对象丢失的指示。响应于上述指示，S811.大窗控制模块可控制大窗显示指示追焦对象丢失的提示信息。同样的，S808与S810无时序上的先后关系。
在已失焦的场景下，当追焦模块重新在一帧图像中定位到追焦对象时，追焦模块可确定该帧图像的追焦状态为成功，并将上述追焦状态传回功能模块。S812.当重新接收到成功的追焦状态时，功能模块向小窗控制模块发送追焦对象找回的指示。响应于上述指示，S813.小窗控制模块消除小窗蒙层。
同时，S814.当重新接收到成功的追焦状态时，功能模块还会向大窗控制模块发送追焦对象找回的指示。响应于上述指示，S815.大窗控制模块在大窗中显示指示追焦对象找回的提示信息。
可以理解的，在图9所示的流程中，在失焦的状态下，功能模块可控制暂停录制小窗图像。结合S808-S809的介绍，功能模块也会向小窗控制模块发送追焦对象丢失的指示，使得小窗相应地显示蒙层。在失焦后追回的状态下，功能模块可恢复录制小窗图像。结合S812-S813的介绍，功能模块也会向小窗控制模块发送追焦对象找回的指示，消除小窗上显示的蒙层。
图11示例性示出了图2H-图2I所示的结束录制的过程中的流程图。
在录制视频的过程中，用户可以选择先关闭小窗，结束录制特写视频，同时，用户还可以关闭大窗，同时结束录制原始视频和特写视频。
S901.结束录制模块检测到结束小窗录制的用户操作。参考图2B中介绍的停止控件212，终端100可检测到作用于上述控件的用户操作。上述用户操作即结束小窗录制的用户操作。
响应于上述结束小窗录制的用户操作，S902.结束录制模块可控制关闭小窗编码器、停止小窗编码器编码送显小窗的特写图像。小窗编码器可将已编码的特写图像封装为视频，即特写视频。
在检测到结束小窗录制的用户操作之后，S903.结束录制模块还可向小窗控制模块发送关闭小窗的指示。响应于上述指示，S904.小窗控制模块可关闭小窗。
S905.结束录制模块检测到结束录制的用户操作。参考图2H所示的用户界面，终端100还可检测到作用于控件2112的用户操作。上述用户操作即结束录制的用户操作。上述结束录制的操作对应同时结束大窗录制和小窗录制。
响应于上述结束录制的用户操作，S906.结束录制模块可控制关闭大窗编码器和小窗编码器、停止编码送显大窗和小窗的图像。小窗编码器可将已编码的特写图像封装为视频，即特写视频。同时，大窗编码器可将已编码的原始图像封装为视频，即原始视频。
在检测到结束录制的用户操作之后，S907.结束录制模块向大窗控制模块发送关闭大窗的指示。在接收到关闭大窗的指示之后，S908.大窗控制模块显示开始录制视频之前的预览界面。大窗控制模块可显示预览界面。
同时，S909.结束录制模块向小窗控制模块发送关闭小窗的指示。在接收到关闭小窗的指示之后，S910.小窗控制模块关闭小窗。这时，终端100可显示图1A所示的用户界面：预览界面、无小窗。同样的，S907与S909之间无时序上的先后关系。
在本申请实施例中：
对象检测模块从摄像头采集的图像中识别的对象可称为第一对象，例如图1B所示的人物1、人物2。摄像头采集的一帧图像中可包括一个或多个第一对象。从对象检测模块识别到的对象中选择的作为追焦对象(主角)的对象可称为第二对象，例如图1D所示的用户确定为主角的人物2。
在确定追焦对象后，用于展示追焦对象特写图像的窗口可称为第一窗口，例如图1D所示的展示人物2(追焦对象)的小窗141。录制预览窗113中的原始图像得到的原始视频即所述第一视频。录制小窗141中的特写图像得到的特写视频即所述第二视频。
参考图8，开始录制模块启用的大窗编码器可称为第一编码器，小窗编码器可称为第二编码器。
图12是本申请实施例提供的终端100的硬件结构示意图。
终端100可以包括处理器110，外部存储器接口120，内部存储器129，音频模块130， 扬声器130A，受话器130B，麦克风130C，耳机接口130D，显示屏140，摄像头150，触摸传感器160。
本发明实施例示意的结构并不构成对终端100的具体限定。在本申请另一些实施例中，终端100可以包括比图示更多或更少的部件，或者组合某些部件，或者拆分某些部件，或者不同的部件布置。图示的部件可以以硬件，软件或软件和硬件的组合实现。本发明实施例示意的各模块间的接口连接关系，只是示意性说明，并不构成对终端100的结构限定。
处理器110可以包括一个或多个处理单元，例如：处理器110可以包括应用处理器(application processor，AP)，调制解调处理器，图形处理器(graphics processing unit，GPU)，图像信号处理器(image signal processor，ISP)，控制器，视频编解码器，数字信号处理器(digital signal processor，DSP)，基带处理器，和/或神经网络处理器(neural-network processing unit，NPU)等。处理器110中还可以设置存储器，用于存储指令和数据。
终端100通过GPU，显示屏140，以及应用处理器等实现显示功能。
GPU为图像处理的微处理器，连接显示屏140和应用处理器。GPU用于执行数学和几何计算，用于图形渲染。处理器110可包括一个或多个GPU，其执行程序指令以生成或改变显示信息。显示屏140用于显示图像，视频等。在一些实施例中，电子设备可以包括1个或N个显示屏140，N为大于1的正整数。
在本申请实施例中，终端100显示摄像头采集的原始图像、经过追焦确定的主角的特写图像，以及图1A-图1J、图2A-图2I、图3A-图3C所示的用户界面的能力，依赖于上述GPU，显示屏140，以及应用处理器提供的显示功能。
终端100可以通过ISP，摄像头150，视频编解码器，GPU，显示屏140以及应用处理器等实现拍摄功能。
摄像头150用于捕获静态图像或视频。ISP用于处理摄像头150反馈的数据。光线通过镜头被传递到摄像头感光元件上，光信号转换为电信号，摄像头感光元件将所述电信号传递给ISP处理，转化为肉眼可见的图像。终端100可以包括1个或N个摄像头150，N为大于1的正整数。视频编解码器用于对数字视频压缩或解压缩。终端100可以支持一种或多种视频编解码器。这样，终端100可以播放或录制多种编码格式的视频，例如：动态图像专家组(moving picture experts group，MPEG)1，MPEG2，MPEG3，MPEG4等。
在本申请实施例中，终端100实施本申请实施例提供的拍摄方法，首先依赖于ISP，摄像头150采集的图像，其次还依赖于视频编解码器，GPU提供的图像计算与处理能力。
内部存储器129可以包括一个或多个随机存取存储器(random access memory，RAM)和一个或多个非易失性存储器(non-volatile memory，NVM)。随机存取存储器可以由处理器110直接进行读写，可以用于存储操作系统或其他正在运行中的程序的可执行程序(例如机器指令)，还可以用于存储用户及应用程序的数据等。非易失性存储器也可以存储可执行程序和存储用户及应用程序的数据等，可以提前加载到随机存取存储器中，用于处理器110直接进行读写。
在本申请实施例中，实现本申请实施例所述的拍摄方法的代码可存储在非易失性存储器上。在运行相机应用时，终端100可将非易失性存储器中存储的可执行代码加载到随机存取存储器。
外部存储器接口120可以用于连接外部的非易失性存储器，实现扩展终端100的存储能力。
终端100可以通过音频模块130，扬声器130A，受话器130B，麦克风130C，耳机接口130D，以及应用处理器等实现音频功能。
音频模块130用于将数字音频信息转换成模拟音频信号输出，也用于将模拟音频输入转换为数字音频信号。扬声器130A，也称“喇叭”，用于将音频电信号转换为声音信号。受话器130B，也称“听筒”，用于将音频电信号转换成声音信号。麦克风130C，也称“话筒”，“传声器”，用于将声音信号转换为电信号。耳机接口130D用于连接有线耳机。
在本申请实施例中，终端100在启用摄像头采集图像的过程中，可以同时启用麦克风130C采集声音信号，并将声音信号转换为电信号存储下来。这样，用户可以得到有声视频。
触摸传感器160，也称“触控器件”。触摸传感器160可以设置于显示屏140，由触摸传感器160与显示屏140组成触摸屏，也称“触控屏”。触摸传感器160用于检测作用于其上或附近的触摸操作。触摸传感器可以将检测到的触摸操作传递给应用处理器，以确定触摸事件类型。可以通过显示屏140提供与触摸操作相关的视觉输出。在另一些实施例中，触摸传感器160也可以设置于终端100的表面，与显示屏140所处的位置不同。
在本申请实施例中，终端100可利用触摸传感器160检测用户作用于显示屏140上的点击、滑动等操作，以实现图1A-图1J、图2A-图2I、图3A-图3C所示的拍摄方法。
本申请的说明书和权利要求书及附图中的术语“用户界面(user interface，UI)”，是应用程序或操作系统与用户之间进行交互和信息交换的介质接口，它实现信息的内部形式与用户可以接受形式之间的转换。应用程序的用户界面是通过java、可扩展标记语言(extensible markup language，XML)等特定计算机语言编写的源代码，界面源代码在终端设备上经过解析，渲染，最终呈现为用户可以识别的内容，比如图片、文字、按钮等控件。控件(control)也称为部件(widget)，是用户界面的基本元素，典型的控件有工具栏(toolbar)、菜单栏(menu bar)、文本框(text box)、按钮(button)、滚动条(scrollbar)、图片和文本。界面中的控件的属性和内容是通过标签或者节点来定义的，比如XML通过<Textview>、<ImgView>、<VideoView>等节点来规定界面所包含的控件。一个节点对应界面中一个控件或属性，节点经过解析和渲染之后呈现为用户可视的内容。此外，很多应用程序，比如混合应用(hybrid application)的界面中通常还包含有网页。网页，也称为页面，可以理解为内嵌在应用程序界面中的一个特殊的控件，网页是通过特定计算机语言编写的源代码，例如超文本标记语言(hyper text markup language，GTML)，层叠样式表(cascading style sheets，CSS)，java脚本(JavaScript，JS)等，网页源代码可以由浏览器或与浏览器功能类似的网页显示组件加载和显示为用户可识别的内容。网页所包含的具体内容也是通过网页源代码中的标签或者节点来定义的，比如GTML通过<p>、<img>、<video>、<canvas>来定义网页的元素和属性。
用户界面常用的表现形式是图形用户界面(graphic user interface，GUI)，是指采用图形方式显示的与计算机操作相关的用户界面。它可以是在电子设备的显示屏中显示的一个图标、窗口、控件等界面元素，其中控件可以包括图标、按钮、菜单、选项卡、文本框、对话框、状态栏、导航栏、Widget等可视的界面元素。
在本申请的说明书和所附权利要求书中所使用的那样，单数表达形式“一个”、“一种”、“所述”、“上述”、“该”和“这一”旨在也包括复数表达形式，除非其上下文中明确地有相反指示。还应当理解，本申请中使用的术语“和/或”是指并包含一个或多个所列出项目的任何或所有可能组合。上述实施例中所用，根据上下文，术语“当…时”可以被解释为意思是“如果…”或“在…后”或“响应于确定…”或“响应于检测到…”。类似地，根据上下文，短语“在确定…时”或“如果检测到(所陈述的条件或事件)”可以被解释为意思是“如果确定…”或“响应于确定…”或“在检测到(所陈述的条件或事件)时”或“响应于检测到(所陈述的条件或事件)”。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时，全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线)或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如DVD)、或者半导体介质(例如固态硬盘)等。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，该流程可以由计算机程序来指令相关的硬件完成，该程序可存储于计算机可读取存储介质中，该程序在执行时，可包括如上述各方法实施例的流程。而前述的存储介质包括：ROM或随机存储记忆体RAM、磁碟或者光盘等各种可存储程序代码的介质。

Claims (20)

  1. 一种拍摄方法，应用于具有摄像头和屏幕的电子设备，其特征在于，所述方法包括：
    在所述屏幕中显示预览窗，所述预览窗中显示所述摄像头采集的图像，所述预览窗中显示的图像包括一个或多个第一对象；
    在所述一个或多个第一对象中选择第二对象；
    在所述屏幕中显示所述预览窗的同时，还显示第一窗口，所述第一窗口中显示的图像包括所述第二对象；
    录制所述预览窗中的图像并保存为第一视频，录制所述第一窗口中的图像并保存为第二视频。
  2. 根据权利要求1所述的方法，其特征在于，所述在所述一个或多个第一对象中选择第二对象，包括：在所述一个或多个第一对象上显示选择框；响应于作用于第一选择框上的用户操作，将所述第一选择框对应的第一对象确定为所述第二对象。
  3. 根据权利要求1或2所述的方法，其特征在于，所述屏幕中还显示有录制控件，在所述屏幕中显示所述第一窗口之后，所述方法还包括：接收到用户对应于所述录制控件的录制操作，同时开始录制所述第一视频和所述第二视频；所述第一视频和所述第二视频的开始录制时刻相同。
  4. 根据权利要求1或2所述的方法，其特征在于，所述屏幕中还显示有录制控件，所述在所述一个或多个第一对象中选择第二对象之前，所述方法还包括：接收到用户对应于所述录制控件的录制操作，开始录制所述第一视频；在显示第一窗口的同时，开始录制所述第二视频；所述第二视频的开始录制时刻晚于所述第一视频的开始录制时刻。
  5. 根据权利要求1-4中任一项所述的方法，其特征在于，所述第一视频和所述第二视频的结束录制时刻是相同的；或者，所述第二视频的结束录制时刻早于所述第一视频的结束录制时刻。
  6. 根据权利要求1-5中任一项所述的方法，其特征在于，在所述录制过程中，所述方法还包括：从所述一个或多个第一对象中变更所述第二对象；第一窗口中显示的图像包括变更后的所述第二对象。
  7. 根据权利要求1-6中任一项所述的方法，其特征在于，在所述录制过程中，所述方法还包括：当监测到所述摄像头采集的图像不包括所述第二对象时，且持续时间长于第二时长后，暂停录制所述第一窗口中的图像。
  8. 根据权利要求7所述的方法，其特征在于，所述方法还包括：当监测到所述摄像头采集的图像包括所述第二对象时，恢复录制所述第一窗口中的图像。
  9. 根据权利要求1-8中任一项所述的方法，其特征在于，所述第一窗口与所述预览窗分屏显示，或者，所述第一窗口显示在所述预览窗之上，但不完全覆盖所述预览窗。
  10. 根据权利要求1-9中任一项所述的方法，其特征在于，所述第一窗口中显示的包括所述第二对象的图像是基于所述预览窗中显示的图像裁剪得到的。
  11. 根据权利要求2所述的方法，其特征在于，所述电子设备包括功能模块、对象检测模块和大窗控制模块，所述在所述一个或多个第一对象上显示选择框，包括：
    所述对象检测模块从所述摄像头采集的图像中识别到所述一个或多个第一对象；
    所述对象检测模块将所述一个或多个第一对象的特征信息发送给所述功能模块，所述特征信息包括指示第一对象位置的中心点；一个第一对象对应一组特征信息；
    所述功能模块依据所述特征信息中的所述中心点确定显示坐标；
    所述大窗控制模块依据所述显示坐标在所述屏幕上显示与所述一个或多个第一对象对应的选择框。
  12. 根据权利要求1-10中任一项所述的方法，其特征在于，所述电子设备包括功能模块、追焦模块、小窗控制模块，所述显示第一窗口，包括：
    所述功能模块向所述追焦模块发送所述第二对象的特征信息；
    所述追焦模块依据所述第二对象的特征信息确定将要送显的图像中是否包括所述第二对象；当包括所述第二对象时，所述追焦模块向所述功能模块发送追焦成功指示，
    所述功能模块依据所述指示控制所述小窗控制模块在所述屏幕中显示所述第一窗口。
  13. 根据权利要求12所述的方法，其特征在于，所述方法还包括：当包括所述第二对象时，所述追焦模块从所述将要送显的图像中裁剪出所述第一窗口中显示的图像。
  14. 根据权利要求3所述的方法，其特征在于，所述电子设备包括开始录制模块，所述接收到用户对应于所述录制控件的录制操作，同时开始录制所述第一视频和所述第二视频，包括：
    所述开始录制模块接收到用户对应于所述录制控件的录制操作；
    响应于所述录制操作，所述开始录制模块开启第一编码器和第二编码器，所述第一编码器开始编码所述预览窗中的图像，所述第二编码器开始编码所述第一窗口中的图像。
  15. 根据权利要求7所述的方法，其特征在于，所述电子设备包括追焦模块、功能模块、暂停录制模块，所述当监测到所述摄像头采集的图像不包括所述第二对象时，且持续时间长于第二时长后，暂停录制所述第一窗口中的图像，包括：
    当识别到将要送显的图像中不包括所述第二对象时，所述追焦模块向所述功能模块发送失焦指示，所述功能模块依据所述失焦指示设置第二计时器，所述第二计时器的计时时长为所述第二时长；
    在所述第二计时器计时结束，且所述功能模块仍未接收到追焦成功指示时，所述暂停录制模块控制所述第二编码器暂停编码所述第一窗口中显示的图像。
  16. 根据权利要求8所述的方法，其特征在于，所述电子设备包括追焦模块、功能模块、开始录制模块，所述当监测到所述摄像头采集的图像包括所述第二对象时，恢复录制所述第一窗口中的图像，包括：
    当识别到将要送显的图像中包括所述第二对象时，所述追焦模块向所述功能模块发送追焦成功指示，
    所述功能模块依据所述追焦成功指示，向所述开始录制模块发送恢复录制指示；
    所述开始录制模块依据所述恢复录制指示重新开启所述第二编码器，继续编码所述第一窗口中显示的图像。
  17. 根据权利要求2所述的方法，其特征在于，在所述一个或多个第一对象上显示选择框之后，所述方法还包括：在第一时长内没有检测到作用于所述屏幕的用户操作，所述选择框消失。
  18. 根据权利要求17所述的方法，其特征在于，所述电子设备包括功能模块、大窗控制模块，所述在第一时长内没有检测到作用于所述屏幕的用户操作，所述选择框消失，包括：
    在显示所述选择框之后，所述功能模块设置第一计时器，所述第一计时器的计时时长为所述第一时长；在所述第一计时器计时结束后，且所述功能模块未检测到作用于所述屏幕的用户操作，所述大窗控制模块消除所述选择框。
  19. 一种电子设备，其特征在于，包括一个或多个处理器和一个或多个存储器；其中，所述一个或多个存储器与所述一个或多个处理器耦合，所述一个或多个存储器用于存储计算机程序代码，所述计算机程序代码包括计算机指令，当所述一个或多个处理器执行所述计算机指令时，使得执行如权利要求1-18任一项所述的方法。
  20. 一种计算机可读存储介质，包括指令，其特征在于，当所述指令在电子设备上运行时，使得执行如权利要求1-18任一项所述的方法。
PCT/CN2023/088623 2022-05-30 2023-04-17 一种拍摄方法和电子设备 WO2023231616A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP23814802.7A EP4391570A1 (en) 2022-05-30 2023-04-17 Photographing method and electronic device

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN202210603605.4 2022-05-30
CN202210603605 2022-05-30
CN202210908497.1 2022-07-29
CN202210908497.1A CN117177063B (zh) 2022-05-30 2022-07-29 一种拍摄方法和电子设备

Publications (2)

Publication Number Publication Date
WO2023231616A1 WO2023231616A1 (zh) 2023-12-07
WO2023231616A9 true WO2023231616A9 (zh) 2024-02-01

Family

ID=88943690

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/088623 WO2023231616A1 (zh) 2022-05-30 2023-04-17 一种拍摄方法和电子设备

Country Status (3)

Country Link
EP (1) EP4391570A1 (zh)
CN (1) CN117177063B (zh)
WO (1) WO2023231616A1 (zh)

Family Cites Families (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1237374A1 (en) * 1997-05-05 2002-09-04 The Trustees of Columbia University in the City of New York A method for extracting video objects from a video clip
JP2007150496A (ja) * 2005-11-25 2007-06-14 Sony Corp 撮像装置、およびデータ記録制御方法、並びにコンピュータ・プログラム
US10645344B2 (en) * 2010-09-10 2020-05-05 Avigilion Analytics Corporation Video system with intelligent visual display
EP3545686B1 (en) * 2017-03-20 2021-09-22 Samsung Electronics Co., Ltd. Methods and apparatus for generating video content
CN112954218A (zh) * 2019-03-18 2021-06-11 荣耀终端有限公司 一种多路录像方法及设备
CN110557566B (zh) * 2019-08-30 2021-12-17 维沃移动通信有限公司 视频拍摄方法及电子设备
CN110602408A (zh) * 2019-09-29 2019-12-20 联想(北京)有限公司 电子设备以及由其执行的拍照方法
CN113365013A (zh) * 2020-03-06 2021-09-07 华为技术有限公司 一种音频处理方法及设备
CN113536866A (zh) * 2020-04-22 2021-10-22 华为技术有限公司 一种人物追踪显示方法和电子设备
CN112135046B (zh) * 2020-09-23 2022-06-28 维沃移动通信有限公司 视频拍摄方法、视频拍摄装置及电子设备
CN115379112A (zh) * 2020-09-29 2022-11-22 华为技术有限公司 一种图像处理方法及相关装置
CN112437232A (zh) * 2020-11-24 2021-03-02 维沃移动通信（杭州）有限公司 拍摄方法、装置、电子设备及可读存储介质
CN112770050B (zh) * 2020-12-31 2023-02-03 Oppo广东移动通信有限公司 视频展示方法及装置、计算机可读介质和电子设备
CN113873166A (zh) * 2021-10-26 2021-12-31 维沃移动通信有限公司 视频拍摄方法、装置、电子设备和可读存储介质
CN114125179B (zh) * 2021-12-07 2024-04-05 维沃移动通信有限公司 拍摄方法和装置
CN114422692B (zh) * 2022-01-12 2023-12-08 西安维沃软件技术有限公司 视频录制方法、装置及电子设备

Also Published As

Publication number Publication date
EP4391570A1 (en) 2024-06-26
CN117177063A (zh) 2023-12-05
CN117177063B (zh) 2024-10-15
WO2023231616A1 (zh) 2023-12-07

Similar Documents

Publication Publication Date Title
US20220366219A1 (en) Systems and Methods for Providing Feedback for Artificial Intelligence-Based Image Capture Devices
CN113099146B (zh) 一种视频生成方法、装置及相关设备
WO2023134583A1 (zh) 视频录制方法、装置及电子设备
CN113794923B (zh) 视频处理方法、装置、电子设备及可读存储介质
WO2023160170A1 (zh) 拍摄方法和电子设备
WO2023160241A1 (zh) 一种视频处理方法及相关装置
WO2024055797A9 (zh) 一种录像中抓拍图像的方法及电子设备
CN115484403B (zh) 录像方法和相关装置
WO2024179101A1 (zh) 一种拍摄方法
WO2024179100A1 (zh) 一种拍摄方法
WO2024109224A1 (zh) 一种拍摄模式推荐方法
CN117609547A (zh) 视频缩略图的显示方法、设备和存储介质
WO2023231616A9 (zh) 一种拍摄方法和电子设备
CN115883958A (zh) 一种人像拍摄方法
CN116055861B (zh) 一种视频编辑方法和电子设备
WO2023231696A1 (zh) 一种拍摄方法及相关设备
CN116055867B (zh) 一种拍摄方法和电子设备
CN117221743A (zh) 一种拍摄方法和电子设备
WO2023226699A1 (zh) 录像方法、装置及存储介质
CN117459825A (zh) 一种拍摄方法和电子设备
CN117202081A (zh) 音频处理方法及电子设备
CN117201713A (zh) 音频处理方法及电子设备
CN117201714A (zh) 音频处理方法及电子设备
CN118555467A (zh) 一种拍照方法及电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23814802

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2023814802

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2023814802

Country of ref document: EP

Effective date: 20240319

WWE Wipo information: entry into national phase

Ref document number: 18695434

Country of ref document: US