WO2022105734A1 - 一种慢动作录像方法及设备 - Google Patents

一种慢动作录像方法及设备 Download PDF

Info

Publication number
WO2022105734A1
WO2022105734A1 PCT/CN2021/130849 CN2021130849W WO2022105734A1 WO 2022105734 A1 WO2022105734 A1 WO 2022105734A1 CN 2021130849 W CN2021130849 W CN 2021130849W WO 2022105734 A1 WO2022105734 A1 WO 2022105734A1
Authority
WO
WIPO (PCT)
Prior art keywords
frame
image
slow
image frame
motion
Prior art date
Application number
PCT/CN2021/130849
Other languages
English (en)
French (fr)
Inventor
肖斌
朱聪超
侯良军
胡斌
李远友
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to US18/253,388 priority Critical patent/US20240007588A1/en
Priority to EP21893881.9A priority patent/EP4236300A4/en
Publication of WO2022105734A1 publication Critical patent/WO2022105734A1/zh

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/76Television signal recording
    • H04N5/78Television signal recording using magnetic recording
    • H04N5/782Television signal recording using magnetic recording on tape
    • H04N5/783Adaptations for reproducing at a rate different from the recording rate
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/76Television signal recording
    • GPHYSICS
    • G11INFORMATION STORAGE
    • G11BINFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/005Reproducing at a different information rate from the information rate of recording
    • GPHYSICS
    • G11INFORMATION STORAGE
    • G11BINFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/02Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
    • G11B27/031Electronic editing of digitised analogue information signals, e.g. audio or video signals
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60Control of cameras or camera modules
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60Control of cameras or camera modules
    • H04N23/63Control of cameras or camera modules by using electronic viewfinders
    • H04N23/631Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/76Television signal recording
    • H04N5/765Interface circuits between an apparatus for recording and another apparatus
    • H04N5/77Interface circuits between an apparatus for recording and another apparatus between a recording apparatus and a television camera
    • H04N5/772Interface circuits between an apparatus for recording and another apparatus between a recording apparatus and a television camera the recording apparatus and the television camera being placed in the same enclosure
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/76Television signal recording
    • H04N5/91Television signal processing therefor

Definitions

  • the present application relates to the field of electronic technology, and in particular, to a slow-motion video recording method and device.
  • slow-motion video can capture slow-motion details of objects in high-speed motion, and record fleeting wonderful moments.
  • the electronic device determines the start/end time of the motion process, and generates a slow-motion video segment according to the start/end time.
  • the slow-motion video segment can facilitate the user to watch slow-motion wonderful moments at a slow speed.
  • the start/end time of the slow motion determined by the electronic device may be inaccurate, and the start position and end position of the generated slow motion video segment usually deviate from the user's expectation, which is difficult to meet the user's personalization of the wonderful slow motion moments. need.
  • the embodiments of the present application provide a slow-motion video recording method and device, which can accurately set the start position and end position of the slow-motion video segment by the user during the recording process, so that the generated slow-motion video segment can accurately correspond to the slow-motion wonderful moments , and can also meet the personalized needs of users for perfect slow-motion moments.
  • an embodiment of the present application provides a video recording method, which is applied to an electronic device.
  • the method includes: the electronic device enables a video recording function. After detecting the user's video recording operation, the electronic device will display the image frame to be selected on the frame selection interface if a slow motion trigger is detected, and the image frame to be selected includes multiple frames of images.
  • the electronic device acquires the start frame and the end frame set by the user based on the image frame to be selected.
  • the electronic device generates a target video, the target video includes a slow-motion video segment, the slow-motion video segment corresponds to the start frame and the end frame, and the recording frame rate of the slow-motion video segment is higher than the encoding frame rate of the slow-motion video segment.
  • the user can accurately select the start frame and the end frame based on the image frames to be selected displayed by the electronic device during the recording process, so that the electronic device can generate the slow motion video in the target video according to the start frame and the end frame. segment, so that the generated slow-motion video segment can accurately correspond to the wonderful slow-motion moments, and can also meet the user's personalized needs for the wonderful slow-motion moments.
  • the recording frame rate of the slow-motion video segment is higher than the encoding frame rate of the slow-motion video segment, so that the user can watch the slow-motion wonderful process through the slow-motion video segment slowly, carefully and for a long time.
  • the method further includes: after the electronic device detects the user's video recording operation, collecting image frames at a first frame rate, and buffering a first set of image frames, where the first set of image frames includes the most recently collected image frames The image frame within the T0 duration. After detecting the slow motion trigger, the electronic device collects image frames of duration T1 at the first frame rate, and buffers a second set of image frames, where the second set of image frames includes image frames of duration T1 collected at the first frame rate.
  • the image frames to be selected come from the first image frame set and the second image frame set.
  • the electronic device can cache the image frames collected at the first frame rate during T0 before the slow motion is triggered, and the image frames collected at the first frame rate during T1 after the slow motion is triggered.
  • the image frame of the frame rate displays the candidate image frame, so that the user can select the start frame and the end frame based on the candidate image frame.
  • the image frame to be selected is obtained by extracting image frames from the first image frame set and the second image frame set.
  • the number of the image frames to be selected is small, and it is convenient for the user to select the start frame and the end frame therefrom.
  • the number of image frames to be selected is positively correlated with the slow-motion double-speed of the slow-motion video segment;
  • the frame sampling interval of the image frame to be selected is positively related to the slow-motion multiplier of the slow-motion video segment.
  • the faster-motion multiplier the greater the number of image frames to be selected; the smaller the slow-motion multiplier, the less the number of frames to be selected.
  • the larger the slow-motion multiplier the larger the frame sampling interval used to extract frames to obtain the candidate image frame; the smaller the slow-motion multiplier, the smaller the frame sampling interval used to extract the frame to obtain the candidate image frame.
  • the slow motion video segment is obtained from the target image frame between the start frame and the end frame by using the encoding frame rate.
  • the target image frame is the image frame between the start frame and the end frame in the first image frame set and the second image frame set, and the video recording frame rate is equal to the first frame rate.
  • the target image frame is an image frame at the first frame rate between the start frame and the end frame in the first image frame set and the second image frame set, and an image frame at the video recording frame rate obtained after frame insertion.
  • the target image frame is obtained according to the start frame and the end frame, and the target image frame is an image frame with a high frame rate.
  • the slow-motion video segment adopts a low encoding frame rate, and the target image frame with a high frame rate is obtained by video encoding, so that the user can watch the slow-motion wonderful process through the slow-motion video segment slowly, carefully and for a long time.
  • the method further includes: after the electronic device detects the user's video recording operation, displaying the recorded image at the second frame rate on the shooting interface, and the recorded image is based on the image at the first frame rate Obtained after frame extraction, the second frame rate is smaller than the first frame rate.
  • the electronic device captures image frames at a high frame rate and displays the recorded image at a low frame rate, which can reduce the load of image processing and image display and save the power consumption of the electronic device.
  • the recording frame rate corresponds to the slow-motion double speed of the slow-motion video segment.
  • the electronic device can determine the size of the recording frame rate according to the slow-motion double speed.
  • the first frame rate is related to the slow-motion double-speed of the slow-motion video segment.
  • the frame rate used by the electronic device to capture the image frames can be determined according to the slow-motion multiplier.
  • the method further includes: if the electronic device detects the slow motion trigger, prompting the user to set the start frame and the end frame on the frame selection interface.
  • the user can select the start frame and the end frame according to the intuitive prompt of the electronic device.
  • the electronic device determines the start frame and the end frame set by the user based on the image frame to be selected, including: if the electronic device detects the user's first preset for the first image frame on the image frame to be selected Set operation, then determine the first image frame as the start frame. If the electronic device detects the user's second preset operation on the second image frame on the to-be-selected image frame, it determines that the second image frame is the end frame.
  • the electronic device may determine the start frame and the end frame set by the user based on the user's preset operation.
  • an image frame located at a preset position among the image frames to be selected is displayed in a large image on the frame selection interface, and the preset position is the middle position of the image frame to be selected or the boundary position of the frame selection interface.
  • the method further includes: if the electronic device detects a slow motion trigger, displaying the first control and the second control on the frame selection interface.
  • the first control and the second control can slide on the image frame to be selected, the first control is used to set the start frame, and the second control is used to set the end frame.
  • the image frame where the first control or the second control is located is displayed in a large image on the frame selection interface.
  • the interface displays the image frame where the first control is located in the form of a large image.
  • the interface displays the image frame where the second control is located in the form of a large image.
  • the electronic device determining the start frame and the end frame set by the user based on the to-be-selected image frame includes: the electronic device determines that in the to-be-selected image frame, the image frame where the first control is located is the start frame. The electronic device determines that among the image frames to be selected, the image frame where the second control is located is the end frame.
  • the user can set the start frame and the end frame respectively through the first control and the second control, and the electronic device can display the image frame where the first control or the second control currently targeted by the user is located in the form of a large image .
  • the image frame to be selected is obtained by extracting frames from the image frames of the first image frame set and the second image frame set at a first interval.
  • the electronic device determining the start frame and the end frame set by the user based on the image frame to be selected includes: the electronic device determining the reference start frame set by the user based on the image frame to be selected.
  • the electronic device displays a third image frame set on the frame selection interface, the third image frame set includes a reference start frame, and the third image frame set adopts a second interval from the image frames of the first image frame set and the second image frame set The middle-drawn frame is obtained, and the second interval is smaller than the first interval.
  • the electronic device determines the start frame set by the user based on the third image frame set, and determines the reference end frame set by the user based on the candidate image frame.
  • the electronic device displays a fourth image frame set on the interface, the fourth image frame set includes the reference end frame, and the fourth image frame set uses a second interval to extract frames from the image frames of the first image frame set and the second image frame set get.
  • the electronic device determines the end frame set by the user based on the fourth set of image frames.
  • the electronic device can first display the image frames to be selected at large intervals, so that the user can select the reference start frame and the reference end frame in a coarse-grained manner; then, the electronic device displays the reference start frame and the reference end frame at small intervals Nearby image frames, allowing users to precisely select start and end frames.
  • the image frames to be selected include a fifth image frame set and a sixth image frame set
  • the fifth image frame set includes a recommended start frame and adjacent multi-frame images
  • the sixth image frame set Including the recommended end frame and adjacent multi-frame images.
  • the electronic device determining the start frame and the end frame set by the user based on the image frame to be selected includes: the electronic device determining the start frame set by the user based on the fifth image frame set and the recommended start frame.
  • the electronic device determines the end frame set by the user based on the sixth image frame set and the recommended end frame.
  • the electronic device can automatically determine the more accurate recommended start frame and recommended end frame, and display them to the user; so that the user can accurately select the start frame near the recommended start frame and accurately near the recommended end frame to select the end frame.
  • the image frame to be selected includes a recommended start frame interval and a recommended end frame interval.
  • the electronic device determining the start frame and end frame set by the user based on the candidate image frame includes: the electronic device determining the start frame set by the user based on the recommended start frame interval; the electronic device determining the end frame set by the user based on the recommended end frame interval.
  • the electronic device can automatically determine the more accurate recommended start frame interval and recommended end frame interval, and display them to the user; so that the user can accurately select the start frame based on the recommended start frame interval and the recommended end frame interval and end frame.
  • the method further includes: after the electronic device detects the user's video recording operation, buffering a seventh image frame set, where the seventh image frame set includes image frames at the first frame rate before the duration T0 The image frame of the second frame rate obtained by the frame extraction.
  • the target video further includes a first video segment, and the first video segment is obtained by encoding the image frames in the seventh image frame set and the eighth image frame set using the encoding frame rate.
  • the eighth image frame set is an image frame of the first frame rate before the start frame in the first image frame set and the second image frame set, and the image frame of the second frame rate obtained by frame extraction.
  • the first video segment is a normal-speed video segment. In this way, when the target video is played, the first video segment at a normal speed can be played first, and then the slow-motion video segment is played, giving the user a visual impact of switching between speed and slow.
  • the target video further includes a second video segment, and the second video segment is obtained by encoding the image frames in the ninth image frame set at an encoding frame rate.
  • the ninth image frame set is an image frame of the first frame rate after the end frame among the first image frame set and the second image frame set, and the image frame of the second frame rate obtained by the frame extraction.
  • the first video segment is a normal-speed video segment.
  • the slow-motion video segment can be played first, and then the second video segment of normal speed can be played, giving the user a visual impact of switching between slow and fast.
  • the method further includes: after the electronic device enables the video recording function, capturing images at a third frame rate, and displaying the preview image at the third frame rate on the preview interface, where the third frame rate is less than first frame rate.
  • the electronic device can capture image frames at a low frame rate and display the preview image.
  • an embodiment of the present application provides a photographing apparatus, and the apparatus is included in an electronic device.
  • the apparatus has the function of implementing the behavior of the electronic device in any of the above aspects and possible designs, so that the electronic device executes the video recording method performed by the electronic device in any of the possible designs in the above-mentioned aspects.
  • This function can be implemented by hardware or by executing corresponding software by hardware.
  • the hardware or software includes at least one module or unit corresponding to the above-mentioned functions.
  • the apparatus may include an opening unit, a detection unit, a display unit, an acquisition unit, a generation unit, and the like.
  • an embodiment of the present application provides an electronic device, including: a camera for capturing images; a display screen for displaying an interface; one or more processors; a memory; and one or more computer programs, one or more One or more computer programs are stored in the memory, and the one or more computer programs include instructions that, when executed by the electronic device, cause the electronic device to perform the video recording method performed by the electronic device in any of the possible designs of the above aspects.
  • an embodiment of the present application provides an electronic device, including: one or more processors; and a memory, where codes are stored in the memory.
  • the electronic device is caused to execute the video recording method executed by the electronic device in any possible design of the above aspect.
  • an embodiment of the present application provides a computer-readable storage medium, including computer instructions, which, when the computer instructions are executed on an electronic device, cause the electronic device to execute the video recording method in any of the possible designs in the foregoing aspect.
  • an embodiment of the present application provides a computer program product, which, when the computer program product runs on a computer, enables the computer to execute the video recording method performed by the electronic device in any of the possible designs in the above aspect.
  • an embodiment of the present application provides a chip system, and the chip system is applied to an electronic device.
  • the chip system includes one or more interface circuits and one or more processors; the interface circuit and the processor are interconnected by lines; the interface circuit is used for receiving signals from the memory of the electronic device and sending signals to the processor, and the signals are included in the memory Stored computer instructions; when the processor executes the computer instructions, it causes the electronic device to execute the video recording method in any possible design of the above aspect.
  • FIG. 1 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
  • FIG. 2 is a schematic flowchart of a slow-motion video recording method provided by an embodiment of the present application
  • 3A is a schematic diagram of a group of interfaces provided by an embodiment of the present application.
  • 3B is another set of interface schematic diagrams provided by an embodiment of the present application.
  • FIG. 4 is a schematic diagram of an interface provided by an embodiment of the present application.
  • FIG. 5 is a schematic diagram of an image processing process provided by an embodiment of the present application.
  • FIG. 6 is another interface schematic diagram provided by an embodiment of the present application.
  • FIG. 7 is a schematic diagram of another image processing process provided by an embodiment of the present application.
  • FIG. 8 is a schematic diagram of a caching solution provided by an embodiment of the present application.
  • FIG. 9 is a schematic diagram of a process for generating a target video according to an embodiment of the present application.
  • FIG. 10 is another set of interface schematic diagrams provided by the embodiment of the present application.
  • 11A is another set of interface schematic diagrams provided by an embodiment of the present application.
  • FIG. 11B is another set of interface schematic diagrams provided by an embodiment of the present application.
  • FIG. 12 is another set of interface schematic diagrams provided by an embodiment of the present application.
  • FIG. 13 is another set of interface schematic diagrams provided by an embodiment of the present application.
  • FIG. 14 is another set of interface schematic diagrams provided by an embodiment of the present application.
  • FIG. 15 is another interface schematic diagram provided by an embodiment of the present application.
  • FIG. 17 is another interface schematic diagram provided by an embodiment of the present application.
  • FIG. 18 is a schematic structural diagram of a target video provided by an embodiment of the application.
  • 19A is a sequence diagram of a slow-motion video recording method provided by an embodiment of the present application.
  • 19B is a schematic diagram of the effect of a video frame played by a target video according to an embodiment of the present application.
  • FIG. 20 is a schematic structural diagram of another electronic device provided by an embodiment of the present application.
  • first and second are only used for descriptive purposes, and should not be construed as indicating or implying relative importance or implicitly indicating the number of indicated technical features.
  • a feature defined as “first” or “second” may expressly or implicitly include one or more of that feature.
  • plural means two or more.
  • words such as “exemplarily” or “for example” are used to represent examples, illustrations or illustrations. Any embodiment or design described in the embodiments of the present application as “exemplarily” or “such as” should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as “exemplarily” or “such as” is intended to present the related concepts in a specific manner.
  • Slow-motion video can capture the high-speed motion of objects, capture details of motion that are difficult to see with the naked eye, and record fleeting moments, which can also be called high-speed video.
  • slow-motion video can record the movement of a bullet when it leaves the chamber, the action process when a football is shot, and the moment when a drop of water splashes and ripples.
  • the electronic device can automatically determine the start/stop time of the motion process through algorithms such as human motion detection algorithm or optical flow motion detection algorithm, and automatically generate a slow-motion video segment according to the start/end time.
  • the start/stop time of the motion process automatically judged by electronic equipment is often inaccurate.
  • the user wants to capture the moment when the child pops out of the chamber, but the change of the picture when the child pops out of the chamber is not large. Accurately grab the starting time of the sub-popping chamber.
  • the start position and end position of the slow-motion video segment automatically generated by the electronic device according to the start/end time also usually deviate from the user's expectation, and it is difficult to meet the user's personalized demand for the wonderful slow-motion moments.
  • the embodiment of the present application provides a slow-motion video recording method, which can be applied to electronic equipment, and can be accurately set by a user during the recording process to start and end positions of a slow-motion video segment, so that the generated slow-motion video segment can be It accurately corresponds to the slow-motion wonderful moments captured, and can also meet the user's personalized needs for the slow-motion wonderful moments.
  • the slow-motion video segment has a high recording frame rate, and a low encoding frame rate and playback frame rate, which can facilitate the user to slowly and carefully watch and review the slow-motion wonderful moments of the object.
  • the electronic device may be a mobile phone, a tablet, a wearable device (eg, a smart watch), an in-vehicle device, an augmented reality (AR)/virtual reality (VR) device, a laptop, a super mobile personal computer
  • a wearable device eg, a smart watch
  • AR augmented reality
  • VR virtual reality
  • UMPC ultra-mobile personal computer
  • PDA personal digital assistant
  • FIG. 1 shows a schematic structural diagram of an electronic device 100 .
  • the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charge management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2 , mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, buttons 190, motor 191, indicator 192, camera 193, display screen 194, and Subscriber identification module (subscriber identification module, SIM) card interface 195 and so on.
  • SIM Subscriber identification module
  • the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, and ambient light. Sensor 180L, bone conduction sensor 180M, etc.
  • the processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), controller, memory, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural-network processing unit (NPU) Wait. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.
  • application processor application processor, AP
  • modem processor graphics processor
  • graphics processor graphics processor
  • ISP image signal processor
  • controller memory
  • video codec digital signal processor
  • DSP digital signal processor
  • NPU neural-network processing unit
  • the controller may be the nerve center and command center of the electronic device 100 .
  • the controller can generate an operation control signal according to the instruction operation code and timing signal, and complete the control of fetching and executing instructions.
  • a memory may also be provided in the processor 110 for storing instructions and data.
  • the memory in processor 110 is cache memory. This memory may hold instructions or data that have just been used or recycled by the processor 110 . If the processor 110 needs to use the instruction or data again, it can be called directly from memory. Repeated accesses are avoided and the latency of the processor 110 is reduced, thereby increasing the efficiency of the system.
  • the electronic device 100 implements a display function through a GPU, a display screen 194, an application processor, and the like.
  • the GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor.
  • the GPU is used to perform mathematical and geometric calculations for graphics rendering.
  • Processor 110 may include one or more GPUs that execute program instructions to generate or alter display information.
  • Display screen 194 is used to display images, videos, and the like.
  • Display screen 194 includes a display panel.
  • the display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (active-matrix organic light).
  • LED diode AMOLED
  • flexible light-emitting diode flexible light-emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light-emitting diode (quantum dot light emitting diodes, QLED) and so on.
  • the electronic device 100 may include one or N display screens 194 , where N is a positive integer greater than one.
  • the display screen 194 may be used to display interface contents such as a preview interface and a shooting interface in a slow-motion video recording scene.
  • the display screen 194 is further configured to display the high frame rate image frames buffered during the slow motion video recording on the shooting interface, so that the user can select the start position and the end position of the slow motion video segment.
  • the electronic device 100 may implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, an application processor, and the like.
  • the ISP is used to process the data fed back by the camera 193 .
  • the shutter is opened, the light is transmitted to the camera photosensitive element through the lens, the light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing, converting it into an image visible to the naked eye.
  • ISP can also perform algorithm optimization on image noise, brightness, and skin tone.
  • ISP can also optimize the exposure, color temperature and other parameters of the shooting scene.
  • the ISP may be provided in the camera 193 .
  • Camera 193 is used to capture still images or video.
  • the object is projected through the lens to generate an optical image onto the photosensitive element.
  • the photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor.
  • CMOS complementary metal-oxide-semiconductor
  • the photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal.
  • the ISP outputs the digital image signal to the DSP for processing.
  • DSP converts digital image signals into standard RGB, YUV and other formats of image signals.
  • the electronic device 100 may include 1 or N cameras 193 , where N is a positive integer greater than 1.
  • the camera 193 may include one or more of the following cameras: a telephoto camera, a wide-angle camera, an ultra-wide-angle camera, a zoom camera, a depth camera, and the like.
  • the telephoto camera has a small shooting range and is suitable for shooting distant scenes;
  • the wide-angle camera has a larger shooting range;
  • the ultra-wide-angle camera has a larger shooting range than the wide-angle camera and is suitable for shooting panorama and other large-scale scenes.
  • the depth camera can be used to measure the object distance of the object to be shot, that is, to measure the depth information of the object to be shot, for example, it can include a three-dimensional (3 dimensions, 3D) depth camera, a time of flight (TOF) depth camera or a binocular depth camera etc.
  • the camera 193 may include a front camera and/or a rear camera.
  • the camera 193 may be used to capture images in a slow-motion video recording scene.
  • a digital signal processor is used to process digital signals, in addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy and so on.
  • Video codecs are used to compress or decompress digital video.
  • the electronic device 100 may support one or more video codecs.
  • the electronic device 100 can play or record videos of various encoding formats, such as: Moving Picture Experts Group (moving picture experts group, MPEG) 1, MPEG2, MPEG3, MPEG4 and so on.
  • MPEG Moving Picture Experts Group
  • MPEG2 moving picture experts group
  • MPEG3 MPEG4
  • MPEG4 Moving Picture Experts Group
  • the NPU is a neural-network (NN) computing processor.
  • NN neural-network
  • Applications such as intelligent cognition of the electronic device 100 can be implemented through the NPU, such as image recognition, face recognition, speech recognition, text understanding, and the like.
  • Internal memory 121 may be used to store computer executable program code, which includes instructions.
  • the processor 110 executes various functional applications and data processing of the electronic device 100 by executing the instructions stored in the internal memory 121.
  • the internal memory 121 may include a storage program area and a storage data area.
  • the storage program area can store an operating system, an application program required for at least one function (such as a sound playback function, an image playback function, etc.), and the like.
  • the storage data area may store data (such as audio data, phone book, etc.) created during the use of the electronic device 100 and the like.
  • the internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), double-rate synchronous dynamic random access memory Take memory (double data rate synchronous dynamic random-access memory, DDR SDRAM), etc.
  • non-volatile memory such as at least one disk storage device, flash memory device, universal flash storage (UFS), double-rate synchronous dynamic random access memory Take memory (double data rate synchronous dynamic random-access memory, DDR SDRAM), etc.
  • the internal memory 121 may be used for buffering high frame rate image frames within a preset duration during the slow motion recording process.
  • the processor 110 generates a slow-motion video segment according to the starting position and ending position accurately set by the user during the slow-motion recording process by running the instructions stored in the internal memory 121, so as to meet the user's personalized needs for the wonderful slow-motion moments. .
  • the electronic device 100 may implement audio functions through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, an application processor, and the like. Such as music playback, recording, etc.
  • Touch sensor 180K also called “touch panel”.
  • the touch sensor 180K may be disposed on the display screen 194 , and the touch sensor 180K and the display screen 194 form a touch screen, also called a “touch screen”.
  • the touch sensor 180K is used to detect a touch operation on or near it.
  • the touch sensor can pass the detected touch operation to the application processor to determine the type of touch event.
  • Visual output related to touch operations may be provided through display screen 194 .
  • the touch sensor 180K may also be disposed on the surface of the electronic device 100 , which is different from the location where the display screen 194 is located.
  • the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100 .
  • the electronic device 100 may include more or less components than shown, or combine some components, or separate some components, or arrange different components.
  • the illustrated components may be implemented in hardware, software, or a combination of software and hardware.
  • the camera 193 can be used to capture images in a slow-motion recording scene; the internal memory 121 can be used to cache high-frame rate image frames within a preset duration during the slow-motion recording process; the display screen 194 can use It is used to display the preview interface or shooting interface, etc. in the slow-motion recording scene, and it is also used to display the high-frame rate image frames cached during the slow-motion recording process on the shooting interface, so that the user can select the slow-motion image frame from the high-frame rate image frames.
  • the start position and end position of the video segment implements the user to accurately set the start position and end position of the slow motion video segment during the recording process by running the instructions stored in the internal memory 121, so as to satisfy the user's preference for the slow motion video segment. Personalized needs for action moments.
  • the slow-motion video recording method provided by the embodiment of the present application will be described below by taking the electronic device as a mobile phone having the structure shown in FIG. 1 as an example.
  • the mobile phone may firstly enable the video recording function. Then, after detecting the user's video recording operation, if a slow motion trigger is detected, the mobile phone displays an image frame to be selected on the frame selection interface, and the image frame to be selected includes multiple frames of images. Then, the mobile phone can obtain the start frame and the end frame set by the user based on the image frame to be selected. Then, the mobile phone can generate a target video, the target video includes a slow-motion video segment, the slow-motion video segment corresponds to the start frame and the end frame, and the recording frame rate of the slow-motion video segment is higher than the encoding frame rate of the slow-motion video segment.
  • the embodiment of the present application also provides another slow-motion video recording method, see FIG. 2 , the method includes:
  • the mobile phone starts the video recording function.
  • the recording function of the mobile phone when the user wants to record in slow motion, the recording function of the mobile phone can be activated.
  • the mobile phone may have multiple video recording functions.
  • the mobile phone can activate the slow-motion video recording function.
  • the phone can launch the camera app, or launch other apps with photo or video functions (such as AR apps such as Douyin or Hetu Cyberverse), and then activate the slow-motion video recording function of these apps.
  • apps such as AR apps such as Douyin or Hetu Cyberverse
  • the mobile phone detects that the user clicks the icon 301 of the camera application shown in (a) of FIG. 3A , the mobile phone starts the photographing function and displays the preview interface shown in (b) of FIG. 3A .
  • the mobile phone detects that the user clicks the control 302 shown in (b) in FIG. 3A , as shown in (c) in FIG. 3A , the slow-motion video recording function is activated.
  • the mobile phone detects that the user clicks on the control 303 shown in (b) in FIG. 3A
  • the interface as shown in (d) in FIG. 3A is displayed; after the mobile phone detects that the user clicks the control 304 , as shown in (c) of FIG. 3A , the slow-motion video recording function is activated.
  • the mobile phone After detecting that the user clicks the control 305 shown in (a) in FIG. 3B , the mobile phone enters the normal recording mode as shown in (b) in FIG. 3B .
  • the preview interface of the normal video recording mode includes a slow-motion double-speed control 306 .
  • the mobile phone starts the slow-motion recording function.
  • the mobile phone starts the slow-motion video recording function after detecting that the user has selected a slow-motion double-speed of 32X through the slow-motion double-speed control 306 .
  • the slow-motion multiplier kX indicates that the recording frame rate of the slow-motion video segment is k times the preset reference recording frame rate, and k is a positive integer.
  • the reference video recording frame rate may be a video recording frame rate usually adopted in a normal video recording mode, such as 30fps or 25fps.
  • the recording frame rate of the slow-motion video segment is the encoding frame rate of the slow-motion video segment.
  • the mobile phone displays the desktop or other non-camera application interface
  • the mobile phone after detecting the user's voice instruction to enter the slow-motion video recording mode, the mobile phone starts the slow-motion video recording function.
  • the mobile phone can also activate the slow-motion video recording function in response to other user operations such as touch operations, voice commands, or quick gestures.
  • the mobile phone uses the slow-motion video recording method provided in the following embodiments to record the target video.
  • the target video includes a slow-motion video segment, and the slow-motion video segment is used to record slow-motion wonderful moments.
  • the mobile phone uses the slow-motion video recording method provided in the following embodiments to record the target video only when the slow-motion video recording function is activated and the slow-motion multiplier set by the user is greater than or equal to the preset multiplier of 1.
  • the preset multiplier 1 is 8X.
  • the slow-motion video recording method provided in the following embodiment is used to record the target video.
  • the mobile phone after the mobile phone starts the slow-motion video recording function, it can enter the slow-motion video recording mode, so that the target video can be recorded in the slow-motion video recording mode by using the slow-motion video recording method provided in the following embodiments.
  • the mobile phone after the mobile phone starts the slow-motion video recording function, it may include multiple slow-motion video recording sub-modes, and the mobile phone uses the slow-motion video recording method provided in the following embodiments to record the target video only in specific sub-modes.
  • the preview interface of the mobile phone includes a normal slow motion control 401 and a super slow motion control 402. After detecting that the user clicks the super slow motion control 402, the mobile phone enters the super slow motion sub-mode, and adopts the following:
  • the slow-motion video recording method provided by the embodiment records the target video.
  • the mobile phone collects the image frame at the frame rate f1, and displays the preview image on the preview interface.
  • the mobile phone After the mobile phone starts the slow-motion video recording function, it can capture an image frame at the frame rate f1 in the preview state, obtain a preview image according to the image frame, and display the preview image on the preview interface at the frame rate f1.
  • the preview image can be generated after the image frames collected by the camera are processed by the ISP.
  • ISP processing operations may include converting a RAW image of an image frame to an RGB image, automatic white balance (AWB), dead pixel cluster correction, pre-filtering for noise reduction, color distortion correction, or chroma noise reduction. operate.
  • the frame rate f1 is relatively low, for example, it can be the frame rate of sending display and video recording usually adopted in the ordinary video recording mode, such as 30fps or 25fps. In a possible implementation solution, the frame rate f1 is equal to the above-mentioned reference video recording frame rate.
  • the process shown in step 202 may include: the camera collects image frames according to the frame rate f1 (for example, 30 fps) (that is, outputs the frames at the frame rate f1 ), and the collected image frames pass through the ISP frame by frame. After processing, a preview image is generated and displayed on the preview interface.
  • the frame rate f1 for example, 30 fps
  • the collected image frames pass through the ISP frame by frame.
  • a preview image is generated and displayed on the preview interface.
  • the method may further include step 203:
  • the mobile phone acquires the slow motion double speed.
  • the slow-motion double speed represents the multiple of the recording frame rate of the slow-motion video segment (hereinafter referred to as the recording frame rate corresponding to the slow-motion double speed) relative to the preset reference recording frame rate, for example, it may be 32X, 64X or 256X, etc., respectively. Expressed as 32 times, 64 times or 256 times the preset reference video frame rate.
  • the recording frame rate corresponding to the slow motion double speed is used to encode the target image frame between the start position and the end position of the slow motion set by the user after the start of recording, so as to generate a slow motion video segment in the target video.
  • the image frames in the slow-motion video segment have a high recording frame rate, and a low encoding frame rate and playback frame rate, which can facilitate users to watch slow-motion wonderful moments slowly, carefully, and for a long time. That is to say, the slow-motion video segment is used to record slow-motion wonderful moments.
  • the slow-motion multiplier is the default multiplier, or the multiplier used in the last slow-motion recording process.
  • the slow-motion multiplier is automatically determined by the mobile phone according to the shooting scene.
  • the slow-motion multiplier can be larger (such as 256X or 128X, etc.)
  • the slow-motion multiplier may be smaller (such as 64X or 32X, etc.).
  • the slow-motion multiplier is the multiplier set by the user.
  • multiple slow-motion double-speed controls 601 are displayed on the preview interface.
  • the mobile phone determines the slow-motion double-speed set by the user according to the slow-motion double-speed control selected by the user.
  • the slow-motion multiplier may be the multiplier set when the user activates the slow-motion video recording function.
  • the mobile phone may acquire the slow-motion multiplier set by the user according to the user's voice instruction.
  • the mobile phone can also modify the slow motion speed in response to the user's operation.
  • step 203 is an optional step.
  • the mobile phone After detecting the user's video recording operation, the mobile phone collects image frames at the frame rate f2, and performs image caching.
  • the mobile phone determines that the recording operation of the user is detected, thereby entering the recording process.
  • the mobile phone detects the operation of the user's voice instructing to start recording, it determines that the user's recording operation is detected, thereby entering the recording process.
  • the manner for triggering the mobile phone to enter the recording process may also include gestures and other manners, which are not specifically limited in this embodiment of the present application.
  • the mobile phone after the mobile phone detects the user's video recording operation, it can collect image frames at the frame rate f2 through the camera.
  • the frame rate f2 may be larger.
  • the frame rate f2 is greater than the frame rate f1, that is, the frame rate of the image acquisition of the mobile phone in the preview state is lower, and the frame rate of the image acquisition during the video recording process is relatively high.
  • the frame rate f1 may be 30fps
  • the frame rate f2 may be 1920fps, 960fps or 240fps, and so on.
  • f2 may be the default frame rate, the frame rate set by the user, or the maximum frame rate that the ISP can support (ie, the maximum frame rate that the ISP can perform real-time image processing).
  • the frame rate f2 may be related to the slow-motion double-speed.
  • the frame rate f2 is the recording frame rate corresponding to the slow motion multiplier.
  • the frame rate f2 is the frame rate corresponding to the preset multiplier 2.
  • the frame rate corresponding to the preset multiplier 2 is the maximum frame rate that can be supported by the ISP.
  • the maximum frame rate that the ISP can support is 1920fps
  • the reference video frame rate is 30fps
  • the frame rate f2 is the maximum frame rate 1920fps that the ISP can support.
  • the mobile phone after the mobile phone detects the user's video recording operation, it can perform image caching according to the image frames collected at the frame rate f2. Among them, due to the high frame rate f2, the ISP may not be able to process frame by frame in real time, so the mobile phone can first cache the image frames with the frame rate f2, so that the involved image frames can be processed by the ISP in subsequent use.
  • the mobile phone after the mobile phone detects the user's video recording operation, it can first buffer the image frames captured at the high frame rate f2 in real time until the buffering duration is equal to T0. After the cache time is longer than T0, the mobile phone caches the image frames collected at the high frame rate f2 during the time T0 between the current time t0 and the previous time t1 in real time, so that after the slow motion trigger is detected, according to the latest cached time T0 time Image frames captured at frame rate f2 generate slow motion video segments.
  • the image frames collected at the frame rate f2 within the most recent T0 duration cached by the mobile phone may be referred to as the first image frame set.
  • the mobile phone can cache the image frames with the frame rate f3 obtained after frame extraction.
  • the duration of T0 is shorter, such as 0.5s, 1s, or 1.5s, etc.; f3 is smaller, such as 25fps or 30fps, etc., and f3 and f1 may be equal or unequal.
  • the mobile phone after starting the recording, the mobile phone only needs to cache the image frames of the high frame rate f2 in the short T0 duration during the recording process, and the image frames of the low frame rate f3 obtained by the frame extraction before the T0 duration during the recording process , instead of caching the high frame rate f2 image frames collected during the entire recording process, so it can reduce the amount of data that the mobile phone caches in real time during the recording process, and reduce the recording process's requirements for the mobile phone's cache capacity.
  • the mobile phone after the mobile phone detects the user's video recording operation, it can initialize three DDR buffer queues as shown in Figure 8: buffer queue 1, buffer queue 2, and buffer queue 3.
  • the length of the cache queue 2 is f2*T0.
  • the camera captures image frames at frame rate f2 (eg, 1920fps).
  • the image frame newly collected by the camera is put into the head of the cache queue 2. If the cache queue 2 is full, that is, the f2*T0 frame image has been cached in the cache queue 2, the mobile phone will extract the image frame at the end of the cache queue 2 into a constant-speed image frame with a frame rate of f3 and move it to the cache queue 1. head of the team.
  • the mobile phone After detecting the user's video recording operation, the mobile phone extracts frames from the image frames collected at the frame rate f2 to generate a recorded image, and displays the recorded image on the shooting interface.
  • the image frame collected at the frame rate f2 is extracted to obtain an image frame with a frame rate of f3, and the image frame with a frame rate of f3 is processed by ISP.
  • a recorded image is generated, and the recorded image is displayed on the shooting interface.
  • the image frames processed by the ISP can also be sent to the slow motion detection module to detect whether the slow motion is triggered.
  • the image frames with the frame rate f2 buffered in the buffer queue 2 can be extracted into image frames with the frame rate f3, and then sent to the ISP for processing to generate a video frame. Take an image and send it to display.
  • the mobile phone uses high frame rate f2 to collect image frames, it only performs ISP processing and display on image frames with low frame rate f3 obtained after frame extraction, so the calculation amount of image processing can be reduced. Reduce the requirement for the real-time computing power of the mobile phone during the recording process.
  • the mobile phone After the mobile phone detects the slow motion trigger, it collects image frames with a duration of T1 at the frame rate f2 and caches them.
  • the mobile phone After the mobile phone detects the slow motion trigger, it can be determined that the photographed object is currently in the process of motion, so that a slow motion video segment can be generated according to the image frames in the motion process.
  • the slow motion trigger may be automatically triggered by the mobile phone, or may be actively triggered by the user.
  • the automatic triggering algorithm adopted by the mobile phone may include a human motion detection algorithm or an optical flow motion detection algorithm.
  • the human motion detection algorithm can detect the motion of the photographed human body. If the mobile phone detects the movement of the human body being photographed, the continuous process of the movement can be processed in slow motion, thereby generating a slow-motion video segment.
  • Table 1 can refer to the correspondence between motion actions and detection conditions. After the mobile phone meets the detection conditions, it is determined that the motion of the human body is detected, so that it can be determined that the slow motion is triggered.
  • the optical flow motion detection algorithm can detect whether the photographed object has motion by detecting the intensity of the picture change between adjacent image frames. If the mobile phone detects that the object to be photographed moves, slow motion processing is performed on the motion process to generate a slow motion video segment, so it can be determined that slow motion is triggered.
  • a slow motion trigger control is displayed on the shooting interface, and the mobile phone determines that the slow motion is triggered after detecting the user's operation of clicking the control. For another example, after the mobile phone detects the user's voice instruction triggered by the slow motion, it is determined that the slow motion is triggered.
  • the mobile phone after detecting the video recording operation, the mobile phone caches the image frames collected at the frame rate f2 during the time T0 between the current time t0 and the previous time t1 in real time during the video recording process, and the image frames before time t1
  • the frame rate after the frame is the image frame of f3.
  • the mobile phone caches the image frames collected at the frame rate f2 within the T0 time period before the slow motion trigger is detected. Since it takes a certain amount of time for the mobile phone to detect the slow motion trigger, after the mobile phone detects the slow motion trigger, the slow motion may have lasted for a period of time. Therefore, the mobile phone caches the data collected at the frame rate f2 during the T0 period before the mobile phone cache detects the slow motion trigger. Image frames, you can try to record the slow motion process completely.
  • the mobile phone can use the frame rate f2 to continue to collect image frames of the subsequent T1 duration and cache them, so as to make the whole slow motion process as complete as possible. recorded.
  • the cached image frames with a frame rate of f2 in the T1 duration after the slow motion trigger is detected may be referred to as a second image frame set.
  • the mobile phone caches an image frame with a frame rate of f2 in the duration T0 before the slow motion trigger is detected, and the cached image frame with a frame rate of f2 in the T1 period after the slow motion trigger is detected, Used to generate slow motion video clips.
  • the length of the cache queue 2 can be extended to f2*(T0+T1);
  • the image frames newly collected by the camera at the frame rate f2 are put into the queue head of the cache queue 2, and the image frames are stopped when the cache queue 2 is full.
  • the above time lengths of T0 and T1 are preset by the mobile phone.
  • the time lengths of T0 and T1 may be set by the user.
  • the user can set the values of T0 and T1 according to the shooting scene.
  • the time lengths of T0 and T1 are automatically determined by the mobile phone according to the shooting scene. For example, if the duration of the motion process in the shooting scene is long (such as the shooting scene of falling water droplets), T0 and T1 may be longer, so that the mobile phone can cache more image frames with a frame rate of f2. In this way, more image frames can be included between the start frame and the end frame selected by the user, so that there are more video frames in the slow-motion video segment generated according to the image frames between the start frame and the end frame, so that Users can watch slow-motion wonderful moments slowly and carefully through more video frames in the slow-motion video segment.
  • T0 and T1 may be short.
  • the mobile phone displays the image frame to be selected according to the cached image frame, and acquires the start frame and the end frame set by the user.
  • the mobile phone After the mobile phone detects the slow motion trigger, it can display the image frame to be selected on the frame selection interface according to the image frames in the first image frame set and the second image frame set, so as to obtain the user-based image frame based on the image frame to be selected as shown in FIG. 9 .
  • the image frame with the frame rate f2 between the start frame and the end frame may be the target image frame or used to generate the target image frame, and the target image frame is used to generate the slow motion video segment in the target video obtained by final shooting.
  • the slow-motion video segment is used to record the wonderful moments of slow-motion desired by the user.
  • the mobile phone can display the image frames to be selected on the frame selection interface in various manners, which are not specifically limited in this embodiment of the present application.
  • the image frames to be selected are image frames in the first image frame set and the second image frame set, so that the user can select a start frame and an end frame therefrom.
  • the number of image frames with the frame rate f2 in the first image frame set and the second image frame set is large, and the difference between adjacent image frames is small, which is inconvenient for the user to select the starting point. frame and end frame. Therefore, the mobile phone can obtain image frames to be selected after extracting frames from the image frames in the first image frame set and the second image frame set. Then, the mobile phone displays the image frames to be selected on the frame selection interface, so that the user can quickly select the start frame and the end frame based on a small number of image frames with large differences.
  • the number of image frames to be selected is also different if the slow-motion multiplier is different.
  • the larger the slow-motion multiplier the higher the motion speed of the motion process the user wants to shoot, and the greater the difference between adjacent image frames.
  • a large number of image frames to be selected can accurately select the start frame and end frame corresponding to the high-speed motion process.
  • the smaller the slow motion multiplier the smaller the number of image frames to be selected. That is to say, the number of image frames to be selected is positively related to the slow motion double speed. For example, if the slow motion multiplier is 32X, the image frames to be selected may be 30 frames; if the slow motion multiplier is 64X, the image frames to be selected may be 60 frames.
  • the frame extraction interval at which the mobile phone extracts frames to obtain the image frame to be selected from the cached first image frame set and the second image frame set is also different.
  • the larger the slow-motion multiplier the more image frames in the first image frame set and the second image frame set, and the frame sampling interval can also be correspondingly larger, so as to facilitate the user to obtain images based on the larger frame sampling interval.
  • the frame sampling interval can be 16 frames; if the slow motion multiplier is 64X, the frame sampling interval can be 32 frames. That is to say, the frame sampling interval used to obtain the image frame to be selected is positively related to the slow motion double speed.
  • the user can set the number of image frames to be selected or the frame sampling interval.
  • the mobile phone may display a frame number control/frame sampling interval control of multiple image frames to be selected on the frame selection interface, so that the user can select the number of image frames to be selected/frame sampling interval.
  • the larger the slow-motion multiplier the larger the number of image frames corresponding to the frame number control displayed on the mobile phone.
  • the frame number controls displayed on the frame selection interface of the mobile phone include: a 15-frame control, a 30-frame control, and a 45-frame control.
  • the frame number controls displayed on the frame selection interface of the mobile phone include: 30-frame control, 60-frame control and 90-frame control.
  • the larger the slow motion multiplier the larger the frame sampling interval corresponding to the frame sampling interval control displayed on the frame selection interface of the mobile phone.
  • the mobile phone can first select a larger image frame from the image frames of the first image frame set and the second image frame set. Frames are drawn at intervals to obtain image frames to be selected and displayed on the frame selection interface, so that the user can first select the reference start frame and reference end frame in a coarse-grained manner. Then, the mobile phone can also extract frames from the image frames of the first image frame set and the second image frame set at a small frame sampling interval to obtain the reference starting frame and the nearby candidate image frames (which may be referred to as the third image frame). frame set), and the reference end frame and nearby candidate image frames (may be referred to as the fourth image frame set). The mobile phone displays the third image frame set and the fourth image frame set on the frame selection interface, so that the user can select the start frame and the end frame in a fine-grained and precise manner.
  • the mobile phone in order to facilitate the user to select the start frame and the end frame, can display the recommended start frame and the recommended end frame automatically determined by the algorithm on the frame selection interface, or mark the recommended start frame in the image frame to be selected.
  • Start frame and recommended end frame The start frame and end frame that the user wants to select are usually the recommended start frame and the recommended end frame. In this way, by prompting the user with the recommended start frame and the recommended end frame on the frame selection interface, the mobile phone can facilitate the user to accurately select the start frame near the recommended start frame and accurately select the end frame near the recommended end frame.
  • the mobile phone in order to facilitate the user to select the start frame and the end frame, can display the recommended start frame interval and the recommended end frame interval automatically determined by the algorithm on the frame selection interface, or mark the image frame to be selected.
  • Recommended start frame interval and recommended end frame interval The start frame and end frame that the user wants to select is usually within or near the recommended start frame interval and the recommended end frame interval.
  • the mobile phone can facilitate the user to accurately select the starting frame in or near the recommended starting frame interval, and accurately select the starting frame in or near the recommended ending frame interval. to select the end frame.
  • the image frames to be selected can be displayed in the form of thumbnails, so that thumbnails of multiple image frames can be displayed on the interface at the same time, which is convenient for the user to select.
  • Thumbnails of image frames to be selected can be displayed anywhere on the frame selection interface, for example, they are arranged horizontally at the bottom of the frame selection interface, or vertically arranged on the right side of the frame selection interface. Among them, the interface may only display the thumbnails of some of the image frames to be selected at the same time, but not all of the image frames to be selected at the same time. Thumbnail of the frame.
  • the mobile phone after the mobile phone detects a slow motion trigger, it performs frame extraction and other processing on the subsequently collected image frames with a frame rate of f2, and generates a recorded image with a frame rate of f3.
  • the recorded image is displayed on the background of the frame selection interface in real time at the frame rate f3, and the thumbnail of the image frame to be selected is displayed on the foreground of the frame selection interface at the same time.
  • the currently displayed recorded image is continuously displayed on the background of the frame selection interface, and the recorded image displayed in the background is no longer refreshed, and is displayed on the foreground of the frame selection interface. Thumbnails of the image frames to be selected are displayed on the screen.
  • the recorded image is no longer displayed on the frame selection interface, but only the thumbnails of the image frames to be selected are displayed on the frame selection interface.
  • a shooting control is displayed on the frame selection interface, and the state of the shooting control is consistent with that of the shooting control on the preview interface.
  • the shooting controls are not displayed on the frame selection interface.
  • the mobile phone continues to display the shooting controls on the frame selection interface. After the mobile phone stops capturing images, the shooting controls are no longer displayed on the frame selection interface; or, the mobile phone displays the shooting controls on the frame selection interface, and the shooting controls are in the same state as the shooting controls on the preview interface.
  • the mobile phone displays the photographing interface and the recorded photographing image as shown in (a) of FIG. 10 .
  • the mobile phone may prompt the user to detect a slow motion trigger on the shooting interface.
  • the mobile phone continues to refresh and display the recorded images on the shooting interface, until the mobile phone stops capturing image frames and stops refreshing and displaying the recorded images.
  • the mobile phone may display thumbnails 1000 of the image frames to be selected on the frame selection interface, and prompt the user to set the start frame and the end frame.
  • the thumbnails 1000 of the image frames to be selected can be slidably switched to display thumbnails of other image frames to be selected in response to the user's sliding operation. For example, in response to the user's operation of sliding to the right based on the thumbnail of the image frame to be selected as shown in (f) in FIG. 10 , as shown in (g) of FIG. 10 , the thumbnail of the image frame to be selected also moves to the right Swipe to display thumbnails of other candidate image frames.
  • the frame selection interface further includes a first control 1001 for indicating the start frame and a second control 1002 for indicating the end frame.
  • the first control 1001 and the second control 1002 can slide on the thumbnails of the image frames to be selected in response to the user's drag operation.
  • the mobile phone may prompt the user to select the starting frame on the frame selection interface .
  • the mobile phone can also prompt the user whether the selection of the starting frame is completed on the frame selection interface. After the mobile phone detects that the user clicks the "OK" control, it is determined that the selection of the starting frame is completed. Then, as shown in (h) of FIG.
  • the mobile phone may prompt the user to select the end frame on the frame selection interface.
  • the selection method of the end frame is similar to that of the start frame, and will not be repeated.
  • the mobile phone after detecting the user's recording operation, can also display the slow-motion double speed on the interface of the recording process, such as "256X" shown in (a)-(h) of FIG. 10 .
  • the user can select the start frame and the end frame based on the first control 1001 and the second control 1002, respectively, and the mobile phone can prompt the user on the frame selection interface, "Are the start and end frames selected to complete?" The mobile phone After detecting that the user clicks the "OK" control, the start frame corresponding to the position of the first control 1001 and the end frame corresponding to the position of the second control 1002 are acquired.
  • the mobile phone After detecting the slow motion trigger, if the mobile phone detects the user's preset operation 1 for the image frame 1 in the thumbnail of the image frame to be selected, the mobile phone determines that the image frame 1 is the starting frame. If the user's preset operation 2 with respect to the image frame 2 in the thumbnail of the image frame to be selected is detected, the image frame 2 is determined as the end frame.
  • the preset operation 1 and the preset operation 2 may be operations such as clicking, double-clicking, or pressure pressing, and the specific forms of the preset operation 1 and the preset operation 2 are not limited in this embodiment of the present application. In some embodiments, referring to (a) in FIG.
  • the mobile phone after detecting the slow motion trigger, the mobile phone prompts the user to select the start frame and the end frame on the frame selection interface.
  • the mobile phone detects the operation after the user clicks the image frame 1 and the image frame 2 in the thumbnail of the image frame to be selected, it determines that the timing of the image frame 1 and the image frame 2 is ahead.
  • the image frame is the start frame, and the image frame later in the timing sequence is the end frame.
  • a thumbnail image 1100 of the image frame to be selected may be displayed on the frame selection interface.
  • the thumbnail image of the image frame to be selected further includes a selection control 1101 for selecting a start frame and an end frame. Selection control 1101 selects one of the image frames. Referring to (b) in FIG. 11B , the selection control 1101 can slide on the thumbnails of the image frames to be selected in response to the user's drag operation, thereby selecting different image frames.
  • the interface also includes a large image 1102 of the image frame selected by the selection control 1101 .
  • the large image 1102 can scroll to display the image frame and adjacent image frames where the control 1101 is located in the thumbnail of the to-be-selected image frame in the form of animation.
  • the image frame corresponding to the selection control 1101 in real time can be displayed in the form of a large image on the frame selection interface, which is convenient for the user to see the specific content of the image frame selected by the selection control 1101, thereby facilitating the user to accurately select the start frame and the end frame.
  • the mobile phone may first prompt the user to select the starting frame.
  • a selection mark for example, the ⁇ shown in the figure or the text mark "starting frame" is displayed on the image frame, And the image frame is determined as the start frame. Then, referring to (c) in FIG. 11B , the mobile phone can prompt the user to select the end frame. As shown in (d) of FIG. 11B , the selection method of the end frame is similar to that of the start frame, and will not be repeated.
  • the mobile phone may display the image frame at the preset position in the image frame to be selected in the form of a large image on the frame selection interface.
  • the preset position is the middle position of the to-be-selected image frame or the interface boundary position or the like.
  • the above selection control can be replaced with a draggable dot 1201 as shown in (a)-(e) in FIG. 12 or other forms of controls. be limited.
  • the mobile phone may display a large image of the image frame selected by the selection control on the frame selection interface, or may not display the large image.
  • the process of the user selecting the start frame and the end frame is similar to the method shown in (a)-(d) in FIG. 11B , and will not be repeated.
  • the mobile phone detects the slow motion trigger, it no longer refreshes and displays a new recording image on the background of the frame selection interface, but continuously displays the recorded recording from the memory until the slow motion trigger is displayed on the background. image, and display the thumbnail of the image frame to be selected on the foreground of the frame selection interface.
  • the thumbnail of the image frame to be selected can be displayed, and the user is prompted to select the interval corresponding to the start frame and the end frame.
  • the mobile phone determines that the first frame of the image frame in the frame selection is the starting frame. The last image frame in the image frame interval is the end frame.
  • the mobile phone may also prompt the user to select the slow-motion double speed.
  • the mobile phone can display multiple multi-speed controls on the frame selection interface for the user to choose.
  • the thumbnails of the image frames to be selected obtained at larger frame sampling intervals can be displayed on the frame selection interface, so that the user can The reference start frame is selected granularly.
  • the frame selection interface displays the image frames to be selected near the reference start frame ( That is, the thumbnail image of the third image frame set), so that the user can accurately select the starting frame with fine granularity.
  • the mobile phone may also choose to refer to the end frame and the end frame, which will not be repeated.
  • the mobile phone after detecting the slow motion trigger, continues to refresh the newly obtained recorded image on the background of the frame selection interface, and at the same time displays the thumbnail of the image frame to be selected on the foreground of the frame selection interface. .
  • the thumbnail of the image frame to be selected is displayed on the frame selection interface, and the recommended start frame and the recommended end frame are marked, so that the user can select the recommended start frame and the recommended end frame respectively.
  • the image frames to be selected include a fifth image frame set and a sixth image frame set
  • the fifth image frame set includes the recommended starting frame as shown in the first row of thumbnails in FIG. 15 and adjacent
  • the sixth image frame set includes the recommended end frame and adjacent multi-frame images as shown in the thumbnail image in the second row in FIG. 15 .
  • the user may set the start frame based on the fifth image frame set and the recommended start frame, and set the end frame based on the sixth image frame set and the recommended end frame.
  • the thumbnails of the image frames to be selected are displayed on the frame selection interface, including the recommended start frame interval and the recommended end. Frame interval, so that the user can quickly and accurately select the start frame and the end frame within or near the recommended start frame interval and the recommended end frame interval, respectively.
  • thumbnails of the image frames to be displayed are arranged horizontally on the frame selection interface.
  • FIG. 17 For a schematic diagram of vertical arrangement of thumbnails of image frames to be displayed, see FIG. 17 .
  • the mobile phone generates a target video, where the target video includes a slow-motion video segment, and the slow-motion video segment corresponds to a start frame and an end frame.
  • a target video can be generated, and the target video includes a slow motion video segment obtained from the target image frames between the start frame and the end frame.
  • the frame rate between the target image frame and the end frame is f2.
  • the target image frame is an image frame obtained by interpolating an image frame with a frame rate of f2 between the start frame and the end frame.
  • the frame rate of the target image frame is f4, which may be greater than or equal to f2.
  • Image post-processing operations can be performed in processors such as CPU, GPU, or NPU.
  • image post-processing operations may include image smoothing, image sharpening, image enhancement, histogram equalization, image super-resolution, noise reduction or frame interpolation, and the like.
  • the frame rate f4 of the target image frame may be referred to as the recording frame rate of the slow-motion video segment.
  • the encoding frame rate of the slow-motion video segment may be much smaller than the recording frame rate.
  • the recording frame rate may be 7680fps, and the encoding frame rate may be 30fps or 25fps.
  • the encoding frame rate is equal to f3.
  • the mobile phone can interpolate image frames with a frame rate of 1920fps between the start frame and the end frame to obtain a target image with a frame rate of 7680fps frame; the mobile phone uses 30fps to encode the target image frame with a frame rate of 7680fps to generate a slow-motion video segment.
  • the recording frame rate f4 of the slow-motion video segment is 7680fps.
  • f4 is the default preset frame rate, or the encoding frame rate used when the target image frame was last encoded to generate the slow-motion video segment.
  • the video recording frame rate f4 corresponds to the obtained slow motion double speed.
  • the video frame rate corresponding to the slow motion double speed is the same as f2
  • f4 is the same as f2.
  • the recording frame rate corresponding to the slow motion double speed is greater than f2
  • f4 is greater than f2
  • f4 is the recording frame rate corresponding to the slow motion double speed.
  • the slow motion speed is 32X
  • the reference recording frame rate is 30fps
  • the slow-motion multiplier is 256X
  • the reference video frame rate is 30fps
  • the mobile phone If the mobile phone does not obtain the slow-motion double-speed in step 203, it can obtain the slow-motion double-speed in step 208, thereby obtaining the recording frame rate corresponding to the slow-motion double-speed, which is f4; or, the mobile phone can obtain it in step 208.
  • the target image frame 80 with the frame rate f2 between the start frame and the end frame in the buffer queue 2 can be used for frame interpolation to generate the target image frame with the frame rate f4, so that Encode to generate slow motion video segments.
  • f1 is 30fps or 25fps
  • f2 is 1920fps
  • the slow-motion multiplier is 256X
  • f3 is equal to f1
  • f4 is 7680fps.
  • the target video may further include a video segment 1 located before the slow-motion video segment, and/or a video segment 2 located after the slow-motion video segment.
  • the mobile phone extracts the image frames with the frame rate of f2 (called the eighth image frame set) before the starting frame, obtains the image frames with the frame rate of f3 and passes Video segment 1 is generated after ISP processing, image post-processing and encoding.
  • the mobile phone extracts an image frame with a frame rate of f2 (called the ninth image frame) after the end frame, obtains an image frame with a frame rate of f3 and passes the ISP Video segment 2 (also referred to as the second video segment) is generated after processing, image post-processing and encoding.
  • the mobile phone can extract image frames with a frame rate of f3 from the image frames in the cache queue 2 whose acquisition time is before the start frame selected by the user, and move them to the image frames of the cache queue 1.
  • the head of the line gets a set 801 of image frames.
  • the image frame set 801 in the buffer queue 1 can be used for encoding to generate the normal-speed video segment 1.
  • the mobile phone can extract the image frames whose collection time in the cache queue 2 is after the end frame selected by the user into an image frame with a frame rate of f3, and move it to the head of the cache queue 3 to obtain the image frame set 802 .
  • the image frame set 802 in the buffer queue 3 can be used for encoding to generate the normal-speed video segment 2 .
  • the mobile phone will no longer generate video segment 1, or the video segment 1 is empty; if the end frame is cached If the frame rate is the last image frame in the image frames of f2, the mobile phone no longer generates video segment 2, or the video segment 2 is empty.
  • the target video may also include video segment 0 preceding video segment 1 .
  • the mobile phone performs ISP processing and image post-processing on the buffered image frames with a frame rate of f3 before the T0 duration (called the seventh image frame set), and encodes them at a frame rate of f3 to generate video segment 0.
  • the image frame set 800 with the frame rate f3 buffered in the buffer queue 1 can be used for encoding to generate the normal-speed video segment 0 .
  • the video segment 0 and the video segment 1 may be referred to as the first video segment.
  • the sequence of the video segments in the target video may be: video segment 0, video segment 1, slow-motion video segment, and video segment 2, ie, the first video segment, the slow-motion video segment, and the second video segment.
  • the start frame and end frame of the target image frame used to generate the slow-motion video segment are precisely selected by the user during the recording process, that is, the user accurately selects the slow-motion video in the target video
  • the start position and end position of the segment can be avoided, so that the false detection that occurs when the mobile phone automatically detects the start/end time of the slow motion can be avoided, so that the generated slow motion video segment can accurately correspond to the captured slow motion moments, and can also satisfy the Users' personalized needs for wonderful moments in slow motion.
  • the mobile phone plays the generated target video, including video segment 0, video segment 1, slow-motion video segment, and video segment 2 in the playback target video.
  • the playback frame rate of the target video is less than f4.
  • the playback frame rate is usually equal to the encoding frame rate, or has a small difference from the encoding frame rate.
  • the playback frame rate is also much smaller than the recording frame rate of the slow-motion video segment.
  • the motion process corresponding to the slow-motion video segment can be stretched and played at a slow speed, so as to facilitate the user to pass a longer period of time. Watch the best moments in slow motion carefully and clearly.
  • the video frame rate and encoding frame rate of video segment 0, video segment 1 and video segment 2 are all f3, and the playback frame rate is equal to the encoded frame rate or not much different from the encoded frame rate, that is, the playback frame rate is equal to the video frame rate or The difference is not big, so it can be called a constant-speed video segment.
  • the target video may also include a normal-speed video segment before the slow-motion video segment, such as video segment 0 or video segment 1, etc., and may also include a normal-speed video after the slow-motion video segment, such as Video segment 2.
  • a normal-speed video segment before the slow-motion video segment such as video segment 0 or video segment 1, etc.
  • a normal-speed video after the slow-motion video segment such as Video segment 2.
  • the generated target video can give the user a visual contrast impact from the normal-speed video segment to the slow-motion video segment played at a slow speed, and then to the normal-speed video segment, and the user experience is better.
  • FIG. 19A for a sequence diagram of a slow-motion video recording method described in the above embodiments.
  • FIG. 19B For a schematic diagram of the effect of the video frame played by the target video generated by the mobile phone, please refer to FIG. 19B .
  • the method may further include: the mobile phone collects image frames with a duration of T2 at the frame rate f3 and buffers them.
  • the target video generated by the mobile phone also includes video segment 3 obtained by performing ISP processing and image post-processing on the cached image frames with a frame rate of f3 and a duration of T2, and after encoding according to the encoding frame rate.
  • the mobile phone automatically stops capturing image frames.
  • the method may further include: the mobile phone collects image frames at the frame rate f2 and buffers them, and does not stop collecting image frames until the user's operation of stopping recording is detected.
  • the encoding frame rate of the slow-motion video is getting higher and higher, and correspondingly, the capture frame rate f2 during the recording process is also getting higher and higher.
  • the duration of the motion process is short. In order to better see the motion details in a short period of time, the encoding frame rate of the slow-motion video is relatively high, and the captured frame rate during the recording process is correspondingly high. The rate f2 is also higher.
  • the mobile phone does not cache the high frame rate image frames collected at f2 during the entire video recording process, but only caches the high frame rate image frames within the duration T0 and T1, and compares the After the high frame rate image frame is extracted, it is processed and sent to the ISP, which can reduce the requirements for the mobile phone image cache transmission and computing power.
  • the mobile phone only performs high frame rate encoding on the high frame rate image frames between the start frame and the end frame selected by the user, and performs low frame rate encoding on other low frame rate image frames during the recording process, which can reduce the size of the image. Processing tasks, reducing the requirements for the computing power of the mobile phone.
  • the user can select the start frame and the end frame during the video recording process to generate a slow-motion video segment in the target video, and does not need to perform two steps after generating the video. Edited to get a slow-motion video clip.
  • the scheme of generating video first and then editing to obtain slow-motion video segments requires the ISP to process the captured image frames in real time during the recording process, and then encode them to generate the video.
  • the ISP has limited processing capabilities and is difficult to achieve high Real-time processing of image frames at the acquisition frame rate.
  • the mobile phone only caches the image frames with the frame rate f2 of the T0 duration after the mobile phone starts recording, and does not perform image caching in the preview state. In some other embodiments, in the preview state, the mobile phone caches the image frames with the frame rate f2 of the duration T0.
  • the mobile phone after the mobile phone starts the slow-motion video recording function, the mobile phone captures image frames at the frame rate f2 in the preview state, and extracts the frames into image frames with the frame rate f3 for display. Moreover, the mobile phone first buffers the image frames collected at the high frame rate f2 in real time until the buffering duration is equal to T0. After the cache time is longer than T0, the mobile phone caches the image frames collected at the high frame rate f2 during the T0 time between the current time t0 and the previous time t1 in real time, and extracts the image frames with the frame rate f2 before the T0 time. frame, get the image frame with the frame rate f3 and then buffer it.
  • the mobile phone After the mobile phone detects the user's video recording operation, it continues to cache the image frames in the previous cache state. That is to say, after the mobile phone detects the user's video recording operation, it continues the cache progress in the preview state, and continues to cache the image frames collected at the frame rate f2 during the period T0 between the current time t0 and the previous time t1, and the image frames before time t1. An image frame with a frame rate of f3 is obtained after the image frames collected at the frame rate of f2 are extracted. Then, the mobile phone can perform steps 205-208 in the above embodiment.
  • the mobile phone may detect the slow motion trigger soon after detecting the user's video recording operation, for example, the time difference between the moment when the slow motion trigger is detected and the moment when the video recording operation is detected is less than T0. At this time, if the mobile phone starts to cache image frames only after detecting the user's video recording operation, the cached image frames with high frame rate f2 may not include the complete slow motion process that occurs before the slow motion is triggered.
  • the mobile phone caches image frames with high frame rate f2 in the preview state, and continues to cache image frames in the preview state after detecting the user's video recording operation, it can make the cached image frames with high frame rate f2 as far as possible. Include the complete slow motion process that occurs before the slow motion trigger. In this way, when the mobile phone detects the slow motion trigger immediately after detecting the recording operation, it can also generate a complete slow motion video segment according to the cached image frame.
  • the electronic device is a mobile phone as an example for illustration.
  • the electronic device is a tablet computer or other device such as a camera
  • the slow-motion recording method provided in the above embodiment can still be used, and the user can accurately set the slow-motion recording method during the recording process.
  • the starting position and ending position of the action video segment enable the generated slow-motion video segment to accurately correspond to the captured slow-motion wonderful moment, and can also meet the user's personalized needs for the slow-motion wonderful moment, which will not be repeated here.
  • the electronic device includes corresponding hardware and/or software modules for executing each function.
  • the present application can be implemented in hardware or in the form of a combination of hardware and computer software in conjunction with the algorithm steps of each example described in conjunction with the embodiments disclosed herein. Whether a function is performed by hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functionality for each particular application in conjunction with the embodiments, but such implementations should not be considered beyond the scope of this application.
  • the electronic device can be divided into functional modules according to the above method examples.
  • each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module.
  • the above-mentioned integrated modules can be implemented in the form of hardware. It should be noted that, the division of modules in this embodiment is schematic, and is only a logical function division, and there may be other division manners in actual implementation.
  • Embodiments of the present application further provide an electronic device, including one or more processors and one or more memories.
  • the one or more memories are coupled to the one or more processors for storing computer program code, the computer program code comprising computer instructions that, when executed by the one or more processors, cause the electronic device to perform
  • the above-mentioned relevant method steps implement the slow-motion video recording method in the above-mentioned embodiment.
  • An embodiment of the present application further provides an electronic device, as shown in FIG. 20 , comprising: a display screen 2001, one or more processors 2002, multiple cameras 2003, a memory 2004, and one or more computer programs 2005, each of the above Devices may be connected by one or more communication buses 2006 .
  • the one or more computer programs 2005 are stored in the aforementioned memory 2004 and configured to be executed by the one or more processors 2002, the one or more computer programs 2005 include instructions that may be used to perform the aforementioned implementations the steps in the example.
  • the processor 2002 may be the processor 110 shown in FIG. 1
  • the memory 2004 may be the internal memory 121 shown in FIG. 1
  • the camera 2003 may be the camera 193 shown in FIG.
  • the display screen 2001 may specifically be the display screen 194 shown in FIG. 1 .
  • Embodiments of the present application further provide a computer-readable storage medium, where computer instructions are stored in the computer-readable storage medium, and when the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to realize the above-mentioned embodiments The slow-motion video recording method in .
  • Embodiments of the present application also provide a computer program product, which, when running on a computer, causes the computer to execute the above-mentioned relevant steps, so as to realize the slow-motion video recording method executed by the electronic device in the above-mentioned embodiment.
  • the embodiments of the present application also provide an apparatus, which may specifically be a chip, a component or a module, and the apparatus may include a connected processor and a memory; wherein, the memory is used to store computer execution instructions, and when the apparatus is running, The processor can execute the computer-executed instructions stored in the memory, so that the chip executes the slow-motion video recording method executed by the electronic device in the above method embodiments.
  • the electronic device, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding method provided above. Therefore, for the beneficial effects that can be achieved, reference may be made to the above-provided method. The beneficial effects in the corresponding method will not be repeated here.
  • the disclosed apparatus and method may be implemented in other manners.
  • the device embodiments described above are only illustrative.
  • the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods.
  • multiple units or components may be Incorporation may either be integrated into another device, or some features may be omitted, or not implemented.
  • the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place, or may be distributed to multiple different places . Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.
  • the above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.
  • the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a readable storage medium.
  • the technical solutions of the embodiments of the present application can be embodied in the form of software products in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, which are stored in a storage medium , including several instructions to make a device (may be a single chip microcomputer, a chip, etc.) or a processor (processor) to execute all or part of the steps of the methods described in the various embodiments of the present application.
  • the aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (random access memory, RAM), magnetic disk or optical disk and other media that can store program codes.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Human Computer Interaction (AREA)
  • Television Signal Processing For Recording (AREA)
  • Studio Devices (AREA)

Abstract

本申请实施例提供一种慢动作录像方法及设备,涉及电子技术领域,能够在录像过程中由用户来精准设置慢动作视频段的起始位置和结束位置,使得生成的慢动作视频段能够准确对应慢动作精彩瞬间,并且还能满足用户对慢动作精彩瞬间的个性化需求。方案包括:电子设备开启录像功能;在检测到用户的录像操作后,若检测到慢动作触发,则在选帧界面上显示待选图像帧,待选图像帧包括多帧图像;获取用户基于待选图像帧设置的起始帧和结束帧;生成目标视频,目标视频包括慢动作视频段,慢动作视频段与起始帧和结束帧相对应,慢动作视频段的录像帧率高于慢动作视频段的编码帧率。本申请实施例用于录制慢动作视频。

Description

一种慢动作录像方法及设备
本申请要求于2020年11月18日提交国家知识产权局、申请号为202011296546.8、申请名称为“一种慢动作录像方法及设备”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。
技术领域
本申请涉及电子技术领域,尤其涉及一种慢动作录像方法及设备。
背景技术
随着电子技术的发展,手机等电子设备的录像功能越来越强大。比如,慢动作录像可以拍摄物体高速运动过程中的慢动作细节,记录稍纵即逝的精彩瞬间。在现有慢动作录像方案中,电子设备判断运动过程的起/止时间,并根据该起/止时间生成慢动作视频段。该慢动作视频段可以方便用户慢速观看慢动作精彩瞬间。
其中,电子设备判断的慢动作的起/止时间可能不准确,生成的慢动作视频段的起始位置和结束位置也通常与用户的期望存在偏差,难以满足用户对慢动作精彩瞬间的个性化需求。
发明内容
本申请实施例提供一种慢动作录像方法及设备,能够在录像过程中由用户来精准设置慢动作视频段的起始位置和结束位置,使得生成的慢动作视频段能够准确对应慢动作精彩瞬间,并且还能满足用户对慢动作精彩瞬间的个性化需求。
为达到上述目的,本申请实施例采用如下技术方案:
一方面,本申请实施例提供了一种录像方法,应用于电子设备,该方法包括:电子设备开启录像功能。电子设备在检测到用户的录像操作后,若检测到慢动作触发,则在选帧界面上显示待选图像帧,待选图像帧包括多帧图像。电子设备获取用户基于待选图像帧设置的起始帧和结束帧。电子设备生成目标视频,目标视频包括慢动作视频段,慢动作视频段与起始帧和结束帧相对应,慢动作视频段的录像帧率高于慢动作视频段的编码帧率。
在该方案中,用户可以在录像过程中基于电子设备显示的待选图像帧,精准地选择起始帧和结束帧,从而使得电子设备根据起始帧和结束帧生成目标视频中的慢动作视频段,使得生成的慢动作视频段能够准确对应慢动作精彩瞬间,而且还能满足用户对慢动作精彩瞬间的个性化需求。慢动作视频段的录像帧率高于慢动作视频段的编码帧率,这样可以使得用户慢速、仔细、长时间地通过慢动作视频段观看慢动作精彩过程。
在一种可能的实现方式中,该方法还包括:电子设备在检测到用户的录像操作后,以第一帧率采集图像帧,并缓存第一图像帧集合,第一图像帧集合包括最近采集到的T0时长内的图像帧。电子设备在检测到慢动作触发后,以第一帧率采集T1时长的图像帧,并缓存第二图像帧集合,第二图像帧集合包括以第一帧率采集的T1时长的图像帧。其中,待选图像帧来自于第一图像帧集合和第二图像帧集合。
也就是说,电子设备可以缓存慢动作触发前T0时长内以第一帧率采集的图像帧,以及慢动作触发后T1时长内以第一帧率采集的图像帧,并根据缓存的该第一帧率的图像帧 显示待选图像帧,以便用户基于待选图像帧选择起始帧和结束帧。
在另一种可能的实现方式中,待选图像帧通过第一图像帧集合和第二图像帧集合中的图像帧抽帧获得。
这样,待选图像帧的数量较少,可以方便用户从中选择起始帧和结束帧。
在另一种可能的实现方式中,待选图像帧的数量与慢动作视频段的慢动作倍速正相关;或者,从第一图像帧集合和第二图像帧集合中的图像帧,抽帧获得待选图像帧的抽帧间隔,与慢动作视频段的慢动作倍速正相关。
也就是说,慢动作倍速越大,待选图像帧的数量也越多;慢动作倍速越小,待选图中帧的量也越少。慢动作倍速越大,用于抽帧获得待选图像帧的抽帧间隔越大;慢动作倍速越小,用于抽帧获得待选图像帧的抽帧间隔越小。
在另一种可能的实现方式中,慢动作视频段采用编码帧率,对起始帧及结束帧之间的目标图像帧获得。其中,目标图像帧为第一图像帧集合和第二图像帧集合中,起始帧及结束帧之间的图像帧,录像帧率等于第一帧率。或者,目标图像帧为第一图像帧集合和第二图像帧集合中,起始帧及结束帧之间第一帧率的图像帧,插帧后获得的录像帧率的图像帧。
也就是说,目标图像帧根据起始帧和结束帧获得,且目标图像帧为高帧率的图像帧。慢动作视频段采用低编码帧率,对高帧率的目标图像帧进行视频编码获得,可以使得用户慢速、仔细、长时间地通过慢动作视频段观看慢动作精彩过程。
在另一种可能的实现方式中,该方法还包括:电子设备在检测到用户的录像操作后,在拍摄界面上以第二帧率显示录拍图像,录拍图像根据第一帧率的图像帧抽帧后获得,第二帧率小于第一帧率。
在该方案中,在录像过程中,电子设备以高帧率采集图像帧,并以低帧率显示录拍图像,可以减小图像处理和图像显示的负载,节省电子设备的功耗。
在另一种可能的实现方式中,录像帧率与慢动作视频段的慢动作倍速相对应。
也就是说,电子设备可以根据慢动作倍速确定录像帧率的大小。
在另一种可能的实现方式中,第一帧率与慢动作视频段的慢动作倍速相关。
也就是说,在录像过程中,电子设备用于采集图像帧的帧率大小,可以根据慢动作倍速来确定。
在另一种可能的实现方式中,该方法还包括:若电子设备检测到慢动作触发,则在选帧界面上提示用户设置起始帧和结束帧。
这样,用户可以根据电子设备的直观提示,来选择起始帧和结束帧。
在另一种可能的实现方式中,电子设备确定用户基于待选图像帧设置的起始帧和结束帧,包括:若电子设备检测到用户针对待选图像帧上第一图像帧的第一预设操作,则确定第一图像帧为起始帧。若电子设备检测到用户针对待选图像帧上第二图像帧的第二预设操作,则确定第二图像帧为结束帧。
在该方案中,电子设备可以基于用户的预设操作,确定用户设置的起始帧和结束帧。
在另一种可能的实现方式中,选帧界面上以大图形式显示有待选图像帧中位于预设位置的图像帧,预设位置为待选图像帧的中间位置或选帧界面的边界位置。
这样,用户可以通过大图清楚地查看图像帧的内容信息。
在另一种可能的实现方式中,该方法还包括:若电子设备检测到慢动作触发,则在选 帧界面上显示第一控件和第二控件。第一控件和第二控件可在待选图像帧上滑动,第一控件用于设置起始帧,第二控件用于设置结束帧。选帧界面上以大图形式显示第一控件或第二控件所在位置的图像帧。当用户在待选图像帧上拖动第一控件时,界面上以大图形式显示第一控件所在位置的图像帧。当用户在待选图像帧上拖动第二控件时,界面上以大图形式显示第二控件所在位置的图像帧。电子设备确定用户基于待选图像帧设置的起始帧和结束帧,包括:电子设备确定待选图像帧中,第一控件所在位置的图像帧为起始帧。电子设备确定待选图像帧中,第二控件所在位置的图像帧为结束帧。
在该方案中,用户可以通过第一控件和第二控件分别设置起始帧和结束帧,且电子设备可以以大图的形式显示用户当前针对的第一控件或第二控件所在位置的图像帧。
在另一种可能的实现方式中,待选图像帧采用第一间隔从第一图像帧集合和第二图像帧集合的图像帧中抽帧获得。电子设备确定用户基于待选图像帧设置的起始帧和结束帧,包括:电子设备确定用户基于待选图像帧设置的参考起始帧。电子设备在选帧界面上显示第三图像帧集合,第三图像帧集合包括参考起始帧,且第三图像帧集合采用第二间隔从第一图像帧集合和第二图像帧集合的图像帧中抽帧获得,第二间隔小于第一间隔。电子设备确定用户基于第三图像帧集合设置的起始帧,确定用户基于待选图像帧设置的参考结束帧。电子设备在界面上显示第四图像帧集合,第四图像帧集合包括参考结束帧,且第四图像帧集合采用第二间隔从第一图像帧集合和第二图像帧集合的图像帧中抽帧获得。电子设备确定用户基于第四图像帧集合设置的结束帧。
在该方案中,电子设备可以先以大间隔显示待选图像帧,以便用户粗粒度地选择参考起始帧和参考结束帧;而后,电子设备再以小间隔显示参考起始帧和参考结束帧附近的图像帧,以便用户精准地选择起始帧和结束帧。
在另一种可能的实现方式中,待选图像帧包括第五图像帧集合和第六图像帧集合,第五图像帧集合包括推荐起始帧及相邻的多帧图像,第六图像帧集合包括推荐结束帧及相邻的多帧图像。电子设备确定用户基于待选图像帧设置的起始帧和结束帧,包括:电子设备确定用户基于第五图像帧集合和推荐起始帧设置的起始帧。电子设备确定用户基于第六图像帧集合和推荐结束帧设置的结束帧。
在该方案中,电子设备可以先自动确定较为准确的推荐起始帧和推荐结束帧,并显示给用户;以便用户在推荐起始帧附近精准地选择起始帧,并在推荐结束帧附近精准地选择结束帧。
在另一种可能的实现方式中,待选图像帧包括推荐起始帧区间和推荐结束帧区间。电子设备确定用户基于待选图像帧设置的起始帧和结束帧,包括:电子设备确定用户基于推荐起始帧区间设置的起始帧;电子设备确定用户基于推荐结束帧区间设置的结束帧。
在该方案中,电子设备可以先自动确定较为准确的推荐起始帧区间和推荐结束帧区间,并显示给用户;以便用户基于推荐起始帧区间和推荐结束帧区间,精准地选择起始帧和结束帧。
在另一种可能的实现方式中,该方法还包括:电子设备在检测到用户的录像操作后,缓存第七图像帧集合,第七图像帧集合包括T0时长之前的第一帧率的图像帧抽帧获得的第二帧率的图像帧。目标视频还包括第一视频段,第一视频段采用编码帧率对第七图像帧集合和第八图像帧集合中的图像帧编码获得。其中,第八图像帧集合为第一图像帧集合和 第二图像帧集合中,起始帧之前的第一帧率的图像帧,抽帧获得的第二帧率的图像帧。
其中,第一视频段为常速视频段。这样,在播放目标视频时可以先播放常速第一视频段,再播放慢动作视频段,给用户以快慢切换的视觉冲击。
在另一种可能的实现方式中,目标视频还包括第二视频段,第二视频段采用编码帧率对第九图像帧集合中的图像帧编码获得。其中,第九图像帧集合为第一图像帧集合和第二图像帧集合中,结束帧之后的第一帧率的图像帧,抽帧获得的第二帧率的图像帧。
其中,第一视频段为常速视频段。这样,在播放目标视频时可以先播放慢动作视频段,再播放常速的第二视频段,给用户以慢快切换的视觉冲击。
在另一种可能的实现方式中,该方法还包括:电子设备在开启录像功能后,以第三帧率采集图像,并在预览界面上以第三帧率显示预览图像,第三帧率小于第一帧率。
也就是说,在预览状态下,电子设备可以采用低帧率采集图像帧并显示预览图像。
另一方面,本申请实施例提供了一种拍摄装置,该装置包含在电子设备中。该装置具有实现上述方面及可能的设计中任一方法中电子设备行为的功能,使得电子设备执行上述方面任一项可能的设计中电子设备执行的录像方法。该功能可以通过硬件实现,也可以通过硬件执行相应的软件实现。硬件或软件包括至少一个与上述功能相对应的模块或单元。例如,该装置可以包括开启单元、检测单元、显示单元、获取单元和生成单元等。
又一方面,本申请实施例提供了一种电子设备,包括:摄像头,用于采集图像;显示屏,用于显示界面;一个或多个处理器;存储器;以及一个或多个计算机程序,一个或多个计算机程序被存储在存储器中,一个或多个计算机程序包括指令,当指令被电子设备执行时,使得电子设备执行上述方面任一项可能的设计中电子设备执行的录像方法。
又一方面,本申请实施例提供了一种电子设备,包括:一个或多个处理器;以及存储器,存储器中存储有代码。当代码被电子设备执行时,使得电子设备执行上述方面任一项可能的设计中电子设备执行的录像方法。
又一方面,本申请实施例提供了一种计算机可读存储介质,包括计算机指令,当计算机指令在电子设备上运行时,使得电子设备执行上述方面任一项可能的设计中的录像方法。
又一方面,本申请实施例提供了一种计算机程序产品,当计算机程序产品在计算机上运行时,使得计算机执行上述方面任一项可能的设计中电子设备执行的录像方法。
又一方面,本申请实施例提供了一种芯片系统,该芯片系统应用于电子设备。该芯片系统包括一个或多个接口电路和一个或多个处理器;接口电路和处理器通过线路互联;接口电路用于从电子设备的存储器接收信号,并向处理器发送信号,信号包括存储器中存储的计算机指令;当处理器执行计算机指令时,使得电子设备执行上述方面任一项可能的设计中的录像方法。
上述其他方面对应的有益效果,可以参见关于方法方面的有益效果的描述,此处不予赘述。
附图说明
图1为本申请实施例提供的一种电子设备的结构示意图;
图2为本申请实施例提供的一种慢动作录像方法的流程示意图;
图3A为本申请实施例提供的一组界面示意图;
图3B为本申请实施例提供的另一组界面示意图;
图4为本申请实施例提供的一种界面示意图;
图5为本申请实施例提供的一种图像处理过程示意图;
图6为本申请实施例提供的另一种界面示意图;
图7为本申请实施例提供的另一种图像处理过程的示意图;
图8为本申请实施例提供的一种缓存方案的示意图;
图9为本申请实施例提供的一种生成目标视频的过程示意图;
图10为本申请实施例提供的另一组界面示意图;
图11A为本申请实施例提供的另一组界面示意图;
图11B为本申请实施例提供的又一组界面示意图;
图12为本申请实施例提供的另一组界面示意图;
图13为本申请实施例提供的又一组界面示意图;
图14为本申请实施例提供的另一组界面示意图;
图15为本申请实施例提供的另一种界面示意图;
图16为本申请实施例提供的又一组界面示意图;
图17为本申请实施例提供的另一种界面示意图;
图18为本申请实施例提供的一种目标视频的结构示意图;
图19A为本申请实施例提供的一种慢动作录像方法的时序图;
图19B为本申请实施例提供的一种目标视频播放的视频帧的效果示意图;
图20为本申请实施例提供的另一种电子设备的结构示意图。
具体实施方式
下面将结合本申请实施例中的附图,对本申请实施例中的技术方案进行描述。其中,在本申请实施例的描述中,除非另有说明,“/”表示或的意思,例如,A/B可以表示A或B;本文中的“和/或”仅仅是一种描述关联对象的关联关系,表示可以存在三种关系,例如,A和/或B,可以表示:单独存在A,同时存在A和B,单独存在B这三种情况。另外,在本申请实施例的描述中,“多个”是指两个或多于两个。
以下,术语“第一”、“第二”仅用于描述目的,而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此,限定有“第一”、“第二”的特征可以明示或者隐含地包括一个或者更多个该特征。在本实施例的描述中,除非另有说明,“多个”的含义是两个或两个以上。
在本申请实施例中,“示例性地”或者“例如”等词用于表示作例子、例证或说明。本申请实施例中被描述为“示例性地”或者“例如”的任何实施例或设计方案不应被解释为比其它实施例或设计方案更优选或更具优势。确切而言,使用“示例性地”或者“例如”等词旨在以具体方式呈现相关概念。
慢动作录像可以拍摄物体的高速运动过程,捕捉肉眼难以看到的运动细节,记录稍纵即逝的精彩瞬间,也可以称为高速录像。比如,慢动作录像可以记录子弹飞出膛时的运动状态,足球射门时的动作过程,水滴下落时溅起涟漪的瞬间等。在现有慢动作录像方案中,电子设备可以通过人体动作检测算法或光流动作检测算法等算法自动判断运动过程的起/止时间,并根据该起/止时间自动生成慢动作视频段。而电子设备自动判断的运动过程的起/止时间常常不准确,比如用户想捕捉子弹出膛的瞬间,但子弹出膛时画面的变化幅度并不 大,电子设备通过光流动作检测算法等难以准确抓取到子弹出膛的起始时间。电子设备根据该起/止时间自动生成的慢动作视频段的起始位置和结束位置,也通常与用户的期望存在偏差,难以满足用户对慢动作精彩瞬间的个性化需求。
本申请实施例提供了一种慢动作录像方法,可以应用于电子设备,能够在录像过程中,由用户来精准设置慢动作视频段的起始位置和结束位置,使得生成的慢动作视频段能够准确对应被拍摄的慢动作精彩瞬间,并且还能满足用户对慢动作精彩瞬间的个性化需求。其中,该慢动作视频段具有较高的录像帧率,以及较低的编码帧率和播放帧率,可以方便用户慢速、仔细地观看和回顾物体的慢动作精彩瞬间。
例如,该电子设备可以是手机、平板电脑、可穿戴设备(例如智能手表)、车载设备、增强现实(augmented reality,AR)/虚拟现实(virtual reality,VR)设备、笔记本电脑、超级移动个人计算机(ultra-mobile personal computer,UMPC)、上网本或个人数字助理(personal digital assistant,PDA)等移动终端,也可以是专业的相机等设备,本申请实施例对电子设备的具体类型不作任何限制。
示例性的,图1示出了电子设备100的一种结构示意图。电子设备100可以包括处理器110,外部存储器接口120,内部存储器121,通用串行总线(universal serial bus,USB)接口130,充电管理模块140,电源管理模块141,电池142,天线1,天线2,移动通信模块150,无线通信模块160,音频模块170,扬声器170A,受话器170B,麦克风170C,耳机接口170D,传感器模块180,按键190,马达191,指示器192,摄像头193,显示屏194,以及用户标识模块(subscriber identification module,SIM)卡接口195等。其中传感器模块180可以包括压力传感器180A,陀螺仪传感器180B,气压传感器180C,磁传感器180D,加速度传感器180E,距离传感器180F,接近光传感器180G,指纹传感器180H,温度传感器180J,触摸传感器180K,环境光传感器180L,骨传导传感器180M等。
处理器110可以包括一个或多个处理单元,例如:处理器110可以包括应用处理器(application processor,AP),调制解调处理器,图形处理器(graphics processing unit,GPU),图像信号处理器(image signal processor,ISP),控制器,存储器,视频编解码器,数字信号处理器(digital signal processor,DSP),基带处理器,和/或神经网络处理器(neural-network processing unit,NPU)等。其中,不同的处理单元可以是独立的器件,也可以集成在一个或多个处理器中。
其中,控制器可以是电子设备100的神经中枢和指挥中心。控制器可以根据指令操作码和时序信号,产生操作控制信号,完成取指令和执行指令的控制。
处理器110中还可以设置存储器,用于存储指令和数据。在一些实施例中,处理器110中的存储器为高速缓冲存储器。该存储器可以保存处理器110刚用过或循环使用的指令或数据。如果处理器110需要再次使用该指令或数据,可从存储器中直接调用。避免了重复存取,减少了处理器110的等待时间,因而提高了系统的效率。
电子设备100通过GPU,显示屏194,以及应用处理器等实现显示功能。GPU为图像处理的微处理器,连接显示屏194和应用处理器。GPU用于执行数学和几何计算,用于图形渲染。处理器110可包括一个或多个GPU,其执行程序指令以生成或改变显示信息。
显示屏194用于显示图像,视频等。显示屏194包括显示面板。显示面板可以采用液晶显示屏(liquid crystal display,LCD),有机发光二极管(organic light-emitting diode,OLED), 有源矩阵有机发光二极体或主动矩阵有机发光二极体(active-matrix organic light emitting diode的,AMOLED),柔性发光二极管(flex light-emitting diode,FLED),Miniled,MicroLed,Micro-oLed,量子点发光二极管(quantum dot light emitting diodes,QLED)等。在一些实施例中,电子设备100可以包括1个或N个显示屏194,N为大于1的正整数。
在本申请的实施例中,显示屏194可以用于在慢动作录像场景下显示预览界面和拍摄界面等界面内容。显示屏194还用于在拍摄界面上显示慢动作录像过程中缓存的高帧率图像帧,以供用户选择慢动作视频段的起始位置和结束位置。
电子设备100可以通过ISP,摄像头193,视频编解码器,GPU,显示屏194以及应用处理器等实现拍摄功能。
ISP用于处理摄像头193反馈的数据。例如,拍照时,打开快门,光线通过镜头被传递到摄像头感光元件上,光信号转换为电信号,摄像头感光元件将电信号传递给ISP处理,转化为肉眼可见的图像。ISP还可以对图像的噪点,亮度,肤色进行算法优化。ISP还可以对拍摄场景的曝光,色温等参数优化。在一些实施例中,ISP可以设置在摄像头193中。
摄像头193用于捕获静态图像或视频。物体通过镜头生成光学图像投射到感光元件。感光元件可以是电荷耦合器件(charge coupled device,CCD)或互补金属氧化物半导体(complementary metal-oxide-semiconductor,CMOS)光电晶体管。感光元件把光信号转换成电信号,之后将电信号传递给ISP转换成数字图像信号。ISP将数字图像信号输出到DSP加工处理。DSP将数字图像信号转换成标准的RGB,YUV等格式的图像信号。在一些实施例中,电子设备100可以包括1个或N个摄像头193,N为大于1的正整数。
在本申请的实施例中,摄像头193可以包括以下一种或多种摄像头:长焦摄像头、广角摄像头、超广角摄像头、变焦摄像头或深度摄像头等。其中,长焦摄像头的拍摄范围小,适用于拍摄远处的景物;广角摄像头的拍摄范围较大;超广角摄像头的拍摄范围大于广角摄像头,适用于拍摄全景等较大画面的景物。深度摄像头可以用于测量待拍摄对象的物距,即测量待拍摄对象的深度信息,例如可以包括三维(3 dimensions,3D)深感摄像头、飞行时间(time of flight,TOF)深度摄像头或双目深度摄像头等。摄像头193可以包括前置摄像头和/或后置摄像头。
在本申请的实施例中,摄像头193可以用于在慢动作录像场景下采集图像。
数字信号处理器用于处理数字信号,除了可以处理数字图像信号,还可以处理其他数字信号。例如,当电子设备100在频点选择时,数字信号处理器用于对频点能量进行傅里叶变换等。
视频编解码器用于对数字视频压缩或解压缩。电子设备100可以支持一种或多种视频编解码器。这样,电子设备100可以播放或录制多种编码格式的视频,例如:动态图像专家组(moving picture experts group,MPEG)1,MPEG2,MPEG3,MPEG4等。
NPU为神经网络(neural-network,NN)计算处理器,通过借鉴生物神经网络结构,例如借鉴人脑神经元之间传递模式,对输入信息快速处理,还可以不断的自学习。通过NPU可以实现电子设备100的智能认知等应用,例如:图像识别,人脸识别,语音识别,文本理解等。
内部存储器121可以用于存储计算机可执行程序代码,可执行程序代码包括指令。处理器110通过运行存储在内部存储器121的指令,从而执行电子设备100的各种功能应用 以及数据处理。内部存储器121可以包括存储程序区和存储数据区。其中,存储程序区可存储操作系统,至少一个功能所需的应用程序(比如声音播放功能,图像播放功能等)等。存储数据区可存储电子设备100使用过程中所创建的数据(比如音频数据,电话本等)等。此外,内部存储器121可以包括高速随机存取存储器,还可以包括非易失性存储器,例如至少一个磁盘存储器件,闪存器件,通用闪存存储器(universal flash storage,UFS),双倍速率同步动态随机存取内存(double data rate synchronous dynamic random-access memory,DDR SDRAM)等。
在本申请的实施例中,内部存储器121可以用于缓存慢动作录像过程中预设时长内的高帧率图像帧。处理器110通过运行存储在内部存储器121的指令,实现在慢动作录像过程中,根据用户精准设置的起始位置和结束位置来生成慢动作视频段,满足用户对慢动作精彩瞬间的个性化需求。
电子设备100可以通过音频模块170,扬声器170A,受话器170B,麦克风170C,耳机接口170D,以及应用处理器等实现音频功能。例如音乐播放,录音等。
触摸传感器180K,也称“触控面板”。触摸传感器180K可以设置于显示屏194,由触摸传感器180K与显示屏194组成触摸屏,也称“触控屏”。触摸传感器180K用于检测作用于其上或附近的触摸操作。触摸传感器可以将检测到的触摸操作传递给应用处理器,以确定触摸事件类型。可以通过显示屏194提供与触摸操作相关的视觉输出。在另一些实施例中,触摸传感器180K也可以设置于电子设备100的表面,与显示屏194所处的位置不同。
可以理解的是,本申请实施例示意的结构并不构成对电子设备100的具体限定。在本申请另一些实施例中,电子设备100可以包括比图示更多或更少的部件,或者组合某些部件,或者拆分某些部件,或者不同的部件布置。图示的部件可以以硬件,软件或软件和硬件的组合实现。
在本申请的实施例中,摄像头193可以用于在慢动作录像场景下采集图像;内部存储器121可以用于缓存慢动作录像过程中预设时长内的高帧率图像帧;显示屏194可以用于在慢动作录像场景下显示预览界面或拍摄界面等,还用于在拍摄界面上显示慢动作录像过程中缓存的高帧率图像帧,以供用户从该高帧率图像帧中选择慢动作视频段的起始位置和结束位置;处理器110通过运行存储在内部存储器121的指令,实现在录像过程中,由用户来精准设置慢动作视频段的起始位置和结束位置,满足用户对慢动作精彩瞬间的个性化需求。
以下将以电子设备为具有图1所示结构的手机为例,对本申请实施例提供的慢动作录像方法进行阐述。
在本申请实施例提供的一种慢动作录像方法中,手机可以首先开启录像功能。而后,手机在检测到用户的录像操作后,若检测到慢动作触发,则在选帧界面上显示待选图像帧,该待选图像帧包括多帧图像。而后,手机可以获取用户基于待选图像帧设置的起始帧和结束帧。然后,手机可以生成目标视频,该目标视频包括慢动作视频段,该慢动作视频段与起始帧和结束帧对应,慢动作视频段的录像帧率高于慢动作视频段的编码帧率。
本申请实施例还提供了另一种慢动作录像方法,参见图2,该方法包括:
201、手机启动录像功能。
在本申请的实施例中,用户想要进行慢动作录像时,可以启动手机的录像功能。
在一些实施例中,手机的录像功能可以有多种,用户想要进行慢动作录像时,手机可以启动慢动作录像功能。
例如,手机可以启动相机应用,或者启动具有拍照或录像功能的其他应用(比如抖音或河图cyberverse等AR应用),进而启动这些应用的慢动作录像功能。
示例性的,手机检测到用户点击图3A中的(a)所示的相机应用的图标301的操作后,启动拍照功能,并显示如图3A中的(b)所示的预览界面。手机检测到用户点击图3A中的(b)所示的控件302的操作后,如图3A中的(c)所示,启动慢动作录像功能。
再示例性的,手机检测到用户点击图3A中的(b)所示的控件303的操作后,显示如图3A中的(d)所示的界面;手机检测到用户点击控件304的操作后,如图3A中的(c)所示,启动慢动作录像功能。
再示例性的,参见图3B,手机检测到用户点击图3B中的(a)所示的控件305的操作后,如图3B中的(b)所示进入普通录像模式。普通录像模式的预览界面上包括慢动作倍速控件306。当用户选择大于1X的慢动作倍速后,手机启动慢动作录像功能。例如,如图3B中的(c)所示,手机检测到用户通过慢动作倍速控件306选择了32X的慢动作倍速后,启动慢动作录像功能。其中,慢动作倍速kX表示慢动作视频段的录像帧率为预设的参考录像帧率的k倍,且k为正整数。例如,参考录像帧率可以为普通录像模式下通常采用的录像帧率,比如30fps或25fps等。慢动作视频段的录像帧率为慢动作视频段的编码帧率。
再示例性的,手机在显示桌面或其他非相机应用界面的情况下,检测到用户进入慢动作录像模式的语音指令后,启动慢动作录像功能。
需要说明的是,手机还可以响应于用户的其他触摸操作、语音指令或快捷手势等操作启动慢动作录像功能,本申请实施例对触发手机启动慢动作录像功能的具体操作不予限定。
在一些实施例中,手机在启动慢动作录像功能后,即采用以下实施例提供的慢动作录像方法录制目标视频。其中,目标视频包括慢动作视频段,该慢动作视频段用于记录慢动作精彩瞬间。
在另一些实施例中,手机在启动慢动作录像功能,且用户设置的慢动作倍速大于或者等于预设倍速1时,才采用以下实施例提供的慢动作录像方法录制目标视频。例如,预设倍速1为8X,当手机确定用户设置的慢动作倍速大于或者等于8X时,才采用以下实施例提供的慢动作录像方法录制目标视频。
在另一些实施例中,手机启动慢动作录像功能后,可以进入慢动作录像模式,从而在慢动作录像模式下采用以下实施例提供的慢动作录像方法录制目标视频。
在其他一些实施例中,手机在启动慢动作录像功能后,可以包括多种慢动作录像子模式,手机在特定的子模式下才采用以下实施例提供的慢动作录像方法录制目标视频。示例性的,参见图4,手机的预览界面上包括普通慢动作控件401和超级慢动作控件402,手机检测到用户点击超级慢动作控件402的操作后,进入超级慢动作子模式,并采用以下实施例提供的慢动作录像方法录制目标视频。
202、手机以帧率f1采集图像帧,并在预览界面上显示预览图像。
手机启动慢动作录像功能后,可以在预览状态下以帧率f1采集图像帧,根据该图像帧获得预览图像,并以帧率f1在预览界面上显示该预览图像。该预览图像可以是摄像头采集 的图像帧经过ISP处理后生成的。例如,ISP处理操作可以包括将图像帧的RAW图转换成RGB图,自动白平衡(automatic white balance,AWB),坏点簇纠正,降噪预滤波,色彩失常校正,或色度噪点降低等处理操作。
其中,帧率f1较低,例如可以为普通录像模式下通常采用的送显和录像帧率,比如为30fps或25fps等。在一种可能的实现方案中,帧率f1等于上述参考录像帧率。
在一些实施例中,参见图5,步骤202所示的过程可以包括:摄像头按照帧率f1(例如30fps)采集图像帧(即以帧率f1出帧),采集的该图像帧经过ISP逐帧处理后,生成预览图像并送显至预览界面上。
在一些实施例中,在上述步骤201之后,该方法还可以包括步骤203:
203、手机获取慢动作倍速。
其中,慢动作倍速表示慢动作视频段的录像帧率(以下简称为慢动作倍速对应的录像帧率)相对于预设的参考录像帧率的倍数,例如可以为32X、64X或256X等,分别表示为预设的参考录像帧率的32倍、64倍或256倍。慢动作倍速对应的录像帧率用于在开始录像后,对用户设置的慢动作的起始位置及结束位置之间的目标图像帧进行编码,从而生成目标视频中的慢动作视频段。该慢动作视频段中的图像帧具有较高的录像帧率,以及较低的编码帧率和播放帧率,可以方便用户慢速、仔细、长时间地观看慢动作精彩瞬间。也就是说,该慢动作视频段用于记录慢动作精彩瞬间。
在一些实施例中,该慢动作倍速为默认的倍速,或者上一次慢动作录像过程中使用的倍速。
在另一些实施例中,该慢动作倍速是手机根据拍摄场景自动确定的。例如,在物体高速运动的拍摄场景下(比如火箭发射或子弹发射等拍摄场景),慢动作倍速可以较大(比如256X或128X等),以便慢动作视频段的录像帧率较大,用于生成慢动作视频段的图像帧较多,使得用户可以通过更多的视频帧,慢速、仔细、长时间地观看慢动作精彩瞬间。相对应地,在物体运动速度较低的拍摄场景下(例如水滴下落或跳跃等拍摄场景),慢动作倍速可以较小(比如64X或32X等)。
在其他一些实施例中,该慢动作倍速为用户设置的倍速。示例性的,参见图6,手机启动慢动作录像功能后,在预览界面上显示多个慢动作倍速控件601。手机根据用户选择的慢动作倍速控件确定用户设置的慢动作倍速。
再示例性的,如图3A中的(b)-(c)所示的,该慢动作倍速可以为用户启动慢动作录像功能时设置的倍速。
再示例性的,手机可以根据用户的语音指令获取用户设置的慢动作倍速。
可以理解的是,用于设置慢动作倍速的方式可以有多种,本申请实施例对该方式不予具体限定。
还可以理解的是,手机还可以响应于用户的操作修改慢动作倍速。
需要说明的是,在本申请的一些实施例中,步骤203为可选步骤。
204、手机检测到用户的录像操作后,以帧率f2采集图像帧,并进行图像缓存。
用户想要开始录制视频时,可以触发录像操作使得手机进入录像过程。示例性的,手机检测到用户点击如图3A中的(c)所示的预览界面上拍摄控件300的操作后,确定检测到用户的录像操作,从而进入录像过程。再示例性的,手机检测到用户语音指示开始录像 的操作后,确定检测到用户的录像操作,从而进入录像过程。
可以理解的是,用于触发手机进入录像过程的方式还可以有手势等其他多种方式,本申请实施例对该方式不予具体限定。
参见图7,手机检测到用户的录像操作后,可以通过摄像头以帧率f2采集图像帧。其中,帧率f2可以较大。在一些实施例中,帧率f2大于帧率f1,即手机在预览状态下的图像采集帧率较低,在录像过程中的图像采集帧率较高。例如,帧率f1可以为30fps,帧率f2可以为1920fps、960fps或240fps等。
在一些实施例中,f2可以为默认的帧率,用户设置的帧率,或者为ISP可支持的最大帧率(即ISP可进行实时图像处理的最大帧率)。
在另一些实施例中,在步骤203中获取了慢动作倍速的情况下,帧率f2可以与慢动作倍速相关。当慢动作倍速小于预设倍速2时,帧率f2为慢动作倍速对应的录像帧率。当慢动作倍速大于或者等于预设倍速2时,帧率f2为预设倍速2对应的帧率。
在一些技术方案中,预设倍速2对应的帧率为ISP可支持的最大帧率。比如,ISP可支持的最大帧率为1920fps,参考录像帧率为30fps,则预设倍速2为1920fps/30fps=64X。也就是说,当慢动作倍速小于或者等于64X时,f2为慢动作倍速对应的录像帧率,即为慢动作倍速*参考录像帧率。示例性的,当慢动作倍速为8X时,帧率f2为8*30fps=240fps;当慢动作倍速为32X时,帧率f2为32*30fps=960fps。当慢动作倍速大于或者等于64X时,帧率f2为ISP可支持的最大帧率1920fps。
如图7所示,手机检测到用户的录像操作后,可以根据帧率f2采集的图像帧进行图像缓存。其中,由于帧率f2较高,ISP可能无法逐帧实时处理,因而手机可以先将帧率为f2的图像帧缓存下来,以便后续使用时再通过ISP对涉及到的图像帧进行处理。
例如,手机检测到用户的录像操作后,可以先实时缓存以高帧率f2采集的图像帧,直至缓存时长等于T0。在缓存时长大于T0后,手机实时缓存当前时刻t0及之前的t1时刻之间T0时长内以高帧率f2采集的图像帧,以便在检测到慢动作触发后,根据缓存的最近T0时长内以帧率f2采集的图像帧生成慢动作视频段。其中,手机缓存的最近T0时长内以帧率f2采集的图像帧,可以称为第一图像帧集合。而对于录像过程中缓存的T0时长之前的图像帧,由于后续不用于生成慢动作视频段,因而手机可以缓存抽帧后获得的帧率为f3的图像帧。
其中,T0时长较短,例如可以为0.5s、1s或1.5s等;f3较小,例如可以为25fps或30fps等,f3与f1可以相等或不等。这样,在开始录像后,手机仅需要缓存录像过程中较短的T0时长内的高帧率f2的图像帧,以及录像过程中在该T0时长之前的抽帧获得的低帧率f3的图像帧,而不用缓存整个录像过程中采集的高帧率f2的图像帧,因而可以减少手机在录像过程中实时缓存的数据量,降低录像过程对手机缓存能力的要求。
在一种可能的实现方案中,手机检测到用户的录像操作后,可以初始化如图8所示的3个DDR缓存队列:缓存队列1、缓存队列2和缓存队列3。其中,缓存队列2的长度为f2*T0。在开始录像后,摄像头以帧率f2(例如1920fps)采集图像帧。摄像头最新采集到的图像帧放入缓存队列2的队头。若缓存队列2已满,即缓存队列2中已缓存f2*T0帧图像,则手机将缓存队列2队尾的图像帧抽帧成帧率为f3的常速图像帧后移到缓存队列1的队头。
205、手机检测到用户的录像操作后,对以帧率f2采集的图像帧进行抽帧后生成录拍图像,并在拍摄界面上显示该录拍图像。
手机检测到用户的录像操作后,如图7所示,对以帧率f2采集的图像帧进行抽帧得到帧率为f3的图像帧,并对该帧率为f3的图像帧进行ISP处理后生成录拍图像,在拍摄界面上显示该录拍图像。此外,参见图7,ISP处理后的图像帧还可以送到慢动作检测模块以检测慢动作是否触发。
在一些实施例中,基于图8所示的缓存方案,缓存队列2中缓存的帧率为f2的图像帧,可以抽帧成帧率为f3的图像帧,而后送给ISP进行处理后生成录拍图像送显。
这样,在开始录像后,手机虽然采用高帧率f2来采集图像帧,但仅对抽帧后获得的低帧率f3的图像帧进行ISP处理和显示,因而可以减小图像处理的计算量,降低录像过程中对手机实时运算能力的要求。
206、手机检测到慢动作触发后,以帧率f2采集T1时长的图像帧并缓存。
手机检测到慢动作触发后,可以确定被拍摄对象当前正处于运动过程,从而可以根据运动过程中的图像帧生成慢动作视频段。
其中,慢动作触发可以是手机自动触发的,也可以是用户主动触发的。
例如,手机采用的自动触发算法可以包括人体动作检测算法或光流动作检测算法等。
其中,人体动作检测算法可以检测被拍摄人体的运动动作。若手机检测到被拍摄人体的运动动作,则该运动动作的持续过程可以进行慢动作处理,从而生成慢动作视频段。
示例性的,运动动作和检测条件的对应关系可以参见表1。手机在满足检测条件后,确定检测到人体的运动动作,从而可以确定触发了慢动作。
表1
运动动作 检测条件
跳跃 原地向上跳跃,有明显下蹲起跳过程
定点投篮 双手抱球高高扬起,准备瞄准投篮
单手挥手 在脖子及以上高度左右挥动手臂
跑步 有明显的摆臂和抬腿,具有一定的跑步速度
抬脚准备射门 区别于走路和足球带球时的踢,幅度较大,腿脚后拉
转身大动作 单次或连续多圈转身,手抬起或举起自然甩动,旋转有一定速度
挥拍 抬手,区别与普通甩手,有较大幅度
劈腿跳跃 区别于普通跳跃,向上跳跃时需有劈腿动作,腿向两边展开
打高尔夫球 大力双手挥杆击球
打乒乓球 大力挥拍打球,有一定幅度
跑步上篮 身体跳跃,单手上篮
欢呼 双手举起挥舞,轻微跳动
鼓掌 双手胸前鼓掌,双臂可能张开
滑板 豚跳时刻,滑板离地或者双脚离开滑板
其中,光流动作检测算法可以通过检测相邻图像帧之间画面变化的剧烈程度,来检测被拍摄物体是否有运动。若手机检测到被拍摄物体有运动,则该运动过程进行慢动作处理从而生成慢动作视频段,因而可以确定触发了慢动作。
可以理解的是,用户主动触发慢动作的方式可以有多种,本申请实施例对该触发方式不予具体限定。示例性的,拍摄界面上显示有慢动作触发控件,手机检测到用户点击该控件的操作后,确定触发了慢动作。再示例性的,手机检测到慢动作触发的用户语音指令后,确定触发了慢动作。
根据上述步骤204中的描述可知,手机在检测到录像操作后,在视频录制过程中实时缓存当前时刻t0及之前的t1时刻之间T0时长内以帧率f2采集的图像帧,以及t1时刻之前的对帧率为f2采集的图像帧抽帧后的帧率为f3的图像帧。因而,手机检测到慢动作触发时,已经缓存有当前时刻t0及之前的t1时刻之间T0时长内以帧率f2采集的图像帧,以及t1时刻之前的对帧率为f2采集的图像帧抽帧后的帧率为f3的图像帧。也就是说,手机缓存有检测到慢动作触发之前的T0时长内以帧率f2采集的图像帧。由于手机检测慢动作触发的过程需要一定的时间,当手机检测到慢动作触发后,慢动作可能已经持续了一段时间,因而手机缓存检测到慢动作触发之前的T0时长内以帧率f2采集的图像帧,可以尽量将慢动作过程完整的记录下来。
并且,手机检测到慢动作触发后,慢动作过程可能还未完成,因而如图9所示,手机可以采用帧率f2继续采集后续T1时长的图像帧并缓存,以尽量将整个慢动作过程完整地记录下来。其中,缓存的检测到慢动作触发之后的T1时长内帧率为f2的图像帧可以称为第二图像帧集合。
在本申请的实施例中,手机缓存的检测到慢动作触发之前的T0时长内帧率为f2的图像帧,以及缓存的检测到慢动作触发之后的T1时长内帧率为f2的图像帧,用于生成慢动作视频段。
在一些实施例中,基于图8所示的缓存方案,手机检测到慢动作触发后,可以将缓存队列2的长度扩展为f2*(T0+T1);手机将检测到慢动作触发后,通过摄像头以帧率f2最新采集到的图像帧放入缓存队列2的队头,直至缓存队列2存满后停止采集图像帧。
在一些实施例中,上述T0和T1的时间长度是手机预设的。
在另一些实施例中,T0和T1的时间长度可以是用户设置的。例如,用户可以根据拍摄场景来设置T0和T1的数值。
在其他一些实施例中,T0和T1的时间长度是手机根据拍摄场景自动确定的。例如,若拍摄场景中运动过程的持续时间较长(比如水滴下落的拍摄场景),则T0和T1可以较长,以便手机可以缓存较多帧率为f2的图像帧。这样,用户从中选择的起始帧和结束帧之间可以包括较多的图像帧,从而使得根据起始帧和结束帧之间的图像帧生成的慢动作视频段中的视频帧较多,使得用户可以通过慢动作视频段中更多的视频帧,慢速、仔细地观看慢动作精彩瞬间。相对应地,若拍摄场景中运动过程的持续时间较短(比如子弹发射的拍摄场景),则T0和T1可以较短。
207、手机根据缓存的图像帧显示待选图像帧,并获取用户设置的起始帧和结束帧。
手机检测到慢动作触发后,可以根据上述第一图像帧集合和第二图像帧集合中的图像帧,在选帧界面上显示待选图像帧,以便如图9所示获取用户基于待选图像帧选择的起始帧和结束帧。其中,起始帧和结束帧之间帧率为f2的图像帧可以为目标图像帧或者用于生成目标图像帧,该目标图像帧用于生成最终拍摄获得的目标视频中的慢动作视频段。该慢动作视频段用于记录用户想要的慢动作精彩瞬间。
其中,手机在选帧界面上显示待选图像帧的方式可以有多种,本申请实施例对该显示方式不予具体限定。
例如,在一些实施例中,待选图像帧为上述第一图像帧集合和第二图像帧集合中的图像帧,以便用户从中选择起始帧和结束帧。
在另一些实施例中,上述第一图像帧集合和第二图像帧集合中帧率为f2的图像帧的数量较多,且相邻图像帧之间的差别较小,不方便用户选择起始帧和结束帧。因而,手机可以对上述第一图像帧集合和第二图像帧集合中的图像帧抽帧后获得待选图像帧。而后,手机再在选帧界面上显示待选图像帧,以方便用户基于数量较少且差别较大的图像帧,快速地选择起始帧和结束帧。
在一些实施例中,慢动作倍速不同,则待选图像帧的数量也不同。例如,慢动作倍速越大,则用户想要拍摄的运动过程的运动速度可能较高,相邻图像帧之间的差别可能较大,因而待选图像帧的数量可以较多,以便用户基于较多数量的待选图像帧,精确地选择高速运动过程对应的起始帧和结束帧。相对应地,慢动作倍速越小,则待选图像帧的数量可以较少。也就是说,待选图像帧的数量与慢动作倍速正相关。举例来说,若慢动作倍速为32X,则待选图像帧可以为30帧;若慢动作倍速为64X,则待选图像帧可以为60帧。
在另一些实施例中,慢动作倍速不同,则手机从缓存的第一图像帧集合和第二图像帧集合的图像帧中,抽帧获得待选图像帧的抽帧间隔也不同。例如,慢动作倍速越大,则第一图像帧集合和第二图像帧集合中的图像帧通常也越多,抽帧间隔也可以相应地越大,以方便用户基于较大抽帧间隔获得的数量较少且差别较大的图像帧,从而快速地选择起始帧和结束帧。相对应地,慢动作倍速越小,则抽帧间隔也可以越小。举例来说,若慢动作倍速为32X,则抽帧间隔可以为16帧;若慢动作倍速为64X,则抽帧间隔可以为32帧。也就是说,用于获得待选图像帧的抽帧间隔,与慢动作倍速正相关。
在另一些实施例中,用户可以设置待选图像帧的数量或抽帧间隔。例如,手机可以在选帧界面上显示多个待选图像帧的帧数控件/抽帧间隔控件,以便用户选择待选图像帧的数量/抽帧间隔。在一些技术方案中,慢动作倍速越大,则手机显示的帧数控件对应的图像帧的数量也越多。示例性的,若慢动作倍速为32X,则手机在选帧界面上显示的帧数控件包括:15帧控件、30帧控件和45帧控件。若慢动作倍速为64X,则手机在选帧界面上显示的帧数控件包括:30帧控件、60帧控件和90帧控件。在另一些技术方案中,慢动作倍速越大,则手机在选帧界面上显示的抽帧间隔控件对应的抽帧间隔也越大。
在另一些实施例中,由于手机的界面的显示面积有限,可显示的图像帧的数量有限,因而手机可以先从第一图像帧集合和第二图像帧集合的图像帧中,以较大的抽帧间隔抽帧获得待选图像帧并显示在选帧界面上,以便用户先粗粒度地选择参考起始帧和参考结束帧。而后,手机还可以从第一图像帧集合和第二图像帧集合的图像帧中,以较小的抽帧间隔抽帧获得参考起始帧及附近的待选图像帧(可以称为第三图像帧集合),和参考结束帧及附近的待选图像帧(可以称为第四图像帧集合)。手机将第三图像帧集合和第四图像帧集合显示在选帧界面上,以便用户细粒度地、精准地选择起始帧和结束帧。
在另一些实施例中,为方便用户选择起始帧和结束帧,手机可以在选帧界面上显示通过算法自动确定的推荐起始帧和推荐结束帧,或者在待选图像帧中标识推荐起始帧和推荐结束帧。而用户想要选择的起始帧和结束帧,通常在推荐起始帧和推荐结束帧。这样,手 机通过在选帧界面上向用户提示推荐起始帧和推荐结束帧,可以方便用户在推荐起始帧附近精准地选择起始帧,并在推荐结束帧附近精准地选择结束帧。
在另一些实施例中,为方便用户选择起始帧和结束帧,手机可以在选帧界面上显示通算法自动确定的推荐起始帧区间和推荐结束帧区间,或者在待选图像帧中标识推荐起始帧区间和推荐结束帧区间。用户想要选择的起始帧和结束帧,通常在推荐起始帧区间和推荐结束帧区间之内或附近。这样,手机通过在选帧界面上向用户提示推荐起始帧区间和推荐结束帧区间,可以方便用户在推荐起始帧区间或附近精准地选择起始帧,并在推荐结束帧区间或附近精准地选择结束帧。
在一些实施例中,由于选帧界面的显示面积有限,因而待选图像帧可以以缩略图的形式进行显示,以便界面上可以同时显示多个图像帧的缩略图,方便用户选择。待选图像帧的缩略图可以显示在选帧界面的任意位置,比如在选帧界面的底部横向排列,或者在选帧界面的右侧竖向排列等。其中,界面上可能仅能同时显示部分待选图像帧的缩略图,不能同时显示所有待选图像帧的缩略图,手机可以响应于用户的滑动操作,在选帧界面上滑动显示其他待选图像帧的缩略图。
并且,用户基于待选图像帧选择起始帧和结束帧的方式也可以有多种,本申请实施例对该方式不予具体限定。
此外,对于录拍图像来说,在一些实施例中,手机检测到慢动作触发后,对后续采集到的帧率为f2的图像帧进行抽帧等处理后,生成帧率为f3的录拍图像,并以帧率f3将录拍图像实时显示在选帧界面的背景上,同时在选帧界面的前景上显示待选图像帧的缩略图。
在另一些实施例中,手机检测到慢动作触发后,将当前显示的录拍图像持续显示在选帧界面的背景上,而不再刷新背景显示的录拍图像,并在选帧界面的前景上显示待选图像帧的缩略图。
在另一些实施例中,手机检测到慢动作触发后,不再在选帧界面上显示录拍图像,而仅在选帧界面上显示待选图像帧的缩略图。
在一些实施例中,手机检测到慢动作触发后,在选帧界面上显示拍摄控件,且拍摄控件与预览界面上拍摄控件的状态一致。
在一些实施例中,手机检测到慢动作触发后,不在选帧界面上显示拍摄控件。
在另一些实施例中,手机在选帧界面上显示录拍图像的过程中,继续在选帧界面上显示拍摄控件。手机在停止采集图像后,不再在选帧界面上显示拍摄控件;或者,手机在选帧界面上显示拍摄控件,且拍摄控件与预览界面上拍摄控件的状态一致。
以下结合附图对用户基于待选图像帧设置起始帧和结束帧的具体方式,进行举例说明。
示例性的,手机检测到用户的录像操作后,显示如图10中的(a)所示的拍摄界面和录拍图像。如图10中的(b)所示,手机可以在拍摄界面上提示用户检测到慢动作触发。而后,如图10中的(c)-(e)所示,手机继续在拍摄界面上刷新显示录拍图像,直至手机停止采集图像帧后停止刷新显示录拍图像。之后,如图10中的(f)所示,手机可以在选帧界面上显示待选图像帧的缩略图1000,并提示用户设置起始帧和结束帧。待选图像帧的缩略图1000可以响应于用户的滑动操作,而滑动切换显示其他待选图像帧的缩略图。比如,响应于用户基于图10中的(f)所示的待选图像帧的缩略图向右滑动的操作,如图10中的(g)所示,待选图像帧的缩略图也向右滑动,从而显示其他待选图像帧的缩略图。 此外,如图10中的(f)所示,选帧界面上还包括用于指示起始帧的第一控件1001和用于指示结束帧的第二控件1002。第一控件1001和第二控件1002可响应于用户的拖动操作,在待选图像帧的缩略图上滑动。当手机检测到用户拖动如图10中的(f)所示的第一控件1001的操作后,可以如图10中的(g)所示,在选帧界面上提示用户正在选择起始帧。参见图10中的(g),手机还可以在选帧界面上提示用户起始帧是否选择完成。手机检测到用户点击“确定”控件的操作后,确定起始帧选择完成。而后,如图10中的(h)所示,手机可以在选帧界面上提示用户请选择结束帧。其中,结束帧的选择方式与起始帧类似,不予赘述。在一些实施例中,手机在检测到用户的录像操作后,还可以在录像过程的界面上显示慢动作倍速,例如可以为图10中的(a)-(h)所示的“256X”。
在其他一些实施例中,用户可以基于第一控件1001和第二控件1002分别选择起始帧和结束帧,手机可以在选帧界面上提示用户“起始帧和结束帧是否选择完成?”手机检测到用户点击“确定”控件的操作后,获取第一控件1001所在位置对应的起始帧,以及第二控件1002所在位置对应的结束帧。
再示例性的,手机在检测到慢动作触发后,若检测到用户针对待选图像帧的缩略图中图像帧1的预设操作1,则确定图像帧1为起始帧。若检测到用户针对待选图像帧的缩略图中图像帧2的预设操作2,则确定图像帧2为结束帧。例如,该预设操作1和预设操作2可以为点击、双击或压力按等操作,本申请实施例对预设操作1和预设操作2的具体形式不予限定。在一些实施例中,参见图11A中的(a),手机在检测到慢动作触发后,在选帧界面上提示用户选择起始帧和结束帧。参见图11A中的(b)-(c),手机检测到用户点击待选图像帧的缩略图中图像帧1和图像帧2后的操作后,确定图像帧1和图像帧2中时序靠前的图像帧为起始帧,且时序靠后的图像帧为结束帧。
再示例性的,手机在检测到慢动作触发后,如图11B中的(a)所示,可以在选帧界面上显示待选图像帧的缩略图1100。待选图像帧的缩略图像上还包括用于选择起始帧和结束帧的选择控件1101。选择控件1101可选中其中一个图像帧。参见图11B中的(b),选择控件1101可响应于用户的拖动操作,在待选图像帧的缩略图上滑动,从而选中不同的图像帧。并且,界面上还包括选择控件1101选中的图像帧的大图1102。当选择控件1101在待选图像帧的缩略图上滑动时,该大图1102可以以动画的形式,滚动显示待选图像帧的缩略图中控件1101所在位置的图像帧和相邻的图像帧。这样,选择控件1101实时对应的图像帧能够在选帧界面上以大图的形式显示,方便用户看清选择控件1101选中的图像帧的具体内容,从而方便用户准确选择起始帧和结束帧。参见图11B中的(a),手机可以先提示用户选择起始帧。如图10中的(b)所示,手机检测到用户点击选择控件1101选中的图像帧后,该图像帧上显示选中标识(例如图中所示的√或文字标识“起始帧”),且该图像帧被确定为起始帧。而后,参见图11B中的(c),手机可以提示用户选择结束帧。如图11B中的(d)所示,结束帧的选择方式与起始帧类似,不予赘述。
在其他一些实施例中,手机在检测到慢动作触发后,可以在选帧界面上以大图形式显示待选图像帧中预设位置的图像帧。例如,该预设位置为待选图像帧的中间位置或界面边界位置等。
再示例性的,上述选择控件可以替换为如图12中的(a)-(e)所示的可拖动的圆点1201或其他形式的控件,本申请实施例对选择控件的具体形式不予限定。并且,手机可以 在选帧界面上显示选择控件选择的图像帧的大图,也可以不显示大图。用户选择起始帧和结束帧的过程与图11B中的(a)-(d)所示的方法类似,不予赘述。并且,在图12所举示例中,手机检测到慢动作触发后,不再在选帧界面的背景上刷新显示新的录拍图像,而在背景上持续显示内存到慢动作触发时的录拍图像,并在选帧界面的前景上显示待选图像帧的缩略图。
再示例性的,手机检测到慢动作触发后,如图13中的(a)所示,可以显示待选图像帧的缩略图,并提示用户选择起始帧和结束帧对应的区间。如图13中的(b)所示,手机检测到用户基于待选图像帧的缩略图的框选操作后,确定框选的图像帧区间内的第一帧图像即为起始帧,框选的图像帧区间内的最后一帧图像即为结束帧。在一些实施例中,若手机未在步骤203中获取慢动作倍速,则如图13中的(a)所示,手机还可以提示用户选择慢动作倍速。手机可以在选帧界面上显示多个倍速控件,以供用户选择。
再示例性的,手机检测到慢动作触发后,如图14中的(a)所示,可以在选帧界面上显示以较大抽帧间隔获得的待选图像帧的缩略图,以便用户粗粒度地选择参考起始帧。手机检测到用户选择参考起始帧的操作后,如图14中的(b)所示,在选帧界面上显示以较小的抽帧间隔获得的参考起始帧附近的待选图像帧(即第三图像帧集合)的缩略图,以便用户细粒度地精准选择起始帧。类似地,如图14中的(c)-(d)所示,手机还可以选择参考结束帧和结束帧,不予赘述。并且,图14所举示例中,手机在检测到慢动作触发后,继续在选帧界面的背景上刷新最新获得的录拍图像,同时在选帧界面的前景上显示待选图像帧的缩略图。
再示例性的,手机检测到慢动作触发后,在选帧界面上显示待选图像帧的缩略图,并标识推荐起始帧和推荐结束帧,以便用户分别在推荐起始帧和推荐结束帧附近快速、精准地选择起始帧和结束帧。比如,如图15所示,待选图像帧包括第五图像帧集合和第六图像帧集合,第五图像帧集合包括如图15中第一行缩略图所示的推荐起始帧及相邻的多帧图像,第六图像帧集合包括如图15中第二行缩略图所示的推荐结束帧及相邻的多帧图像。用户可以基于第五图像帧集合和推荐起始帧设置起始帧,并基于第六图像帧集合和推荐结束帧设置结束帧。
再示例性的,手机检测到慢动作触发后,如图16中的(a)-(b)所示在选帧界面上显示待选图像帧的缩略图,包括推荐起始帧区间和推荐结束帧区间,以便用户分别在推荐起始帧区间和推荐结束帧区间之内或附近快速、精准地选择起始帧和结束帧。
其中,图10-图16是以待显示图像帧的缩略图在选帧界面上横向排列为例进行说明的。示例性的,待显示图像帧的缩略图竖向排列的示意图可以参见图17。
208、手机生成目标视频,该目标视频包括慢动作视频段,该慢动作视频段与起始帧和结束帧相对应。
如图9所示,手机获取到起始帧和结束帧后可以生成目标视频,该目标视频包括慢动作视频段,该慢动作视频段根据起始帧及结束帧之间的目标图像帧得到。
需要说明的是,虽然在步骤207中,用户基于待选图像帧的缩略图来选择目标图像帧的起始帧和结束帧,但目标图像帧为起始帧及结束帧之间帧率为f2的图像帧,或者目标图像帧为起始帧及结束帧之间帧率为f2的图像帧插帧得到的图像帧。其中,目标图像帧的帧率为f4,该f4可以大于或者等于f2。
手机对帧率为f4的目标图像帧进行ISP处理和图像后处理后,根据编码帧率编码生成慢动作视频段。图像后处理操作可以在CPU、GPU或NPU等处理器中进行。例如,图像后处理操作可以包括图像平滑、图像锐化、图像增强、直方图均衡、图像超分辨率、降噪或插帧等。
其中,目标图像帧的帧率f4可以称为慢动作视频段的录像帧率。慢动作视频段的编码帧率可以远小于录像帧率,例如,录像帧率可以为7680fps,编码帧率可以为30fps或25fps等。在一些实施例中,编码帧率等于f3。举例来说,若f2为1920fps,f4为7680fps,编码帧率为30fps,则手机可以将起始帧及结束帧之间帧率为1920fps的图像帧进行插帧,获得帧率为7680fps的目标图像帧;手机采用30fps对帧率为7680fps的目标图像帧进行视频编码,从而生成慢动作视频段。其中,慢动作视频段的录像帧率f4为7680fps。
在一些实施例中,f4为默认的预设帧率,或者上一次对目标图像帧进行编码生成慢动作视频段时采用的编码帧率。
在另一些实施例中,若手机在步骤203中获取了慢动作倍速,则录像帧率f4与获取的慢动作倍速相对应。其中,当慢动作倍速对应的录像帧率与f2相同时,f4与f2相同。当慢动作倍速对应的录像帧率大于f2时,f4大于f2,且f4为慢动作倍速对应的录像帧率。举例来说,若f2为960fps,慢动作倍速为32X,参考录像帧率为30fps,则慢动作倍速对应的录像帧率也为32*30fps=960fps与f2相同,因而f4也为960fps。若f2为960ps,慢动作倍速为256X,参考录像帧率为30fps,则慢动作倍速对应的录像帧率为256*30fps=7680fps大于f2,因而f4为7680fps。
若手机未在步骤203中获取慢动作倍速,则可以在步骤208中获取慢动作倍速,从而获取慢动作倍速对应的录像帧率,该帧率即为f4;或者,手机可以在步骤208中获取目标图像帧的编码帧率f4。
例如,在图8所示的缓存方案中,缓存队列2中起始帧和结束帧之间帧率为f2的目标图像帧80,可以用于插帧生成帧率为f4的目标图像帧,从而进行编码生成慢动作视频段。
在一种可能的实现方案中,上述f1为30fps或25fps,f2为1920fps,慢动作倍速为256X,f3与f1相等,f4为7680fps。
在一些实施例中,参见图18,目标视频还可以包括位于慢动作视频段之前的视频段1,和/或位于慢动作视频段之后的视频段2。在第一图像帧集合和第二图像帧集合中,手机对起始帧之前帧率为f2的图像帧(称为第八图像帧集合)进行抽帧,获得帧率为f3的图像帧并经过ISP处理、图像后处理和编码后生成视频段1。在第一图像帧集合和第二图像帧集合中,手机对结束帧之后的帧率为f2的图像帧(称为第九图像帧)进行抽帧,获得帧率为f3的图像帧并经过ISP处理、图像后处理和编码后生成视频段2(也称第二视频段)。
例如,在图8所示的缓存方案中,手机可以将缓存队列2内采集时刻在用户选择的起始帧之前的图像帧抽帧成帧率为f3的图像帧,并移到缓存队列1的队头得到图像帧集合801。缓存队列1中的图像帧集合801,可以用于编码生成常速视频段1。手机可以将缓存队列2内采集时刻在用户选择的结束帧之后的图像帧抽帧成帧率为f3的图像帧,并移到缓存队列3的队头得到图像帧集合802。缓存队列3中的图像帧集合802,可以用于编码生成常速视频段2。
可以理解的是,若起始帧为缓存的帧率为f2的图像帧中的第一个图像帧,则手机不再 生成视频段1,或者说视频段1为空;若结束帧为缓存的帧率为f2的图像帧中的最后一个图像帧,则手机不再生成视频段2,或者说视频段2为空。
在一些实施例中,参见图18,目标视频还可以包括位于视频段1之前的视频段0。手机对缓存的T0时长之前的帧率为f3的图像帧(称为第七图像帧集合),进行ISP处理和图像后处理之后,以帧率f3进行编码后生成视频段0。例如,在图8所示的缓存方案中,缓存队列1中缓存的帧率为f3的图像帧集合800,可以用于编码生成常速视频段0。其中,视频段0和视频段1可以称为第一视频段。
也就是说,目标视频中各视频段的顺序可以依次为:视频段0、视频段1、慢动作视频段,以及视频段2,即第一视频段、慢动作视频段和第二视频段。
在以上实施例描述的方案中,由于用于生成慢动作视频段的目标图像帧的起始帧和结束帧,是用户在录像过程中精准选择的,即用户精准选择了目标视频中慢动作视频段的起始位置和结束位置,因而可以避免手机自动检测慢动作的起/止时间时发生的误检测,使得生成的慢动作视频段能够准确对应被拍摄的慢动作精彩瞬间,而且还能够满足用户对慢动作精彩瞬间的个性化需求。
后续,手机检测到用户的播放操作后,播放生成的目标视频,包括播放目标视频中的视频段0、视频段1、慢动作视频段和视频段2。其中,目标视频的播放帧率小于f4。其中,该播放帧率通常等于编码帧率,或者与编码帧率相差较小。当编码帧率远小于慢动作视频段的录像帧率时,播放帧率也远小于慢动作视频段的录像帧率。这样,当以播放帧率播放目标视频中以录像帧率f4录制的慢动作视频段,可以将慢动作视频段对应的被拍摄运动过程拉长后慢速播放,以方便用户通过较长的时间仔细、清楚地观看慢动作精彩瞬间。
而视频段0、视频段1和视频段2的录像帧率和编码帧率均为f3,播放帧率等于编码帧率或与编码帧率相差不大,即播放帧率与录像帧率相等或相差不大,因而可以称为常速视频段。
并且,目标视频除了包括慢动作视频段以外,还可以包括慢动作视频段之前的常速视频段,比如视频段0或视频段1等,还可以包括慢动作视频段之后的常速视频,比如视频段2。这样,生成的目标视频可以给用户以从常速视频段到慢速播放的慢动作视频段,再到常速视频段的视觉对比冲击,用户体验较好。
示例性的,以上实施例描述的一种慢动作录像方法的时序图可以参见图19A。手机生成的目标视频播放的视频帧的效果示意图可以参见图19B。
在其他一些实施例中,在上述步骤206之后,该方法还可以包括:手机以帧率f3采集T2时长的图像帧并缓存。手机生成的目标视频还包括,对缓存的帧率为f3的T2时长的图像帧进行ISP处理和图像后处理,并根据编码帧率编码后获得的视频段3。
在以上实施例描述的方案中,手机自动停止采集图像帧。在其他一些实施例中,在手机检测到慢动作触发后,该方法还可以包括:手机以帧率f2采集图像帧并缓存,直至检测到用户的停止录像操作后,才停止采集图像帧。
对于慢动作录像来说,一方面,为了获得更好的慢动作效果,慢动作视频的编码帧率越来越高,相应地录像过程中的采集帧率f2也越来越高。另一方面,在拍摄高速运动过程时,运动过程的持续时间较短,为了更好地看清短时间内的运动细节,慢动作视频的编码帧率较高,相应地录像过程中的采集帧率f2也较高。在本申请实施例提供的慢动作录像方 法中,手机并不缓存整个录像过程中以f2采集到的高帧率图像帧,而仅缓存T0时长和T1时长内的高帧率图像帧,并对高帧率图像帧抽帧后进行ISP处理和送显,因而可以减少对手机图像缓存传输及运算能力的要求。
此外,手机仅对用户选择的起始帧和结束帧之间的高帧率图像帧进行高帧率编码,而对录像过程中的其他低帧率图像帧进行低帧率编码,可以减小图像处理任务,降低对手机运算能力的要求。
而且,在本申请实施例提供的慢动作录像方法中,用户在录像过程中即可选择起始帧和结束帧从而生成目标视频中的慢动作视频段,而不需要在生成视频后再进行二次编辑从而获得慢动作视频段。先生成视频后再编辑获得慢动作视频段的方案,要求ISP能够在录像过程中对采集到的图像帧进行实时处理,进而进行编码后生成视频,而ISP的处理能力有限,难以实现对较高采集帧率的图像帧的实时处理。
在以上实施例中,手机在开始录像后才缓存T0时长的帧率为f2的图像帧,而在预览状态下不进行图像缓存。在其他一些实施例中,手机在预览状态下即缓存T0时长的帧率为f2的图像帧。
例如,在一种可能的实现方式中,手机启动慢动作录像功能后,在预览状态下以帧率f2采集图像帧,并抽帧成帧率为f3的图像帧后送显。并且,手机先实时缓存以高帧率f2采集的图像帧,直至缓存时长等于T0。在缓存时长大于T0后,手机实时缓存当前时刻t0及之前的t1时刻之间T0时长内以高帧率f2采集的图像帧,并将T0时长之前的帧率为f2的图像帧抽帧进行抽帧,获得帧率为f3的图像帧后进行缓存。手机检测到用户的录像操作后,接续之前的缓存状态继续缓存图像帧。也就是说,手机检测到用户的录像操作后,接续预览状态下的缓存进度,继续缓存当前时刻t0及之前的t1时刻之间T0时长内以帧率f2采集的图像帧,以及t1时刻之前的对帧率为f2采集的图像帧抽帧后的帧率为f3的图像帧。而后,手机可以执行以上实施例中的步骤205-208。
在一些情况下,手机可能在检测到用户的录像操作后很快就检测到慢动作触发,例如检测到慢动作触发的时刻与检测到录像操作的时刻之间的时差小于T0。此时,若手机仅在检测到用户的录像操作后才开始缓存图像帧,则缓存的高帧率f2的图像帧中可能不能包括完整的发生在慢动作触发之前的慢动作过程。
若手机在预览状态下即缓存高帧率f2的图像帧,而在检测到用户的录像操作后接续预览状态下的缓存状态继续缓存图像帧,则能够使得缓存的高帧率f2的图像帧尽量包括完整的发生在慢动作触发之前的慢动作过程。这样,手机在检测到录像操作后又立即检测到慢动作触发的情况下,也能根据缓存的图像帧生成完整的慢动作视频段。
以上是以电子设备为手机为例进行举例说明的,当电子设备为平板电脑或相机等其他设备时,仍可以采用以上实施例提供的慢动作录像方法,在录像过程中由用户来精准设置慢动作视频段的起始位置和结束位置,使得生成的慢动作视频段能够准确对应被拍摄的慢动作精彩瞬间,还能够满足用户对慢动作精彩瞬间的个性化需求,这里不予赘述。
可以理解的是,为了实现上述功能,电子设备包含了执行各个功能相应的硬件和/或软件模块。结合本文中所公开的实施例描述的各示例的算法步骤,本申请能够以硬件或硬件和计算机软件的结合形式来实现。某个功能究竟以硬件还是计算机软件驱动硬件的方式来执行,取决于技术方案的特定应用和设计约束条件。本领域技术人员可以结合实施例对每 个特定的应用来使用不同方法来实现所描述的功能,但是这种实现不应认为超出本申请的范围。
本实施例可以根据上述方法示例对电子设备进行功能模块的划分,例如,可以对应各个功能划分各个功能模块,也可以将两个或两个以上的功能集成在一个处理模块中。上述集成的模块可以采用硬件的形式实现。需要说明的是,本实施例中对模块的划分是示意性的,仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式。
本申请实施例还提供一种电子设备,包括一个或多个处理器以及一个或多个存储器。该一个或多个存储器与一个或多个处理器耦合,一个或多个存储器用于存储计算机程序代码,计算机程序代码包括计算机指令,当一个或多个处理器执行计算机指令时,使得电子设备执行上述相关方法步骤实现上述实施例中的慢动作录像方法。
本申请实施例还提供一种电子设备,如图20所示,包括:显示屏2001,一个或多个处理器2002,多个摄像头2003,存储器2004,以及一个或多个计算机程序2005,上述各器件可以通过一个或多个通信总线2006连接。其中该一个或多个计算机程序2005被存储在上述存储器2004中并被配置为被该一个或多个处理器2002执行,该一个或多个计算机程序2005包括指令,上述指令可以用于执行上述实施例中的各个步骤。其中,上述方法实施例涉及的各步骤的所有相关内容均可以援引到对应实体器件的功能描述,在此不再赘述。
示例性的,上述处理器2002具体可以为图1所示的处理器110,上述存储器2004具体可以为图1所示的内部存储器121,上述摄像头2003具体可以为图1所示的摄像头193,上述显示屏2001具体可以为图1所示的显示屏194。
本申请的实施例还提供一种计算机可读存储介质,该计算机可读存储介质中存储有计算机指令,当该计算机指令在电子设备上运行时,使得电子设备执行上述相关方法步骤实现上述实施例中的慢动作录像方法。
本申请的实施例还提供了一种计算机程序产品,当该计算机程序产品在计算机上运行时,使得计算机执行上述相关步骤,以实现上述实施例中电子设备执行的慢动作录像方法。
另外,本申请的实施例还提供一种装置,这个装置具体可以是芯片,组件或模块,该装置可包括相连的处理器和存储器;其中,存储器用于存储计算机执行指令,当装置运行时,处理器可执行存储器存储的计算机执行指令,以使芯片执行上述各方法实施例中电子设备执行的慢动作录像方法。
其中,本实施例提供的电子设备、计算机可读存储介质、计算机程序产品或芯片均用于执行上文所提供的对应的方法,因此,其所能达到的有益效果可参考上文所提供的对应的方法中的有益效果,此处不再赘述。
通过以上实施方式的描述,所属领域的技术人员可以了解到,为描述的方便和简洁,仅以上述各功能模块的划分进行举例说明,实际应用中,可以根据需要而将上述功能分配由不同的功能模块完成,即将装置的内部结构划分成不同的功能模块,以完成以上描述的全部或者部分功能。
在本申请所提供的几个实施例中,应该理解到,所揭露的装置和方法,可以通过其它的方式实现。例如,以上所描述的装置实施例仅仅是示意性的,例如,所述模块或单元的划分,仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式,例如多个单元或组 件可以结合或者可以集成到另一个装置,或一些特征可以忽略,或不执行。另一点,所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口,装置或单元的间接耦合或通信连接,可以是电性,机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的,作为单元显示的部件可以是一个物理单元或多个物理单元,即可以位于一个地方,或者也可以分布到多个不同地方。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外,在本申请各个实施例中的各功能单元可以集成在一个处理单元中,也可以是各个单元单独物理存在,也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现,也可以采用软件功能单元的形式实现。
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时,可以存储在一个可读取存储介质中。基于这样的理解,本申请实施例的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来,该软件产品存储在一个存储介质中,包括若干指令用以使得一个设备(可以是单片机,芯片等)或处理器(processor)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括:U盘、移动硬盘、只读存储器(read only memory,ROM)、随机存取存储器(random access memory,RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上内容,仅为本申请的具体实施方式,但本申请的保护范围并不局限于此,任何熟悉本技术领域的技术人员在本申请揭露的技术范围内,可轻易想到变化或替换,都应涵盖在本申请的保护范围之内。因此,本申请的保护范围应以所述权利要求的保护范围为准。

Claims (21)

  1. 一种录像方法,应用于电子设备,其特征在于,包括:
    开启录像功能;
    在检测到用户的录像操作后,若检测到慢动作触发,则在选帧界面上显示待选图像帧,所述待选图像帧包括多帧图像;
    获取所述用户基于所述待选图像帧设置的起始帧和结束帧;
    生成目标视频,所述目标视频包括慢动作视频段,所述慢动作视频段与所述起始帧和所述结束帧相对应,所述慢动作视频段的录像帧率高于所述慢动作视频段的编码帧率。
  2. 根据权利要求1所述的方法,其特征在于,所述方法还包括:
    在检测到所述用户的录像操作后,以第一帧率采集图像帧,并缓存第一图像帧集合,所述第一图像帧集合包括最近采集到的T0时长内的图像帧;
    在检测到慢动作触发后,以所述第一帧率采集T1时长的图像帧,并缓存第二图像帧集合,所述第二图像帧集合包括以所述第一帧率采集的所述T1时长的图像帧;
    所述待选图像帧来自于所述第一图像帧集合和所述第二图像帧集合。
  3. 根据权利要求1或2所述的方法,其特征在于,所述待选图像帧通过所述第一图像帧集合和所述第二图像帧集合中的图像帧抽帧获得。
  4. 根据权利要求3所述的方法,其特征在于,所述待选图像帧的数量与所述慢动作视频段的慢动作倍速正相关;
    或者,从所述第一图像帧集合和所述第二图像帧集合中的图像帧,抽帧获得所述待选图像帧的抽帧间隔,与所述慢动作视频段的慢动作倍速正相关。
  5. 根据权利要求2-4任一项所述的方法,其特征在于,所述慢动作视频段采用所述编码帧率,对所述起始帧及所述结束帧之间的目标图像帧获得;其中,
    所述目标图像帧为所述第一图像帧集合和所述第二图像帧集合中,所述起始帧及所述结束帧之间的图像帧,所述录像帧率等于所述第一帧率;
    或者,所述目标图像帧为所述第一图像帧集合和所述第二图像帧集合中,所述起始帧及所述结束帧之间第一帧率的图像帧,插帧后获得的所述录像帧率的图像帧。
  6. 根据权利要求2-5任一项所述的方法,其特征在于,所述方法还包括:
    在检测到用户的录像操作后,在所述拍摄界面上以第二帧率显示录拍图像,所述录拍图像根据所述第一帧率的图像帧抽帧后获得,所述第二帧率小于所述第一帧率。
  7. 根据权利要求1-6任一项所述的方法,其特征在于,所述录像帧率与所述慢动作视频段的慢动作倍速相对应。
  8. 根据权利要求2-7任一项所述的方法,其特征在于,所述第一帧率与所述慢动作视频段的慢动作倍速相关。
  9. 根据权利要求1-6任一项所述的方法,其特征在于,所述方法还包括:
    若检测到慢动作触发,则在所述选帧界面上提示用户设置所述起始帧和所述结束帧。
  10. 根据权利要求1-9任一项所述的方法,其特征在于,所述确定所述用户基于所述待选图像帧设置的起始帧和结束帧,包括:
    若检测到所述用户针对所述待选图像帧上第一图像帧的第一预设操作,则确定所述第一图像帧为所述起始帧;
    若检测到所述用户针对所述待选图像帧上第二图像帧的第二预设操作,则确定所述第二图像帧为所述结束帧。
  11. 根据权利要求10所述的方法,其特征在于,所述选帧界面上以大图形式显示有所述待选图像帧中位于预设位置的图像帧,所述预设位置为所述待选图像帧的中间位置或所述选帧界面的边界位置。
  12. 根据权利要求1-9任一项所述的方法,其特征在于,所述方法还包括:
    若检测到慢动作触发,则在选帧界面上显示第一控件和第二控件,所述第一控件和所述第二控件可在所述待选图像帧上滑动,所述第一控件用于设置所述起始帧,所述第二控件用于设置所述结束帧;所述选帧界面上以大图形式显示所述第一控件或所述第二控件所在位置的图像帧;
    当所述用户在所述待选图像帧上拖动所述第一控件时,所述界面上以大图形式显示所述第一控件所在位置的图像帧;
    当所述用户在所述待选图像帧上拖动所述第二控件时,所述界面上以大图形式显示所述第二控件所在位置的图像帧;
    所述确定所述用户基于所述待选图像帧设置的起始帧和结束帧,包括:
    确定所述待选图像帧中,所述第一控件所在位置的图像帧为所述起始帧;
    确定所述待选图像帧中,所述第二控件所在位置的图像帧为所述结束帧。
  13. 根据权利要求3-9任一项所述的方法,其特征在于,所述待选图像帧采用第一间隔从所述第一图像帧集合和所述第二图像帧集合的图像帧中抽帧获得;
    所述确定所述用户基于所述待选图像帧设置的起始帧和结束帧,包括:
    确定所述用户基于所述待选图像帧设置的参考起始帧;
    在所述选帧界面上显示第三图像帧集合,所述第三图像帧集合包括所述参考起始帧,且所述第三图像帧集合采用第二间隔从所述第一图像帧集合和所述第二图像帧集合的图像帧中抽帧获得,所述第二间隔小于所述第一间隔;
    确定所述用户基于所述第三图像帧集合设置的所述起始帧;
    确定所述用户基于所述待选图像帧设置的参考结束帧;
    在所述界面上显示第四图像帧集合,所述第四图像帧集合包括所述参考结束帧,且所述第四图像帧集合采用所述第二间隔从所述第一图像帧集合和所述第二图像帧集合的图像帧中抽帧获得;
    确定所述用户基于所述第四图像帧集合设置的所述结束帧。
  14. 根据权利要求1-9任一项所述的方法,其特征在于,所述待选图像帧包括第五图像帧集合和第六图像帧集合,所述第五图像帧集合包括推荐起始帧及相邻的多帧图像,所述第六图像帧集合包括推荐结束帧及相邻的多帧图像;
    所述确定所述用户基于所述待选图像帧设置的起始帧和结束帧,包括:
    确定所述用户基于所述第五图像帧集合和所述推荐起始帧设置的所述起始帧;
    确定所述用户基于所述第六图像帧集合和所述推荐结束帧设置的所述结束帧。
  15. 根据权利要求1-9任一项所述的方法,其特征在于,所述待选图像帧包括推荐起始帧区间和推荐结束帧区间;
    所述确定所述用户基于所述待选图像帧设置的起始帧和结束帧,包括:
    确定所述用户基于所述推荐起始帧区间设置的所述起始帧;
    确定所述用户基于所述推荐结束帧区间设置的所述结束帧。
  16. 根据权利要求1-15任一项所述的方法,其特征在于,所述方法还包括:
    在检测到所述用户的录像操作后,缓存第七图像帧集合,所述第七图像帧集合包括所述T0时长之前的第一帧率的图像帧抽帧获得的第二帧率的图像帧;
    所述目标视频还包括第一视频段,所述第一视频段采用所述编码帧率对所述第七图像帧集合和第八图像帧集合中的图像帧编码获得;
    其中,所述第八图像帧集合为所述第一图像帧集合和所述第二图像帧集合中,所述起始帧之前的第一帧率的图像帧,抽帧获得的所述第二帧率的图像帧。
  17. 根据权利要求1-16任一项所述的方法,其特征在于,所述目标视频还包括第二视频段,所述第二视频段采用所述编码帧率对第九图像帧集合中的图像帧编码获得;
    其中,所述第九图像帧集合为所述第一图像帧集合和所述第二图像帧集合中,所述结束帧之后的第一帧率的图像帧,抽帧获得的所述第二帧率的图像帧。
  18. 根据权利要求1-17任一项所述的方法,其特征在于,所述方法还包括:
    在开启录像功能后,以第三帧率采集图像,并在预览界面上以所述第三帧率显示预览图像,所述第三帧率小于所述第一帧率。
  19. 一种电子设备,其特征在于,包括:
    摄像头,用于采集图像;
    显示屏,用于显示界面;
    一个或多个处理器和一个或多个存储器;
    所述一个或多个存储器与所述一个或多个处理器耦合,所述一个或多个存储器用于存储计算机程序代码,所述计算机程序代码包括计算机指令,当所述一个或多个处理器执行所述计算机指令时,所述电子设备执行如权利要求1-18中任一项所述的录像方法。
  20. 一种计算机存储介质,其特征在于,包括计算机指令,当所述计算机指令在电子设备上运行时,使得所述电子设备执行如权利要求1-18中任一项所述的录像方法。
  21. 一种计算机程序产品,其特征在于,当所述计算机程序产品在计算机上运行时,使得所述计算机执行如权利要求1-18中任一项所述的录像方法。
PCT/CN2021/130849 2020-11-18 2021-11-16 一种慢动作录像方法及设备 WO2022105734A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US18/253,388 US20240007588A1 (en) 2020-11-18 2021-11-16 Slow-motion video recording method and device
EP21893881.9A EP4236300A4 (en) 2020-11-18 2021-11-16 METHOD AND DEVICE FOR SLOW-MOTION VIDEO RECORDING

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011296546.8A CN114520886A (zh) 2020-11-18 2020-11-18 一种慢动作录像方法及设备
CN202011296546.8 2020-11-18

Publications (1)

Publication Number Publication Date
WO2022105734A1 true WO2022105734A1 (zh) 2022-05-27

Family

ID=81594168

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/130849 WO2022105734A1 (zh) 2020-11-18 2021-11-16 一种慢动作录像方法及设备

Country Status (4)

Country Link
US (1) US20240007588A1 (zh)
EP (1) EP4236300A4 (zh)
CN (1) CN114520886A (zh)
WO (1) WO2022105734A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115022679A (zh) * 2022-05-30 2022-09-06 北京百度网讯科技有限公司 视频处理方法、装置、电子设备和介质
CN117479000A (zh) * 2022-08-08 2024-01-30 荣耀终端有限公司 录像方法和相关装置

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115225826A (zh) * 2022-06-30 2022-10-21 联想(北京)有限公司 图像拍摄方法、装置、设备和存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107666581A (zh) * 2016-07-29 2018-02-06 三星电子株式会社 提供视频内容的方法和支持该方法的电子装置
CN110086905A (zh) * 2018-03-26 2019-08-02 华为技术有限公司 一种录像方法及电子设备
CN110506415A (zh) * 2018-03-26 2019-11-26 华为技术有限公司 一种录像方法及电子设备
CN111010521A (zh) * 2018-10-04 2020-04-14 三星电子株式会社 在便携式电子设备中记录超慢动作视频的方法和系统
US20200244884A1 (en) * 2019-01-25 2020-07-30 Samsung Electronics Co., Ltd. Apparatus and method for producing slow motion video

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107251551B (zh) * 2015-02-10 2020-10-13 索尼半导体解决方案公司 图像处理设备、图像捕获装置、图像处理方法和存储介质
CN112805996B (zh) * 2018-10-08 2022-04-22 华为技术有限公司 一种用于生成慢动作视频片段的设备和方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107666581A (zh) * 2016-07-29 2018-02-06 三星电子株式会社 提供视频内容的方法和支持该方法的电子装置
CN110086905A (zh) * 2018-03-26 2019-08-02 华为技术有限公司 一种录像方法及电子设备
CN110506415A (zh) * 2018-03-26 2019-11-26 华为技术有限公司 一种录像方法及电子设备
CN111010521A (zh) * 2018-10-04 2020-04-14 三星电子株式会社 在便携式电子设备中记录超慢动作视频的方法和系统
US20200244884A1 (en) * 2019-01-25 2020-07-30 Samsung Electronics Co., Ltd. Apparatus and method for producing slow motion video

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4236300A4

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115022679A (zh) * 2022-05-30 2022-09-06 北京百度网讯科技有限公司 视频处理方法、装置、电子设备和介质
CN115022679B (zh) * 2022-05-30 2023-08-29 北京百度网讯科技有限公司 视频处理方法、装置、电子设备和介质
CN117479000A (zh) * 2022-08-08 2024-01-30 荣耀终端有限公司 录像方法和相关装置

Also Published As

Publication number Publication date
EP4236300A4 (en) 2024-04-17
CN114520886A (zh) 2022-05-20
EP4236300A1 (en) 2023-08-30
US20240007588A1 (en) 2024-01-04

Similar Documents

Publication Publication Date Title
WO2022105734A1 (zh) 一种慢动作录像方法及设备
JP7371227B2 (ja) インテリジェントビデオ録画方法及び装置
US20200227089A1 (en) Method and device for processing multimedia information
WO2022062318A1 (zh) 一种拍摄方法及设备
WO2021052414A1 (zh) 慢动作视频拍摄方法及电子设备
US9088721B2 (en) Imaging apparatus and display control method thereof
WO2021223500A1 (zh) 一种拍摄方法及设备
CN113099146B (zh) 一种视频生成方法、装置及相关设备
WO2019183784A1 (zh) 一种录像方法及电子设备
WO2021244104A1 (zh) 一种延时摄影的拍摄方法及设备
CN114339102B (zh) 一种录像方法及设备
WO2021043061A1 (zh) 图像处理方法、装置、存储介质及电子设备
CN115176455A (zh) 功率高效的动态电子图像稳定
WO2022252649A1 (zh) 一种视频的处理方法及电子设备
EP4109879A1 (en) Image color retention method and device
CN114339101B (zh) 一种录像方法及设备
CN117278839A (zh) 一种拍摄方法、电子设备及存储介质
RU2818577C1 (ru) Способ и устройство видеозаписи с замедленной съемкой
WO2022228259A1 (zh) 一种目标追踪方法及相关装置
EP4258130A1 (en) Method and apparatus for viewing multimedia content
CN114697525B (zh) 一种确定跟踪目标的方法及电子设备
WO2024114569A1 (zh) 一种视频处理的方法以及电子设备
US11601588B2 (en) Take-off capture method and electronic device, and storage medium
KR20220080063A (ko) 골프 스윙 촬영 방법 및 장치
CN117119284A (zh) 一种拍摄方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21893881

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 18253388

Country of ref document: US

ENP Entry into the national phase

Ref document number: 2021893881

Country of ref document: EP

Effective date: 20230523

REG Reference to national code

Ref country code: BR

Ref legal event code: B01A

Ref document number: 112023009577

Country of ref document: BR

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 112023009577

Country of ref document: BR

Kind code of ref document: A2

Effective date: 20230518