WO2022105734A1 - 一种慢动作录像方法及设备 - Google Patents
一种慢动作录像方法及设备 Download PDFInfo
- Publication number
- WO2022105734A1 WO2022105734A1 PCT/CN2021/130849 CN2021130849W WO2022105734A1 WO 2022105734 A1 WO2022105734 A1 WO 2022105734A1 CN 2021130849 W CN2021130849 W CN 2021130849W WO 2022105734 A1 WO2022105734 A1 WO 2022105734A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- frame
- image
- slow
- image frame
- motion
- Prior art date
Links
- 230000033001 locomotion Effects 0.000 title claims abstract description 167
- 238000000034 method Methods 0.000 title claims abstract description 155
- 230000006870 function Effects 0.000 claims description 47
- 230000015654 memory Effects 0.000 claims description 39
- 238000001454 recorded image Methods 0.000 claims description 19
- 238000005070 sampling Methods 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 16
- 238000003860 storage Methods 0.000 claims description 12
- 238000000605 extraction Methods 0.000 claims description 11
- 230000003139 buffering effect Effects 0.000 claims description 9
- 230000002596 correlated effect Effects 0.000 claims description 3
- 238000003780 insertion Methods 0.000 claims description 2
- 230000037431 insertion Effects 0.000 claims description 2
- 230000008569 process Effects 0.000 abstract description 68
- 238000012545 processing Methods 0.000 description 30
- 230000000875 corresponding effect Effects 0.000 description 28
- 238000010586 diagram Methods 0.000 description 26
- 239000000872 buffer Substances 0.000 description 15
- 230000001960 triggered effect Effects 0.000 description 13
- 238000004422 calculation algorithm Methods 0.000 description 12
- 238000001514 detection method Methods 0.000 description 12
- 238000013461 design Methods 0.000 description 10
- 239000000284 extract Substances 0.000 description 10
- 238000012805 post-processing Methods 0.000 description 7
- 230000003287 optical effect Effects 0.000 description 6
- 230000004044 response Effects 0.000 description 6
- 230000009471 action Effects 0.000 description 5
- 230000009286 beneficial effect Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 4
- 229920001621 AMOLED Polymers 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000009467 reduction Effects 0.000 description 3
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000009191 jumping Effects 0.000 description 2
- 239000002096 quantum dot Substances 0.000 description 2
- 238000010187 selection method Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 241001481833 Coryphaena hippurus Species 0.000 description 1
- 206010028347 Muscle twitching Diseases 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000013529 biological neural network Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 210000000988 bone and bone Anatomy 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 230000019771 cognition Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000010924 continuous production Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000003707 image sharpening Methods 0.000 description 1
- 238000003706 image smoothing Methods 0.000 description 1
- 238000010348 incorporation Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000003825 pressing Methods 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
- H04N5/78—Television signal recording using magnetic recording
- H04N5/782—Television signal recording using magnetic recording on tape
- H04N5/783—Adaptations for reproducing at a rate different from the recording rate
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
- G11B27/005—Reproducing at a different information rate from the information rate of recording
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
- G11B27/02—Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
- G11B27/031—Electronic editing of digitised analogue information signals, e.g. audio or video signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/63—Control of cameras or camera modules by using electronic viewfinders
- H04N23/631—Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
- H04N5/765—Interface circuits between an apparatus for recording and another apparatus
- H04N5/77—Interface circuits between an apparatus for recording and another apparatus between a recording apparatus and a television camera
- H04N5/772—Interface circuits between an apparatus for recording and another apparatus between a recording apparatus and a television camera the recording apparatus and the television camera being placed in the same enclosure
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
- H04N5/91—Television signal processing therefor
Definitions
- the present application relates to the field of electronic technology, and in particular, to a slow-motion video recording method and device.
- slow-motion video can capture slow-motion details of objects in high-speed motion, and record fleeting wonderful moments.
- the electronic device determines the start/end time of the motion process, and generates a slow-motion video segment according to the start/end time.
- the slow-motion video segment can facilitate the user to watch slow-motion wonderful moments at a slow speed.
- the start/end time of the slow motion determined by the electronic device may be inaccurate, and the start position and end position of the generated slow motion video segment usually deviate from the user's expectation, which is difficult to meet the user's personalization of the wonderful slow motion moments. need.
- the embodiments of the present application provide a slow-motion video recording method and device, which can accurately set the start position and end position of the slow-motion video segment by the user during the recording process, so that the generated slow-motion video segment can accurately correspond to the slow-motion wonderful moments , and can also meet the personalized needs of users for perfect slow-motion moments.
- an embodiment of the present application provides a video recording method, which is applied to an electronic device.
- the method includes: the electronic device enables a video recording function. After detecting the user's video recording operation, the electronic device will display the image frame to be selected on the frame selection interface if a slow motion trigger is detected, and the image frame to be selected includes multiple frames of images.
- the electronic device acquires the start frame and the end frame set by the user based on the image frame to be selected.
- the electronic device generates a target video, the target video includes a slow-motion video segment, the slow-motion video segment corresponds to the start frame and the end frame, and the recording frame rate of the slow-motion video segment is higher than the encoding frame rate of the slow-motion video segment.
- the user can accurately select the start frame and the end frame based on the image frames to be selected displayed by the electronic device during the recording process, so that the electronic device can generate the slow motion video in the target video according to the start frame and the end frame. segment, so that the generated slow-motion video segment can accurately correspond to the wonderful slow-motion moments, and can also meet the user's personalized needs for the wonderful slow-motion moments.
- the recording frame rate of the slow-motion video segment is higher than the encoding frame rate of the slow-motion video segment, so that the user can watch the slow-motion wonderful process through the slow-motion video segment slowly, carefully and for a long time.
- the method further includes: after the electronic device detects the user's video recording operation, collecting image frames at a first frame rate, and buffering a first set of image frames, where the first set of image frames includes the most recently collected image frames The image frame within the T0 duration. After detecting the slow motion trigger, the electronic device collects image frames of duration T1 at the first frame rate, and buffers a second set of image frames, where the second set of image frames includes image frames of duration T1 collected at the first frame rate.
- the image frames to be selected come from the first image frame set and the second image frame set.
- the electronic device can cache the image frames collected at the first frame rate during T0 before the slow motion is triggered, and the image frames collected at the first frame rate during T1 after the slow motion is triggered.
- the image frame of the frame rate displays the candidate image frame, so that the user can select the start frame and the end frame based on the candidate image frame.
- the image frame to be selected is obtained by extracting image frames from the first image frame set and the second image frame set.
- the number of the image frames to be selected is small, and it is convenient for the user to select the start frame and the end frame therefrom.
- the number of image frames to be selected is positively correlated with the slow-motion double-speed of the slow-motion video segment;
- the frame sampling interval of the image frame to be selected is positively related to the slow-motion multiplier of the slow-motion video segment.
- the faster-motion multiplier the greater the number of image frames to be selected; the smaller the slow-motion multiplier, the less the number of frames to be selected.
- the larger the slow-motion multiplier the larger the frame sampling interval used to extract frames to obtain the candidate image frame; the smaller the slow-motion multiplier, the smaller the frame sampling interval used to extract the frame to obtain the candidate image frame.
- the slow motion video segment is obtained from the target image frame between the start frame and the end frame by using the encoding frame rate.
- the target image frame is the image frame between the start frame and the end frame in the first image frame set and the second image frame set, and the video recording frame rate is equal to the first frame rate.
- the target image frame is an image frame at the first frame rate between the start frame and the end frame in the first image frame set and the second image frame set, and an image frame at the video recording frame rate obtained after frame insertion.
- the target image frame is obtained according to the start frame and the end frame, and the target image frame is an image frame with a high frame rate.
- the slow-motion video segment adopts a low encoding frame rate, and the target image frame with a high frame rate is obtained by video encoding, so that the user can watch the slow-motion wonderful process through the slow-motion video segment slowly, carefully and for a long time.
- the method further includes: after the electronic device detects the user's video recording operation, displaying the recorded image at the second frame rate on the shooting interface, and the recorded image is based on the image at the first frame rate Obtained after frame extraction, the second frame rate is smaller than the first frame rate.
- the electronic device captures image frames at a high frame rate and displays the recorded image at a low frame rate, which can reduce the load of image processing and image display and save the power consumption of the electronic device.
- the recording frame rate corresponds to the slow-motion double speed of the slow-motion video segment.
- the electronic device can determine the size of the recording frame rate according to the slow-motion double speed.
- the first frame rate is related to the slow-motion double-speed of the slow-motion video segment.
- the frame rate used by the electronic device to capture the image frames can be determined according to the slow-motion multiplier.
- the method further includes: if the electronic device detects the slow motion trigger, prompting the user to set the start frame and the end frame on the frame selection interface.
- the user can select the start frame and the end frame according to the intuitive prompt of the electronic device.
- the electronic device determines the start frame and the end frame set by the user based on the image frame to be selected, including: if the electronic device detects the user's first preset for the first image frame on the image frame to be selected Set operation, then determine the first image frame as the start frame. If the electronic device detects the user's second preset operation on the second image frame on the to-be-selected image frame, it determines that the second image frame is the end frame.
- the electronic device may determine the start frame and the end frame set by the user based on the user's preset operation.
- an image frame located at a preset position among the image frames to be selected is displayed in a large image on the frame selection interface, and the preset position is the middle position of the image frame to be selected or the boundary position of the frame selection interface.
- the method further includes: if the electronic device detects a slow motion trigger, displaying the first control and the second control on the frame selection interface.
- the first control and the second control can slide on the image frame to be selected, the first control is used to set the start frame, and the second control is used to set the end frame.
- the image frame where the first control or the second control is located is displayed in a large image on the frame selection interface.
- the interface displays the image frame where the first control is located in the form of a large image.
- the interface displays the image frame where the second control is located in the form of a large image.
- the electronic device determining the start frame and the end frame set by the user based on the to-be-selected image frame includes: the electronic device determines that in the to-be-selected image frame, the image frame where the first control is located is the start frame. The electronic device determines that among the image frames to be selected, the image frame where the second control is located is the end frame.
- the user can set the start frame and the end frame respectively through the first control and the second control, and the electronic device can display the image frame where the first control or the second control currently targeted by the user is located in the form of a large image .
- the image frame to be selected is obtained by extracting frames from the image frames of the first image frame set and the second image frame set at a first interval.
- the electronic device determining the start frame and the end frame set by the user based on the image frame to be selected includes: the electronic device determining the reference start frame set by the user based on the image frame to be selected.
- the electronic device displays a third image frame set on the frame selection interface, the third image frame set includes a reference start frame, and the third image frame set adopts a second interval from the image frames of the first image frame set and the second image frame set The middle-drawn frame is obtained, and the second interval is smaller than the first interval.
- the electronic device determines the start frame set by the user based on the third image frame set, and determines the reference end frame set by the user based on the candidate image frame.
- the electronic device displays a fourth image frame set on the interface, the fourth image frame set includes the reference end frame, and the fourth image frame set uses a second interval to extract frames from the image frames of the first image frame set and the second image frame set get.
- the electronic device determines the end frame set by the user based on the fourth set of image frames.
- the electronic device can first display the image frames to be selected at large intervals, so that the user can select the reference start frame and the reference end frame in a coarse-grained manner; then, the electronic device displays the reference start frame and the reference end frame at small intervals Nearby image frames, allowing users to precisely select start and end frames.
- the image frames to be selected include a fifth image frame set and a sixth image frame set
- the fifth image frame set includes a recommended start frame and adjacent multi-frame images
- the sixth image frame set Including the recommended end frame and adjacent multi-frame images.
- the electronic device determining the start frame and the end frame set by the user based on the image frame to be selected includes: the electronic device determining the start frame set by the user based on the fifth image frame set and the recommended start frame.
- the electronic device determines the end frame set by the user based on the sixth image frame set and the recommended end frame.
- the electronic device can automatically determine the more accurate recommended start frame and recommended end frame, and display them to the user; so that the user can accurately select the start frame near the recommended start frame and accurately near the recommended end frame to select the end frame.
- the image frame to be selected includes a recommended start frame interval and a recommended end frame interval.
- the electronic device determining the start frame and end frame set by the user based on the candidate image frame includes: the electronic device determining the start frame set by the user based on the recommended start frame interval; the electronic device determining the end frame set by the user based on the recommended end frame interval.
- the electronic device can automatically determine the more accurate recommended start frame interval and recommended end frame interval, and display them to the user; so that the user can accurately select the start frame based on the recommended start frame interval and the recommended end frame interval and end frame.
- the method further includes: after the electronic device detects the user's video recording operation, buffering a seventh image frame set, where the seventh image frame set includes image frames at the first frame rate before the duration T0 The image frame of the second frame rate obtained by the frame extraction.
- the target video further includes a first video segment, and the first video segment is obtained by encoding the image frames in the seventh image frame set and the eighth image frame set using the encoding frame rate.
- the eighth image frame set is an image frame of the first frame rate before the start frame in the first image frame set and the second image frame set, and the image frame of the second frame rate obtained by frame extraction.
- the first video segment is a normal-speed video segment. In this way, when the target video is played, the first video segment at a normal speed can be played first, and then the slow-motion video segment is played, giving the user a visual impact of switching between speed and slow.
- the target video further includes a second video segment, and the second video segment is obtained by encoding the image frames in the ninth image frame set at an encoding frame rate.
- the ninth image frame set is an image frame of the first frame rate after the end frame among the first image frame set and the second image frame set, and the image frame of the second frame rate obtained by the frame extraction.
- the first video segment is a normal-speed video segment.
- the slow-motion video segment can be played first, and then the second video segment of normal speed can be played, giving the user a visual impact of switching between slow and fast.
- the method further includes: after the electronic device enables the video recording function, capturing images at a third frame rate, and displaying the preview image at the third frame rate on the preview interface, where the third frame rate is less than first frame rate.
- the electronic device can capture image frames at a low frame rate and display the preview image.
- an embodiment of the present application provides a photographing apparatus, and the apparatus is included in an electronic device.
- the apparatus has the function of implementing the behavior of the electronic device in any of the above aspects and possible designs, so that the electronic device executes the video recording method performed by the electronic device in any of the possible designs in the above-mentioned aspects.
- This function can be implemented by hardware or by executing corresponding software by hardware.
- the hardware or software includes at least one module or unit corresponding to the above-mentioned functions.
- the apparatus may include an opening unit, a detection unit, a display unit, an acquisition unit, a generation unit, and the like.
- an embodiment of the present application provides an electronic device, including: a camera for capturing images; a display screen for displaying an interface; one or more processors; a memory; and one or more computer programs, one or more One or more computer programs are stored in the memory, and the one or more computer programs include instructions that, when executed by the electronic device, cause the electronic device to perform the video recording method performed by the electronic device in any of the possible designs of the above aspects.
- an embodiment of the present application provides an electronic device, including: one or more processors; and a memory, where codes are stored in the memory.
- the electronic device is caused to execute the video recording method executed by the electronic device in any possible design of the above aspect.
- an embodiment of the present application provides a computer-readable storage medium, including computer instructions, which, when the computer instructions are executed on an electronic device, cause the electronic device to execute the video recording method in any of the possible designs in the foregoing aspect.
- an embodiment of the present application provides a computer program product, which, when the computer program product runs on a computer, enables the computer to execute the video recording method performed by the electronic device in any of the possible designs in the above aspect.
- an embodiment of the present application provides a chip system, and the chip system is applied to an electronic device.
- the chip system includes one or more interface circuits and one or more processors; the interface circuit and the processor are interconnected by lines; the interface circuit is used for receiving signals from the memory of the electronic device and sending signals to the processor, and the signals are included in the memory Stored computer instructions; when the processor executes the computer instructions, it causes the electronic device to execute the video recording method in any possible design of the above aspect.
- FIG. 1 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
- FIG. 2 is a schematic flowchart of a slow-motion video recording method provided by an embodiment of the present application
- 3A is a schematic diagram of a group of interfaces provided by an embodiment of the present application.
- 3B is another set of interface schematic diagrams provided by an embodiment of the present application.
- FIG. 4 is a schematic diagram of an interface provided by an embodiment of the present application.
- FIG. 5 is a schematic diagram of an image processing process provided by an embodiment of the present application.
- FIG. 6 is another interface schematic diagram provided by an embodiment of the present application.
- FIG. 7 is a schematic diagram of another image processing process provided by an embodiment of the present application.
- FIG. 8 is a schematic diagram of a caching solution provided by an embodiment of the present application.
- FIG. 9 is a schematic diagram of a process for generating a target video according to an embodiment of the present application.
- FIG. 10 is another set of interface schematic diagrams provided by the embodiment of the present application.
- 11A is another set of interface schematic diagrams provided by an embodiment of the present application.
- FIG. 11B is another set of interface schematic diagrams provided by an embodiment of the present application.
- FIG. 12 is another set of interface schematic diagrams provided by an embodiment of the present application.
- FIG. 13 is another set of interface schematic diagrams provided by an embodiment of the present application.
- FIG. 14 is another set of interface schematic diagrams provided by an embodiment of the present application.
- FIG. 15 is another interface schematic diagram provided by an embodiment of the present application.
- FIG. 17 is another interface schematic diagram provided by an embodiment of the present application.
- FIG. 18 is a schematic structural diagram of a target video provided by an embodiment of the application.
- 19A is a sequence diagram of a slow-motion video recording method provided by an embodiment of the present application.
- 19B is a schematic diagram of the effect of a video frame played by a target video according to an embodiment of the present application.
- FIG. 20 is a schematic structural diagram of another electronic device provided by an embodiment of the present application.
- first and second are only used for descriptive purposes, and should not be construed as indicating or implying relative importance or implicitly indicating the number of indicated technical features.
- a feature defined as “first” or “second” may expressly or implicitly include one or more of that feature.
- plural means two or more.
- words such as “exemplarily” or “for example” are used to represent examples, illustrations or illustrations. Any embodiment or design described in the embodiments of the present application as “exemplarily” or “such as” should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as “exemplarily” or “such as” is intended to present the related concepts in a specific manner.
- Slow-motion video can capture the high-speed motion of objects, capture details of motion that are difficult to see with the naked eye, and record fleeting moments, which can also be called high-speed video.
- slow-motion video can record the movement of a bullet when it leaves the chamber, the action process when a football is shot, and the moment when a drop of water splashes and ripples.
- the electronic device can automatically determine the start/stop time of the motion process through algorithms such as human motion detection algorithm or optical flow motion detection algorithm, and automatically generate a slow-motion video segment according to the start/end time.
- the start/stop time of the motion process automatically judged by electronic equipment is often inaccurate.
- the user wants to capture the moment when the child pops out of the chamber, but the change of the picture when the child pops out of the chamber is not large. Accurately grab the starting time of the sub-popping chamber.
- the start position and end position of the slow-motion video segment automatically generated by the electronic device according to the start/end time also usually deviate from the user's expectation, and it is difficult to meet the user's personalized demand for the wonderful slow-motion moments.
- the embodiment of the present application provides a slow-motion video recording method, which can be applied to electronic equipment, and can be accurately set by a user during the recording process to start and end positions of a slow-motion video segment, so that the generated slow-motion video segment can be It accurately corresponds to the slow-motion wonderful moments captured, and can also meet the user's personalized needs for the slow-motion wonderful moments.
- the slow-motion video segment has a high recording frame rate, and a low encoding frame rate and playback frame rate, which can facilitate the user to slowly and carefully watch and review the slow-motion wonderful moments of the object.
- the electronic device may be a mobile phone, a tablet, a wearable device (eg, a smart watch), an in-vehicle device, an augmented reality (AR)/virtual reality (VR) device, a laptop, a super mobile personal computer
- a wearable device eg, a smart watch
- AR augmented reality
- VR virtual reality
- UMPC ultra-mobile personal computer
- PDA personal digital assistant
- FIG. 1 shows a schematic structural diagram of an electronic device 100 .
- the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charge management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2 , mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, buttons 190, motor 191, indicator 192, camera 193, display screen 194, and Subscriber identification module (subscriber identification module, SIM) card interface 195 and so on.
- SIM Subscriber identification module
- the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, and ambient light. Sensor 180L, bone conduction sensor 180M, etc.
- the processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), controller, memory, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural-network processing unit (NPU) Wait. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.
- application processor application processor, AP
- modem processor graphics processor
- graphics processor graphics processor
- ISP image signal processor
- controller memory
- video codec digital signal processor
- DSP digital signal processor
- NPU neural-network processing unit
- the controller may be the nerve center and command center of the electronic device 100 .
- the controller can generate an operation control signal according to the instruction operation code and timing signal, and complete the control of fetching and executing instructions.
- a memory may also be provided in the processor 110 for storing instructions and data.
- the memory in processor 110 is cache memory. This memory may hold instructions or data that have just been used or recycled by the processor 110 . If the processor 110 needs to use the instruction or data again, it can be called directly from memory. Repeated accesses are avoided and the latency of the processor 110 is reduced, thereby increasing the efficiency of the system.
- the electronic device 100 implements a display function through a GPU, a display screen 194, an application processor, and the like.
- the GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor.
- the GPU is used to perform mathematical and geometric calculations for graphics rendering.
- Processor 110 may include one or more GPUs that execute program instructions to generate or alter display information.
- Display screen 194 is used to display images, videos, and the like.
- Display screen 194 includes a display panel.
- the display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (active-matrix organic light).
- LED diode AMOLED
- flexible light-emitting diode flexible light-emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light-emitting diode (quantum dot light emitting diodes, QLED) and so on.
- the electronic device 100 may include one or N display screens 194 , where N is a positive integer greater than one.
- the display screen 194 may be used to display interface contents such as a preview interface and a shooting interface in a slow-motion video recording scene.
- the display screen 194 is further configured to display the high frame rate image frames buffered during the slow motion video recording on the shooting interface, so that the user can select the start position and the end position of the slow motion video segment.
- the electronic device 100 may implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, an application processor, and the like.
- the ISP is used to process the data fed back by the camera 193 .
- the shutter is opened, the light is transmitted to the camera photosensitive element through the lens, the light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing, converting it into an image visible to the naked eye.
- ISP can also perform algorithm optimization on image noise, brightness, and skin tone.
- ISP can also optimize the exposure, color temperature and other parameters of the shooting scene.
- the ISP may be provided in the camera 193 .
- Camera 193 is used to capture still images or video.
- the object is projected through the lens to generate an optical image onto the photosensitive element.
- the photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor.
- CMOS complementary metal-oxide-semiconductor
- the photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal.
- the ISP outputs the digital image signal to the DSP for processing.
- DSP converts digital image signals into standard RGB, YUV and other formats of image signals.
- the electronic device 100 may include 1 or N cameras 193 , where N is a positive integer greater than 1.
- the camera 193 may include one or more of the following cameras: a telephoto camera, a wide-angle camera, an ultra-wide-angle camera, a zoom camera, a depth camera, and the like.
- the telephoto camera has a small shooting range and is suitable for shooting distant scenes;
- the wide-angle camera has a larger shooting range;
- the ultra-wide-angle camera has a larger shooting range than the wide-angle camera and is suitable for shooting panorama and other large-scale scenes.
- the depth camera can be used to measure the object distance of the object to be shot, that is, to measure the depth information of the object to be shot, for example, it can include a three-dimensional (3 dimensions, 3D) depth camera, a time of flight (TOF) depth camera or a binocular depth camera etc.
- the camera 193 may include a front camera and/or a rear camera.
- the camera 193 may be used to capture images in a slow-motion video recording scene.
- a digital signal processor is used to process digital signals, in addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy and so on.
- Video codecs are used to compress or decompress digital video.
- the electronic device 100 may support one or more video codecs.
- the electronic device 100 can play or record videos of various encoding formats, such as: Moving Picture Experts Group (moving picture experts group, MPEG) 1, MPEG2, MPEG3, MPEG4 and so on.
- MPEG Moving Picture Experts Group
- MPEG2 moving picture experts group
- MPEG3 MPEG4
- MPEG4 Moving Picture Experts Group
- the NPU is a neural-network (NN) computing processor.
- NN neural-network
- Applications such as intelligent cognition of the electronic device 100 can be implemented through the NPU, such as image recognition, face recognition, speech recognition, text understanding, and the like.
- Internal memory 121 may be used to store computer executable program code, which includes instructions.
- the processor 110 executes various functional applications and data processing of the electronic device 100 by executing the instructions stored in the internal memory 121.
- the internal memory 121 may include a storage program area and a storage data area.
- the storage program area can store an operating system, an application program required for at least one function (such as a sound playback function, an image playback function, etc.), and the like.
- the storage data area may store data (such as audio data, phone book, etc.) created during the use of the electronic device 100 and the like.
- the internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), double-rate synchronous dynamic random access memory Take memory (double data rate synchronous dynamic random-access memory, DDR SDRAM), etc.
- non-volatile memory such as at least one disk storage device, flash memory device, universal flash storage (UFS), double-rate synchronous dynamic random access memory Take memory (double data rate synchronous dynamic random-access memory, DDR SDRAM), etc.
- the internal memory 121 may be used for buffering high frame rate image frames within a preset duration during the slow motion recording process.
- the processor 110 generates a slow-motion video segment according to the starting position and ending position accurately set by the user during the slow-motion recording process by running the instructions stored in the internal memory 121, so as to meet the user's personalized needs for the wonderful slow-motion moments. .
- the electronic device 100 may implement audio functions through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, an application processor, and the like. Such as music playback, recording, etc.
- Touch sensor 180K also called “touch panel”.
- the touch sensor 180K may be disposed on the display screen 194 , and the touch sensor 180K and the display screen 194 form a touch screen, also called a “touch screen”.
- the touch sensor 180K is used to detect a touch operation on or near it.
- the touch sensor can pass the detected touch operation to the application processor to determine the type of touch event.
- Visual output related to touch operations may be provided through display screen 194 .
- the touch sensor 180K may also be disposed on the surface of the electronic device 100 , which is different from the location where the display screen 194 is located.
- the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100 .
- the electronic device 100 may include more or less components than shown, or combine some components, or separate some components, or arrange different components.
- the illustrated components may be implemented in hardware, software, or a combination of software and hardware.
- the camera 193 can be used to capture images in a slow-motion recording scene; the internal memory 121 can be used to cache high-frame rate image frames within a preset duration during the slow-motion recording process; the display screen 194 can use It is used to display the preview interface or shooting interface, etc. in the slow-motion recording scene, and it is also used to display the high-frame rate image frames cached during the slow-motion recording process on the shooting interface, so that the user can select the slow-motion image frame from the high-frame rate image frames.
- the start position and end position of the video segment implements the user to accurately set the start position and end position of the slow motion video segment during the recording process by running the instructions stored in the internal memory 121, so as to satisfy the user's preference for the slow motion video segment. Personalized needs for action moments.
- the slow-motion video recording method provided by the embodiment of the present application will be described below by taking the electronic device as a mobile phone having the structure shown in FIG. 1 as an example.
- the mobile phone may firstly enable the video recording function. Then, after detecting the user's video recording operation, if a slow motion trigger is detected, the mobile phone displays an image frame to be selected on the frame selection interface, and the image frame to be selected includes multiple frames of images. Then, the mobile phone can obtain the start frame and the end frame set by the user based on the image frame to be selected. Then, the mobile phone can generate a target video, the target video includes a slow-motion video segment, the slow-motion video segment corresponds to the start frame and the end frame, and the recording frame rate of the slow-motion video segment is higher than the encoding frame rate of the slow-motion video segment.
- the embodiment of the present application also provides another slow-motion video recording method, see FIG. 2 , the method includes:
- the mobile phone starts the video recording function.
- the recording function of the mobile phone when the user wants to record in slow motion, the recording function of the mobile phone can be activated.
- the mobile phone may have multiple video recording functions.
- the mobile phone can activate the slow-motion video recording function.
- the phone can launch the camera app, or launch other apps with photo or video functions (such as AR apps such as Douyin or Hetu Cyberverse), and then activate the slow-motion video recording function of these apps.
- apps such as AR apps such as Douyin or Hetu Cyberverse
- the mobile phone detects that the user clicks the icon 301 of the camera application shown in (a) of FIG. 3A , the mobile phone starts the photographing function and displays the preview interface shown in (b) of FIG. 3A .
- the mobile phone detects that the user clicks the control 302 shown in (b) in FIG. 3A , as shown in (c) in FIG. 3A , the slow-motion video recording function is activated.
- the mobile phone detects that the user clicks on the control 303 shown in (b) in FIG. 3A
- the interface as shown in (d) in FIG. 3A is displayed; after the mobile phone detects that the user clicks the control 304 , as shown in (c) of FIG. 3A , the slow-motion video recording function is activated.
- the mobile phone After detecting that the user clicks the control 305 shown in (a) in FIG. 3B , the mobile phone enters the normal recording mode as shown in (b) in FIG. 3B .
- the preview interface of the normal video recording mode includes a slow-motion double-speed control 306 .
- the mobile phone starts the slow-motion recording function.
- the mobile phone starts the slow-motion video recording function after detecting that the user has selected a slow-motion double-speed of 32X through the slow-motion double-speed control 306 .
- the slow-motion multiplier kX indicates that the recording frame rate of the slow-motion video segment is k times the preset reference recording frame rate, and k is a positive integer.
- the reference video recording frame rate may be a video recording frame rate usually adopted in a normal video recording mode, such as 30fps or 25fps.
- the recording frame rate of the slow-motion video segment is the encoding frame rate of the slow-motion video segment.
- the mobile phone displays the desktop or other non-camera application interface
- the mobile phone after detecting the user's voice instruction to enter the slow-motion video recording mode, the mobile phone starts the slow-motion video recording function.
- the mobile phone can also activate the slow-motion video recording function in response to other user operations such as touch operations, voice commands, or quick gestures.
- the mobile phone uses the slow-motion video recording method provided in the following embodiments to record the target video.
- the target video includes a slow-motion video segment, and the slow-motion video segment is used to record slow-motion wonderful moments.
- the mobile phone uses the slow-motion video recording method provided in the following embodiments to record the target video only when the slow-motion video recording function is activated and the slow-motion multiplier set by the user is greater than or equal to the preset multiplier of 1.
- the preset multiplier 1 is 8X.
- the slow-motion video recording method provided in the following embodiment is used to record the target video.
- the mobile phone after the mobile phone starts the slow-motion video recording function, it can enter the slow-motion video recording mode, so that the target video can be recorded in the slow-motion video recording mode by using the slow-motion video recording method provided in the following embodiments.
- the mobile phone after the mobile phone starts the slow-motion video recording function, it may include multiple slow-motion video recording sub-modes, and the mobile phone uses the slow-motion video recording method provided in the following embodiments to record the target video only in specific sub-modes.
- the preview interface of the mobile phone includes a normal slow motion control 401 and a super slow motion control 402. After detecting that the user clicks the super slow motion control 402, the mobile phone enters the super slow motion sub-mode, and adopts the following:
- the slow-motion video recording method provided by the embodiment records the target video.
- the mobile phone collects the image frame at the frame rate f1, and displays the preview image on the preview interface.
- the mobile phone After the mobile phone starts the slow-motion video recording function, it can capture an image frame at the frame rate f1 in the preview state, obtain a preview image according to the image frame, and display the preview image on the preview interface at the frame rate f1.
- the preview image can be generated after the image frames collected by the camera are processed by the ISP.
- ISP processing operations may include converting a RAW image of an image frame to an RGB image, automatic white balance (AWB), dead pixel cluster correction, pre-filtering for noise reduction, color distortion correction, or chroma noise reduction. operate.
- the frame rate f1 is relatively low, for example, it can be the frame rate of sending display and video recording usually adopted in the ordinary video recording mode, such as 30fps or 25fps. In a possible implementation solution, the frame rate f1 is equal to the above-mentioned reference video recording frame rate.
- the process shown in step 202 may include: the camera collects image frames according to the frame rate f1 (for example, 30 fps) (that is, outputs the frames at the frame rate f1 ), and the collected image frames pass through the ISP frame by frame. After processing, a preview image is generated and displayed on the preview interface.
- the frame rate f1 for example, 30 fps
- the collected image frames pass through the ISP frame by frame.
- a preview image is generated and displayed on the preview interface.
- the method may further include step 203:
- the mobile phone acquires the slow motion double speed.
- the slow-motion double speed represents the multiple of the recording frame rate of the slow-motion video segment (hereinafter referred to as the recording frame rate corresponding to the slow-motion double speed) relative to the preset reference recording frame rate, for example, it may be 32X, 64X or 256X, etc., respectively. Expressed as 32 times, 64 times or 256 times the preset reference video frame rate.
- the recording frame rate corresponding to the slow motion double speed is used to encode the target image frame between the start position and the end position of the slow motion set by the user after the start of recording, so as to generate a slow motion video segment in the target video.
- the image frames in the slow-motion video segment have a high recording frame rate, and a low encoding frame rate and playback frame rate, which can facilitate users to watch slow-motion wonderful moments slowly, carefully, and for a long time. That is to say, the slow-motion video segment is used to record slow-motion wonderful moments.
- the slow-motion multiplier is the default multiplier, or the multiplier used in the last slow-motion recording process.
- the slow-motion multiplier is automatically determined by the mobile phone according to the shooting scene.
- the slow-motion multiplier can be larger (such as 256X or 128X, etc.)
- the slow-motion multiplier may be smaller (such as 64X or 32X, etc.).
- the slow-motion multiplier is the multiplier set by the user.
- multiple slow-motion double-speed controls 601 are displayed on the preview interface.
- the mobile phone determines the slow-motion double-speed set by the user according to the slow-motion double-speed control selected by the user.
- the slow-motion multiplier may be the multiplier set when the user activates the slow-motion video recording function.
- the mobile phone may acquire the slow-motion multiplier set by the user according to the user's voice instruction.
- the mobile phone can also modify the slow motion speed in response to the user's operation.
- step 203 is an optional step.
- the mobile phone After detecting the user's video recording operation, the mobile phone collects image frames at the frame rate f2, and performs image caching.
- the mobile phone determines that the recording operation of the user is detected, thereby entering the recording process.
- the mobile phone detects the operation of the user's voice instructing to start recording, it determines that the user's recording operation is detected, thereby entering the recording process.
- the manner for triggering the mobile phone to enter the recording process may also include gestures and other manners, which are not specifically limited in this embodiment of the present application.
- the mobile phone after the mobile phone detects the user's video recording operation, it can collect image frames at the frame rate f2 through the camera.
- the frame rate f2 may be larger.
- the frame rate f2 is greater than the frame rate f1, that is, the frame rate of the image acquisition of the mobile phone in the preview state is lower, and the frame rate of the image acquisition during the video recording process is relatively high.
- the frame rate f1 may be 30fps
- the frame rate f2 may be 1920fps, 960fps or 240fps, and so on.
- f2 may be the default frame rate, the frame rate set by the user, or the maximum frame rate that the ISP can support (ie, the maximum frame rate that the ISP can perform real-time image processing).
- the frame rate f2 may be related to the slow-motion double-speed.
- the frame rate f2 is the recording frame rate corresponding to the slow motion multiplier.
- the frame rate f2 is the frame rate corresponding to the preset multiplier 2.
- the frame rate corresponding to the preset multiplier 2 is the maximum frame rate that can be supported by the ISP.
- the maximum frame rate that the ISP can support is 1920fps
- the reference video frame rate is 30fps
- the frame rate f2 is the maximum frame rate 1920fps that the ISP can support.
- the mobile phone after the mobile phone detects the user's video recording operation, it can perform image caching according to the image frames collected at the frame rate f2. Among them, due to the high frame rate f2, the ISP may not be able to process frame by frame in real time, so the mobile phone can first cache the image frames with the frame rate f2, so that the involved image frames can be processed by the ISP in subsequent use.
- the mobile phone after the mobile phone detects the user's video recording operation, it can first buffer the image frames captured at the high frame rate f2 in real time until the buffering duration is equal to T0. After the cache time is longer than T0, the mobile phone caches the image frames collected at the high frame rate f2 during the time T0 between the current time t0 and the previous time t1 in real time, so that after the slow motion trigger is detected, according to the latest cached time T0 time Image frames captured at frame rate f2 generate slow motion video segments.
- the image frames collected at the frame rate f2 within the most recent T0 duration cached by the mobile phone may be referred to as the first image frame set.
- the mobile phone can cache the image frames with the frame rate f3 obtained after frame extraction.
- the duration of T0 is shorter, such as 0.5s, 1s, or 1.5s, etc.; f3 is smaller, such as 25fps or 30fps, etc., and f3 and f1 may be equal or unequal.
- the mobile phone after starting the recording, the mobile phone only needs to cache the image frames of the high frame rate f2 in the short T0 duration during the recording process, and the image frames of the low frame rate f3 obtained by the frame extraction before the T0 duration during the recording process , instead of caching the high frame rate f2 image frames collected during the entire recording process, so it can reduce the amount of data that the mobile phone caches in real time during the recording process, and reduce the recording process's requirements for the mobile phone's cache capacity.
- the mobile phone after the mobile phone detects the user's video recording operation, it can initialize three DDR buffer queues as shown in Figure 8: buffer queue 1, buffer queue 2, and buffer queue 3.
- the length of the cache queue 2 is f2*T0.
- the camera captures image frames at frame rate f2 (eg, 1920fps).
- the image frame newly collected by the camera is put into the head of the cache queue 2. If the cache queue 2 is full, that is, the f2*T0 frame image has been cached in the cache queue 2, the mobile phone will extract the image frame at the end of the cache queue 2 into a constant-speed image frame with a frame rate of f3 and move it to the cache queue 1. head of the team.
- the mobile phone After detecting the user's video recording operation, the mobile phone extracts frames from the image frames collected at the frame rate f2 to generate a recorded image, and displays the recorded image on the shooting interface.
- the image frame collected at the frame rate f2 is extracted to obtain an image frame with a frame rate of f3, and the image frame with a frame rate of f3 is processed by ISP.
- a recorded image is generated, and the recorded image is displayed on the shooting interface.
- the image frames processed by the ISP can also be sent to the slow motion detection module to detect whether the slow motion is triggered.
- the image frames with the frame rate f2 buffered in the buffer queue 2 can be extracted into image frames with the frame rate f3, and then sent to the ISP for processing to generate a video frame. Take an image and send it to display.
- the mobile phone uses high frame rate f2 to collect image frames, it only performs ISP processing and display on image frames with low frame rate f3 obtained after frame extraction, so the calculation amount of image processing can be reduced. Reduce the requirement for the real-time computing power of the mobile phone during the recording process.
- the mobile phone After the mobile phone detects the slow motion trigger, it collects image frames with a duration of T1 at the frame rate f2 and caches them.
- the mobile phone After the mobile phone detects the slow motion trigger, it can be determined that the photographed object is currently in the process of motion, so that a slow motion video segment can be generated according to the image frames in the motion process.
- the slow motion trigger may be automatically triggered by the mobile phone, or may be actively triggered by the user.
- the automatic triggering algorithm adopted by the mobile phone may include a human motion detection algorithm or an optical flow motion detection algorithm.
- the human motion detection algorithm can detect the motion of the photographed human body. If the mobile phone detects the movement of the human body being photographed, the continuous process of the movement can be processed in slow motion, thereby generating a slow-motion video segment.
- Table 1 can refer to the correspondence between motion actions and detection conditions. After the mobile phone meets the detection conditions, it is determined that the motion of the human body is detected, so that it can be determined that the slow motion is triggered.
- the optical flow motion detection algorithm can detect whether the photographed object has motion by detecting the intensity of the picture change between adjacent image frames. If the mobile phone detects that the object to be photographed moves, slow motion processing is performed on the motion process to generate a slow motion video segment, so it can be determined that slow motion is triggered.
- a slow motion trigger control is displayed on the shooting interface, and the mobile phone determines that the slow motion is triggered after detecting the user's operation of clicking the control. For another example, after the mobile phone detects the user's voice instruction triggered by the slow motion, it is determined that the slow motion is triggered.
- the mobile phone after detecting the video recording operation, the mobile phone caches the image frames collected at the frame rate f2 during the time T0 between the current time t0 and the previous time t1 in real time during the video recording process, and the image frames before time t1
- the frame rate after the frame is the image frame of f3.
- the mobile phone caches the image frames collected at the frame rate f2 within the T0 time period before the slow motion trigger is detected. Since it takes a certain amount of time for the mobile phone to detect the slow motion trigger, after the mobile phone detects the slow motion trigger, the slow motion may have lasted for a period of time. Therefore, the mobile phone caches the data collected at the frame rate f2 during the T0 period before the mobile phone cache detects the slow motion trigger. Image frames, you can try to record the slow motion process completely.
- the mobile phone can use the frame rate f2 to continue to collect image frames of the subsequent T1 duration and cache them, so as to make the whole slow motion process as complete as possible. recorded.
- the cached image frames with a frame rate of f2 in the T1 duration after the slow motion trigger is detected may be referred to as a second image frame set.
- the mobile phone caches an image frame with a frame rate of f2 in the duration T0 before the slow motion trigger is detected, and the cached image frame with a frame rate of f2 in the T1 period after the slow motion trigger is detected, Used to generate slow motion video clips.
- the length of the cache queue 2 can be extended to f2*(T0+T1);
- the image frames newly collected by the camera at the frame rate f2 are put into the queue head of the cache queue 2, and the image frames are stopped when the cache queue 2 is full.
- the above time lengths of T0 and T1 are preset by the mobile phone.
- the time lengths of T0 and T1 may be set by the user.
- the user can set the values of T0 and T1 according to the shooting scene.
- the time lengths of T0 and T1 are automatically determined by the mobile phone according to the shooting scene. For example, if the duration of the motion process in the shooting scene is long (such as the shooting scene of falling water droplets), T0 and T1 may be longer, so that the mobile phone can cache more image frames with a frame rate of f2. In this way, more image frames can be included between the start frame and the end frame selected by the user, so that there are more video frames in the slow-motion video segment generated according to the image frames between the start frame and the end frame, so that Users can watch slow-motion wonderful moments slowly and carefully through more video frames in the slow-motion video segment.
- T0 and T1 may be short.
- the mobile phone displays the image frame to be selected according to the cached image frame, and acquires the start frame and the end frame set by the user.
- the mobile phone After the mobile phone detects the slow motion trigger, it can display the image frame to be selected on the frame selection interface according to the image frames in the first image frame set and the second image frame set, so as to obtain the user-based image frame based on the image frame to be selected as shown in FIG. 9 .
- the image frame with the frame rate f2 between the start frame and the end frame may be the target image frame or used to generate the target image frame, and the target image frame is used to generate the slow motion video segment in the target video obtained by final shooting.
- the slow-motion video segment is used to record the wonderful moments of slow-motion desired by the user.
- the mobile phone can display the image frames to be selected on the frame selection interface in various manners, which are not specifically limited in this embodiment of the present application.
- the image frames to be selected are image frames in the first image frame set and the second image frame set, so that the user can select a start frame and an end frame therefrom.
- the number of image frames with the frame rate f2 in the first image frame set and the second image frame set is large, and the difference between adjacent image frames is small, which is inconvenient for the user to select the starting point. frame and end frame. Therefore, the mobile phone can obtain image frames to be selected after extracting frames from the image frames in the first image frame set and the second image frame set. Then, the mobile phone displays the image frames to be selected on the frame selection interface, so that the user can quickly select the start frame and the end frame based on a small number of image frames with large differences.
- the number of image frames to be selected is also different if the slow-motion multiplier is different.
- the larger the slow-motion multiplier the higher the motion speed of the motion process the user wants to shoot, and the greater the difference between adjacent image frames.
- a large number of image frames to be selected can accurately select the start frame and end frame corresponding to the high-speed motion process.
- the smaller the slow motion multiplier the smaller the number of image frames to be selected. That is to say, the number of image frames to be selected is positively related to the slow motion double speed. For example, if the slow motion multiplier is 32X, the image frames to be selected may be 30 frames; if the slow motion multiplier is 64X, the image frames to be selected may be 60 frames.
- the frame extraction interval at which the mobile phone extracts frames to obtain the image frame to be selected from the cached first image frame set and the second image frame set is also different.
- the larger the slow-motion multiplier the more image frames in the first image frame set and the second image frame set, and the frame sampling interval can also be correspondingly larger, so as to facilitate the user to obtain images based on the larger frame sampling interval.
- the frame sampling interval can be 16 frames; if the slow motion multiplier is 64X, the frame sampling interval can be 32 frames. That is to say, the frame sampling interval used to obtain the image frame to be selected is positively related to the slow motion double speed.
- the user can set the number of image frames to be selected or the frame sampling interval.
- the mobile phone may display a frame number control/frame sampling interval control of multiple image frames to be selected on the frame selection interface, so that the user can select the number of image frames to be selected/frame sampling interval.
- the larger the slow-motion multiplier the larger the number of image frames corresponding to the frame number control displayed on the mobile phone.
- the frame number controls displayed on the frame selection interface of the mobile phone include: a 15-frame control, a 30-frame control, and a 45-frame control.
- the frame number controls displayed on the frame selection interface of the mobile phone include: 30-frame control, 60-frame control and 90-frame control.
- the larger the slow motion multiplier the larger the frame sampling interval corresponding to the frame sampling interval control displayed on the frame selection interface of the mobile phone.
- the mobile phone can first select a larger image frame from the image frames of the first image frame set and the second image frame set. Frames are drawn at intervals to obtain image frames to be selected and displayed on the frame selection interface, so that the user can first select the reference start frame and reference end frame in a coarse-grained manner. Then, the mobile phone can also extract frames from the image frames of the first image frame set and the second image frame set at a small frame sampling interval to obtain the reference starting frame and the nearby candidate image frames (which may be referred to as the third image frame). frame set), and the reference end frame and nearby candidate image frames (may be referred to as the fourth image frame set). The mobile phone displays the third image frame set and the fourth image frame set on the frame selection interface, so that the user can select the start frame and the end frame in a fine-grained and precise manner.
- the mobile phone in order to facilitate the user to select the start frame and the end frame, can display the recommended start frame and the recommended end frame automatically determined by the algorithm on the frame selection interface, or mark the recommended start frame in the image frame to be selected.
- Start frame and recommended end frame The start frame and end frame that the user wants to select are usually the recommended start frame and the recommended end frame. In this way, by prompting the user with the recommended start frame and the recommended end frame on the frame selection interface, the mobile phone can facilitate the user to accurately select the start frame near the recommended start frame and accurately select the end frame near the recommended end frame.
- the mobile phone in order to facilitate the user to select the start frame and the end frame, can display the recommended start frame interval and the recommended end frame interval automatically determined by the algorithm on the frame selection interface, or mark the image frame to be selected.
- Recommended start frame interval and recommended end frame interval The start frame and end frame that the user wants to select is usually within or near the recommended start frame interval and the recommended end frame interval.
- the mobile phone can facilitate the user to accurately select the starting frame in or near the recommended starting frame interval, and accurately select the starting frame in or near the recommended ending frame interval. to select the end frame.
- the image frames to be selected can be displayed in the form of thumbnails, so that thumbnails of multiple image frames can be displayed on the interface at the same time, which is convenient for the user to select.
- Thumbnails of image frames to be selected can be displayed anywhere on the frame selection interface, for example, they are arranged horizontally at the bottom of the frame selection interface, or vertically arranged on the right side of the frame selection interface. Among them, the interface may only display the thumbnails of some of the image frames to be selected at the same time, but not all of the image frames to be selected at the same time. Thumbnail of the frame.
- the mobile phone after the mobile phone detects a slow motion trigger, it performs frame extraction and other processing on the subsequently collected image frames with a frame rate of f2, and generates a recorded image with a frame rate of f3.
- the recorded image is displayed on the background of the frame selection interface in real time at the frame rate f3, and the thumbnail of the image frame to be selected is displayed on the foreground of the frame selection interface at the same time.
- the currently displayed recorded image is continuously displayed on the background of the frame selection interface, and the recorded image displayed in the background is no longer refreshed, and is displayed on the foreground of the frame selection interface. Thumbnails of the image frames to be selected are displayed on the screen.
- the recorded image is no longer displayed on the frame selection interface, but only the thumbnails of the image frames to be selected are displayed on the frame selection interface.
- a shooting control is displayed on the frame selection interface, and the state of the shooting control is consistent with that of the shooting control on the preview interface.
- the shooting controls are not displayed on the frame selection interface.
- the mobile phone continues to display the shooting controls on the frame selection interface. After the mobile phone stops capturing images, the shooting controls are no longer displayed on the frame selection interface; or, the mobile phone displays the shooting controls on the frame selection interface, and the shooting controls are in the same state as the shooting controls on the preview interface.
- the mobile phone displays the photographing interface and the recorded photographing image as shown in (a) of FIG. 10 .
- the mobile phone may prompt the user to detect a slow motion trigger on the shooting interface.
- the mobile phone continues to refresh and display the recorded images on the shooting interface, until the mobile phone stops capturing image frames and stops refreshing and displaying the recorded images.
- the mobile phone may display thumbnails 1000 of the image frames to be selected on the frame selection interface, and prompt the user to set the start frame and the end frame.
- the thumbnails 1000 of the image frames to be selected can be slidably switched to display thumbnails of other image frames to be selected in response to the user's sliding operation. For example, in response to the user's operation of sliding to the right based on the thumbnail of the image frame to be selected as shown in (f) in FIG. 10 , as shown in (g) of FIG. 10 , the thumbnail of the image frame to be selected also moves to the right Swipe to display thumbnails of other candidate image frames.
- the frame selection interface further includes a first control 1001 for indicating the start frame and a second control 1002 for indicating the end frame.
- the first control 1001 and the second control 1002 can slide on the thumbnails of the image frames to be selected in response to the user's drag operation.
- the mobile phone may prompt the user to select the starting frame on the frame selection interface .
- the mobile phone can also prompt the user whether the selection of the starting frame is completed on the frame selection interface. After the mobile phone detects that the user clicks the "OK" control, it is determined that the selection of the starting frame is completed. Then, as shown in (h) of FIG.
- the mobile phone may prompt the user to select the end frame on the frame selection interface.
- the selection method of the end frame is similar to that of the start frame, and will not be repeated.
- the mobile phone after detecting the user's recording operation, can also display the slow-motion double speed on the interface of the recording process, such as "256X" shown in (a)-(h) of FIG. 10 .
- the user can select the start frame and the end frame based on the first control 1001 and the second control 1002, respectively, and the mobile phone can prompt the user on the frame selection interface, "Are the start and end frames selected to complete?" The mobile phone After detecting that the user clicks the "OK" control, the start frame corresponding to the position of the first control 1001 and the end frame corresponding to the position of the second control 1002 are acquired.
- the mobile phone After detecting the slow motion trigger, if the mobile phone detects the user's preset operation 1 for the image frame 1 in the thumbnail of the image frame to be selected, the mobile phone determines that the image frame 1 is the starting frame. If the user's preset operation 2 with respect to the image frame 2 in the thumbnail of the image frame to be selected is detected, the image frame 2 is determined as the end frame.
- the preset operation 1 and the preset operation 2 may be operations such as clicking, double-clicking, or pressure pressing, and the specific forms of the preset operation 1 and the preset operation 2 are not limited in this embodiment of the present application. In some embodiments, referring to (a) in FIG.
- the mobile phone after detecting the slow motion trigger, the mobile phone prompts the user to select the start frame and the end frame on the frame selection interface.
- the mobile phone detects the operation after the user clicks the image frame 1 and the image frame 2 in the thumbnail of the image frame to be selected, it determines that the timing of the image frame 1 and the image frame 2 is ahead.
- the image frame is the start frame, and the image frame later in the timing sequence is the end frame.
- a thumbnail image 1100 of the image frame to be selected may be displayed on the frame selection interface.
- the thumbnail image of the image frame to be selected further includes a selection control 1101 for selecting a start frame and an end frame. Selection control 1101 selects one of the image frames. Referring to (b) in FIG. 11B , the selection control 1101 can slide on the thumbnails of the image frames to be selected in response to the user's drag operation, thereby selecting different image frames.
- the interface also includes a large image 1102 of the image frame selected by the selection control 1101 .
- the large image 1102 can scroll to display the image frame and adjacent image frames where the control 1101 is located in the thumbnail of the to-be-selected image frame in the form of animation.
- the image frame corresponding to the selection control 1101 in real time can be displayed in the form of a large image on the frame selection interface, which is convenient for the user to see the specific content of the image frame selected by the selection control 1101, thereby facilitating the user to accurately select the start frame and the end frame.
- the mobile phone may first prompt the user to select the starting frame.
- a selection mark for example, the ⁇ shown in the figure or the text mark "starting frame" is displayed on the image frame, And the image frame is determined as the start frame. Then, referring to (c) in FIG. 11B , the mobile phone can prompt the user to select the end frame. As shown in (d) of FIG. 11B , the selection method of the end frame is similar to that of the start frame, and will not be repeated.
- the mobile phone may display the image frame at the preset position in the image frame to be selected in the form of a large image on the frame selection interface.
- the preset position is the middle position of the to-be-selected image frame or the interface boundary position or the like.
- the above selection control can be replaced with a draggable dot 1201 as shown in (a)-(e) in FIG. 12 or other forms of controls. be limited.
- the mobile phone may display a large image of the image frame selected by the selection control on the frame selection interface, or may not display the large image.
- the process of the user selecting the start frame and the end frame is similar to the method shown in (a)-(d) in FIG. 11B , and will not be repeated.
- the mobile phone detects the slow motion trigger, it no longer refreshes and displays a new recording image on the background of the frame selection interface, but continuously displays the recorded recording from the memory until the slow motion trigger is displayed on the background. image, and display the thumbnail of the image frame to be selected on the foreground of the frame selection interface.
- the thumbnail of the image frame to be selected can be displayed, and the user is prompted to select the interval corresponding to the start frame and the end frame.
- the mobile phone determines that the first frame of the image frame in the frame selection is the starting frame. The last image frame in the image frame interval is the end frame.
- the mobile phone may also prompt the user to select the slow-motion double speed.
- the mobile phone can display multiple multi-speed controls on the frame selection interface for the user to choose.
- the thumbnails of the image frames to be selected obtained at larger frame sampling intervals can be displayed on the frame selection interface, so that the user can The reference start frame is selected granularly.
- the frame selection interface displays the image frames to be selected near the reference start frame ( That is, the thumbnail image of the third image frame set), so that the user can accurately select the starting frame with fine granularity.
- the mobile phone may also choose to refer to the end frame and the end frame, which will not be repeated.
- the mobile phone after detecting the slow motion trigger, continues to refresh the newly obtained recorded image on the background of the frame selection interface, and at the same time displays the thumbnail of the image frame to be selected on the foreground of the frame selection interface. .
- the thumbnail of the image frame to be selected is displayed on the frame selection interface, and the recommended start frame and the recommended end frame are marked, so that the user can select the recommended start frame and the recommended end frame respectively.
- the image frames to be selected include a fifth image frame set and a sixth image frame set
- the fifth image frame set includes the recommended starting frame as shown in the first row of thumbnails in FIG. 15 and adjacent
- the sixth image frame set includes the recommended end frame and adjacent multi-frame images as shown in the thumbnail image in the second row in FIG. 15 .
- the user may set the start frame based on the fifth image frame set and the recommended start frame, and set the end frame based on the sixth image frame set and the recommended end frame.
- the thumbnails of the image frames to be selected are displayed on the frame selection interface, including the recommended start frame interval and the recommended end. Frame interval, so that the user can quickly and accurately select the start frame and the end frame within or near the recommended start frame interval and the recommended end frame interval, respectively.
- thumbnails of the image frames to be displayed are arranged horizontally on the frame selection interface.
- FIG. 17 For a schematic diagram of vertical arrangement of thumbnails of image frames to be displayed, see FIG. 17 .
- the mobile phone generates a target video, where the target video includes a slow-motion video segment, and the slow-motion video segment corresponds to a start frame and an end frame.
- a target video can be generated, and the target video includes a slow motion video segment obtained from the target image frames between the start frame and the end frame.
- the frame rate between the target image frame and the end frame is f2.
- the target image frame is an image frame obtained by interpolating an image frame with a frame rate of f2 between the start frame and the end frame.
- the frame rate of the target image frame is f4, which may be greater than or equal to f2.
- Image post-processing operations can be performed in processors such as CPU, GPU, or NPU.
- image post-processing operations may include image smoothing, image sharpening, image enhancement, histogram equalization, image super-resolution, noise reduction or frame interpolation, and the like.
- the frame rate f4 of the target image frame may be referred to as the recording frame rate of the slow-motion video segment.
- the encoding frame rate of the slow-motion video segment may be much smaller than the recording frame rate.
- the recording frame rate may be 7680fps, and the encoding frame rate may be 30fps or 25fps.
- the encoding frame rate is equal to f3.
- the mobile phone can interpolate image frames with a frame rate of 1920fps between the start frame and the end frame to obtain a target image with a frame rate of 7680fps frame; the mobile phone uses 30fps to encode the target image frame with a frame rate of 7680fps to generate a slow-motion video segment.
- the recording frame rate f4 of the slow-motion video segment is 7680fps.
- f4 is the default preset frame rate, or the encoding frame rate used when the target image frame was last encoded to generate the slow-motion video segment.
- the video recording frame rate f4 corresponds to the obtained slow motion double speed.
- the video frame rate corresponding to the slow motion double speed is the same as f2
- f4 is the same as f2.
- the recording frame rate corresponding to the slow motion double speed is greater than f2
- f4 is greater than f2
- f4 is the recording frame rate corresponding to the slow motion double speed.
- the slow motion speed is 32X
- the reference recording frame rate is 30fps
- the slow-motion multiplier is 256X
- the reference video frame rate is 30fps
- the mobile phone If the mobile phone does not obtain the slow-motion double-speed in step 203, it can obtain the slow-motion double-speed in step 208, thereby obtaining the recording frame rate corresponding to the slow-motion double-speed, which is f4; or, the mobile phone can obtain it in step 208.
- the target image frame 80 with the frame rate f2 between the start frame and the end frame in the buffer queue 2 can be used for frame interpolation to generate the target image frame with the frame rate f4, so that Encode to generate slow motion video segments.
- f1 is 30fps or 25fps
- f2 is 1920fps
- the slow-motion multiplier is 256X
- f3 is equal to f1
- f4 is 7680fps.
- the target video may further include a video segment 1 located before the slow-motion video segment, and/or a video segment 2 located after the slow-motion video segment.
- the mobile phone extracts the image frames with the frame rate of f2 (called the eighth image frame set) before the starting frame, obtains the image frames with the frame rate of f3 and passes Video segment 1 is generated after ISP processing, image post-processing and encoding.
- the mobile phone extracts an image frame with a frame rate of f2 (called the ninth image frame) after the end frame, obtains an image frame with a frame rate of f3 and passes the ISP Video segment 2 (also referred to as the second video segment) is generated after processing, image post-processing and encoding.
- the mobile phone can extract image frames with a frame rate of f3 from the image frames in the cache queue 2 whose acquisition time is before the start frame selected by the user, and move them to the image frames of the cache queue 1.
- the head of the line gets a set 801 of image frames.
- the image frame set 801 in the buffer queue 1 can be used for encoding to generate the normal-speed video segment 1.
- the mobile phone can extract the image frames whose collection time in the cache queue 2 is after the end frame selected by the user into an image frame with a frame rate of f3, and move it to the head of the cache queue 3 to obtain the image frame set 802 .
- the image frame set 802 in the buffer queue 3 can be used for encoding to generate the normal-speed video segment 2 .
- the mobile phone will no longer generate video segment 1, or the video segment 1 is empty; if the end frame is cached If the frame rate is the last image frame in the image frames of f2, the mobile phone no longer generates video segment 2, or the video segment 2 is empty.
- the target video may also include video segment 0 preceding video segment 1 .
- the mobile phone performs ISP processing and image post-processing on the buffered image frames with a frame rate of f3 before the T0 duration (called the seventh image frame set), and encodes them at a frame rate of f3 to generate video segment 0.
- the image frame set 800 with the frame rate f3 buffered in the buffer queue 1 can be used for encoding to generate the normal-speed video segment 0 .
- the video segment 0 and the video segment 1 may be referred to as the first video segment.
- the sequence of the video segments in the target video may be: video segment 0, video segment 1, slow-motion video segment, and video segment 2, ie, the first video segment, the slow-motion video segment, and the second video segment.
- the start frame and end frame of the target image frame used to generate the slow-motion video segment are precisely selected by the user during the recording process, that is, the user accurately selects the slow-motion video in the target video
- the start position and end position of the segment can be avoided, so that the false detection that occurs when the mobile phone automatically detects the start/end time of the slow motion can be avoided, so that the generated slow motion video segment can accurately correspond to the captured slow motion moments, and can also satisfy the Users' personalized needs for wonderful moments in slow motion.
- the mobile phone plays the generated target video, including video segment 0, video segment 1, slow-motion video segment, and video segment 2 in the playback target video.
- the playback frame rate of the target video is less than f4.
- the playback frame rate is usually equal to the encoding frame rate, or has a small difference from the encoding frame rate.
- the playback frame rate is also much smaller than the recording frame rate of the slow-motion video segment.
- the motion process corresponding to the slow-motion video segment can be stretched and played at a slow speed, so as to facilitate the user to pass a longer period of time. Watch the best moments in slow motion carefully and clearly.
- the video frame rate and encoding frame rate of video segment 0, video segment 1 and video segment 2 are all f3, and the playback frame rate is equal to the encoded frame rate or not much different from the encoded frame rate, that is, the playback frame rate is equal to the video frame rate or The difference is not big, so it can be called a constant-speed video segment.
- the target video may also include a normal-speed video segment before the slow-motion video segment, such as video segment 0 or video segment 1, etc., and may also include a normal-speed video after the slow-motion video segment, such as Video segment 2.
- a normal-speed video segment before the slow-motion video segment such as video segment 0 or video segment 1, etc.
- a normal-speed video after the slow-motion video segment such as Video segment 2.
- the generated target video can give the user a visual contrast impact from the normal-speed video segment to the slow-motion video segment played at a slow speed, and then to the normal-speed video segment, and the user experience is better.
- FIG. 19A for a sequence diagram of a slow-motion video recording method described in the above embodiments.
- FIG. 19B For a schematic diagram of the effect of the video frame played by the target video generated by the mobile phone, please refer to FIG. 19B .
- the method may further include: the mobile phone collects image frames with a duration of T2 at the frame rate f3 and buffers them.
- the target video generated by the mobile phone also includes video segment 3 obtained by performing ISP processing and image post-processing on the cached image frames with a frame rate of f3 and a duration of T2, and after encoding according to the encoding frame rate.
- the mobile phone automatically stops capturing image frames.
- the method may further include: the mobile phone collects image frames at the frame rate f2 and buffers them, and does not stop collecting image frames until the user's operation of stopping recording is detected.
- the encoding frame rate of the slow-motion video is getting higher and higher, and correspondingly, the capture frame rate f2 during the recording process is also getting higher and higher.
- the duration of the motion process is short. In order to better see the motion details in a short period of time, the encoding frame rate of the slow-motion video is relatively high, and the captured frame rate during the recording process is correspondingly high. The rate f2 is also higher.
- the mobile phone does not cache the high frame rate image frames collected at f2 during the entire video recording process, but only caches the high frame rate image frames within the duration T0 and T1, and compares the After the high frame rate image frame is extracted, it is processed and sent to the ISP, which can reduce the requirements for the mobile phone image cache transmission and computing power.
- the mobile phone only performs high frame rate encoding on the high frame rate image frames between the start frame and the end frame selected by the user, and performs low frame rate encoding on other low frame rate image frames during the recording process, which can reduce the size of the image. Processing tasks, reducing the requirements for the computing power of the mobile phone.
- the user can select the start frame and the end frame during the video recording process to generate a slow-motion video segment in the target video, and does not need to perform two steps after generating the video. Edited to get a slow-motion video clip.
- the scheme of generating video first and then editing to obtain slow-motion video segments requires the ISP to process the captured image frames in real time during the recording process, and then encode them to generate the video.
- the ISP has limited processing capabilities and is difficult to achieve high Real-time processing of image frames at the acquisition frame rate.
- the mobile phone only caches the image frames with the frame rate f2 of the T0 duration after the mobile phone starts recording, and does not perform image caching in the preview state. In some other embodiments, in the preview state, the mobile phone caches the image frames with the frame rate f2 of the duration T0.
- the mobile phone after the mobile phone starts the slow-motion video recording function, the mobile phone captures image frames at the frame rate f2 in the preview state, and extracts the frames into image frames with the frame rate f3 for display. Moreover, the mobile phone first buffers the image frames collected at the high frame rate f2 in real time until the buffering duration is equal to T0. After the cache time is longer than T0, the mobile phone caches the image frames collected at the high frame rate f2 during the T0 time between the current time t0 and the previous time t1 in real time, and extracts the image frames with the frame rate f2 before the T0 time. frame, get the image frame with the frame rate f3 and then buffer it.
- the mobile phone After the mobile phone detects the user's video recording operation, it continues to cache the image frames in the previous cache state. That is to say, after the mobile phone detects the user's video recording operation, it continues the cache progress in the preview state, and continues to cache the image frames collected at the frame rate f2 during the period T0 between the current time t0 and the previous time t1, and the image frames before time t1. An image frame with a frame rate of f3 is obtained after the image frames collected at the frame rate of f2 are extracted. Then, the mobile phone can perform steps 205-208 in the above embodiment.
- the mobile phone may detect the slow motion trigger soon after detecting the user's video recording operation, for example, the time difference between the moment when the slow motion trigger is detected and the moment when the video recording operation is detected is less than T0. At this time, if the mobile phone starts to cache image frames only after detecting the user's video recording operation, the cached image frames with high frame rate f2 may not include the complete slow motion process that occurs before the slow motion is triggered.
- the mobile phone caches image frames with high frame rate f2 in the preview state, and continues to cache image frames in the preview state after detecting the user's video recording operation, it can make the cached image frames with high frame rate f2 as far as possible. Include the complete slow motion process that occurs before the slow motion trigger. In this way, when the mobile phone detects the slow motion trigger immediately after detecting the recording operation, it can also generate a complete slow motion video segment according to the cached image frame.
- the electronic device is a mobile phone as an example for illustration.
- the electronic device is a tablet computer or other device such as a camera
- the slow-motion recording method provided in the above embodiment can still be used, and the user can accurately set the slow-motion recording method during the recording process.
- the starting position and ending position of the action video segment enable the generated slow-motion video segment to accurately correspond to the captured slow-motion wonderful moment, and can also meet the user's personalized needs for the slow-motion wonderful moment, which will not be repeated here.
- the electronic device includes corresponding hardware and/or software modules for executing each function.
- the present application can be implemented in hardware or in the form of a combination of hardware and computer software in conjunction with the algorithm steps of each example described in conjunction with the embodiments disclosed herein. Whether a function is performed by hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functionality for each particular application in conjunction with the embodiments, but such implementations should not be considered beyond the scope of this application.
- the electronic device can be divided into functional modules according to the above method examples.
- each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module.
- the above-mentioned integrated modules can be implemented in the form of hardware. It should be noted that, the division of modules in this embodiment is schematic, and is only a logical function division, and there may be other division manners in actual implementation.
- Embodiments of the present application further provide an electronic device, including one or more processors and one or more memories.
- the one or more memories are coupled to the one or more processors for storing computer program code, the computer program code comprising computer instructions that, when executed by the one or more processors, cause the electronic device to perform
- the above-mentioned relevant method steps implement the slow-motion video recording method in the above-mentioned embodiment.
- An embodiment of the present application further provides an electronic device, as shown in FIG. 20 , comprising: a display screen 2001, one or more processors 2002, multiple cameras 2003, a memory 2004, and one or more computer programs 2005, each of the above Devices may be connected by one or more communication buses 2006 .
- the one or more computer programs 2005 are stored in the aforementioned memory 2004 and configured to be executed by the one or more processors 2002, the one or more computer programs 2005 include instructions that may be used to perform the aforementioned implementations the steps in the example.
- the processor 2002 may be the processor 110 shown in FIG. 1
- the memory 2004 may be the internal memory 121 shown in FIG. 1
- the camera 2003 may be the camera 193 shown in FIG.
- the display screen 2001 may specifically be the display screen 194 shown in FIG. 1 .
- Embodiments of the present application further provide a computer-readable storage medium, where computer instructions are stored in the computer-readable storage medium, and when the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to realize the above-mentioned embodiments The slow-motion video recording method in .
- Embodiments of the present application also provide a computer program product, which, when running on a computer, causes the computer to execute the above-mentioned relevant steps, so as to realize the slow-motion video recording method executed by the electronic device in the above-mentioned embodiment.
- the embodiments of the present application also provide an apparatus, which may specifically be a chip, a component or a module, and the apparatus may include a connected processor and a memory; wherein, the memory is used to store computer execution instructions, and when the apparatus is running, The processor can execute the computer-executed instructions stored in the memory, so that the chip executes the slow-motion video recording method executed by the electronic device in the above method embodiments.
- the electronic device, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding method provided above. Therefore, for the beneficial effects that can be achieved, reference may be made to the above-provided method. The beneficial effects in the corresponding method will not be repeated here.
- the disclosed apparatus and method may be implemented in other manners.
- the device embodiments described above are only illustrative.
- the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods.
- multiple units or components may be Incorporation may either be integrated into another device, or some features may be omitted, or not implemented.
- the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.
- the units described as separate components may or may not be physically separated, and components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place, or may be distributed to multiple different places . Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
- each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.
- the above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.
- the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a readable storage medium.
- the technical solutions of the embodiments of the present application can be embodied in the form of software products in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, which are stored in a storage medium , including several instructions to make a device (may be a single chip microcomputer, a chip, etc.) or a processor (processor) to execute all or part of the steps of the methods described in the various embodiments of the present application.
- the aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (random access memory, RAM), magnetic disk or optical disk and other media that can store program codes.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Human Computer Interaction (AREA)
- Television Signal Processing For Recording (AREA)
- Studio Devices (AREA)
Abstract
Description
运动动作 | 检测条件 |
跳跃 | 原地向上跳跃,有明显下蹲起跳过程 |
定点投篮 | 双手抱球高高扬起,准备瞄准投篮 |
单手挥手 | 在脖子及以上高度左右挥动手臂 |
跑步 | 有明显的摆臂和抬腿,具有一定的跑步速度 |
抬脚准备射门 | 区别于走路和足球带球时的踢,幅度较大,腿脚后拉 |
转身大动作 | 单次或连续多圈转身,手抬起或举起自然甩动,旋转有一定速度 |
挥拍 | 抬手,区别与普通甩手,有较大幅度 |
劈腿跳跃 | 区别于普通跳跃,向上跳跃时需有劈腿动作,腿向两边展开 |
打高尔夫球 | 大力双手挥杆击球 |
打乒乓球 | 大力挥拍打球,有一定幅度 |
跑步上篮 | 身体跳跃,单手上篮 |
欢呼 | 双手举起挥舞,轻微跳动 |
鼓掌 | 双手胸前鼓掌,双臂可能张开 |
滑板 | 豚跳时刻,滑板离地或者双脚离开滑板 |
Claims (21)
- 一种录像方法,应用于电子设备,其特征在于,包括:开启录像功能;在检测到用户的录像操作后,若检测到慢动作触发,则在选帧界面上显示待选图像帧,所述待选图像帧包括多帧图像;获取所述用户基于所述待选图像帧设置的起始帧和结束帧;生成目标视频,所述目标视频包括慢动作视频段,所述慢动作视频段与所述起始帧和所述结束帧相对应,所述慢动作视频段的录像帧率高于所述慢动作视频段的编码帧率。
- 根据权利要求1所述的方法,其特征在于,所述方法还包括:在检测到所述用户的录像操作后,以第一帧率采集图像帧,并缓存第一图像帧集合,所述第一图像帧集合包括最近采集到的T0时长内的图像帧;在检测到慢动作触发后,以所述第一帧率采集T1时长的图像帧,并缓存第二图像帧集合,所述第二图像帧集合包括以所述第一帧率采集的所述T1时长的图像帧;所述待选图像帧来自于所述第一图像帧集合和所述第二图像帧集合。
- 根据权利要求1或2所述的方法,其特征在于,所述待选图像帧通过所述第一图像帧集合和所述第二图像帧集合中的图像帧抽帧获得。
- 根据权利要求3所述的方法,其特征在于,所述待选图像帧的数量与所述慢动作视频段的慢动作倍速正相关;或者,从所述第一图像帧集合和所述第二图像帧集合中的图像帧,抽帧获得所述待选图像帧的抽帧间隔,与所述慢动作视频段的慢动作倍速正相关。
- 根据权利要求2-4任一项所述的方法,其特征在于,所述慢动作视频段采用所述编码帧率,对所述起始帧及所述结束帧之间的目标图像帧获得;其中,所述目标图像帧为所述第一图像帧集合和所述第二图像帧集合中,所述起始帧及所述结束帧之间的图像帧,所述录像帧率等于所述第一帧率;或者,所述目标图像帧为所述第一图像帧集合和所述第二图像帧集合中,所述起始帧及所述结束帧之间第一帧率的图像帧,插帧后获得的所述录像帧率的图像帧。
- 根据权利要求2-5任一项所述的方法,其特征在于,所述方法还包括:在检测到用户的录像操作后,在所述拍摄界面上以第二帧率显示录拍图像,所述录拍图像根据所述第一帧率的图像帧抽帧后获得,所述第二帧率小于所述第一帧率。
- 根据权利要求1-6任一项所述的方法,其特征在于,所述录像帧率与所述慢动作视频段的慢动作倍速相对应。
- 根据权利要求2-7任一项所述的方法,其特征在于,所述第一帧率与所述慢动作视频段的慢动作倍速相关。
- 根据权利要求1-6任一项所述的方法,其特征在于,所述方法还包括:若检测到慢动作触发,则在所述选帧界面上提示用户设置所述起始帧和所述结束帧。
- 根据权利要求1-9任一项所述的方法,其特征在于,所述确定所述用户基于所述待选图像帧设置的起始帧和结束帧,包括:若检测到所述用户针对所述待选图像帧上第一图像帧的第一预设操作,则确定所述第一图像帧为所述起始帧;若检测到所述用户针对所述待选图像帧上第二图像帧的第二预设操作,则确定所述第二图像帧为所述结束帧。
- 根据权利要求10所述的方法,其特征在于,所述选帧界面上以大图形式显示有所述待选图像帧中位于预设位置的图像帧,所述预设位置为所述待选图像帧的中间位置或所述选帧界面的边界位置。
- 根据权利要求1-9任一项所述的方法,其特征在于,所述方法还包括:若检测到慢动作触发,则在选帧界面上显示第一控件和第二控件,所述第一控件和所述第二控件可在所述待选图像帧上滑动,所述第一控件用于设置所述起始帧,所述第二控件用于设置所述结束帧;所述选帧界面上以大图形式显示所述第一控件或所述第二控件所在位置的图像帧;当所述用户在所述待选图像帧上拖动所述第一控件时,所述界面上以大图形式显示所述第一控件所在位置的图像帧;当所述用户在所述待选图像帧上拖动所述第二控件时,所述界面上以大图形式显示所述第二控件所在位置的图像帧;所述确定所述用户基于所述待选图像帧设置的起始帧和结束帧,包括:确定所述待选图像帧中,所述第一控件所在位置的图像帧为所述起始帧;确定所述待选图像帧中,所述第二控件所在位置的图像帧为所述结束帧。
- 根据权利要求3-9任一项所述的方法,其特征在于,所述待选图像帧采用第一间隔从所述第一图像帧集合和所述第二图像帧集合的图像帧中抽帧获得;所述确定所述用户基于所述待选图像帧设置的起始帧和结束帧,包括:确定所述用户基于所述待选图像帧设置的参考起始帧;在所述选帧界面上显示第三图像帧集合,所述第三图像帧集合包括所述参考起始帧,且所述第三图像帧集合采用第二间隔从所述第一图像帧集合和所述第二图像帧集合的图像帧中抽帧获得,所述第二间隔小于所述第一间隔;确定所述用户基于所述第三图像帧集合设置的所述起始帧;确定所述用户基于所述待选图像帧设置的参考结束帧;在所述界面上显示第四图像帧集合,所述第四图像帧集合包括所述参考结束帧,且所述第四图像帧集合采用所述第二间隔从所述第一图像帧集合和所述第二图像帧集合的图像帧中抽帧获得;确定所述用户基于所述第四图像帧集合设置的所述结束帧。
- 根据权利要求1-9任一项所述的方法,其特征在于,所述待选图像帧包括第五图像帧集合和第六图像帧集合,所述第五图像帧集合包括推荐起始帧及相邻的多帧图像,所述第六图像帧集合包括推荐结束帧及相邻的多帧图像;所述确定所述用户基于所述待选图像帧设置的起始帧和结束帧,包括:确定所述用户基于所述第五图像帧集合和所述推荐起始帧设置的所述起始帧;确定所述用户基于所述第六图像帧集合和所述推荐结束帧设置的所述结束帧。
- 根据权利要求1-9任一项所述的方法,其特征在于,所述待选图像帧包括推荐起始帧区间和推荐结束帧区间;所述确定所述用户基于所述待选图像帧设置的起始帧和结束帧,包括:确定所述用户基于所述推荐起始帧区间设置的所述起始帧;确定所述用户基于所述推荐结束帧区间设置的所述结束帧。
- 根据权利要求1-15任一项所述的方法,其特征在于,所述方法还包括:在检测到所述用户的录像操作后,缓存第七图像帧集合,所述第七图像帧集合包括所述T0时长之前的第一帧率的图像帧抽帧获得的第二帧率的图像帧;所述目标视频还包括第一视频段,所述第一视频段采用所述编码帧率对所述第七图像帧集合和第八图像帧集合中的图像帧编码获得;其中,所述第八图像帧集合为所述第一图像帧集合和所述第二图像帧集合中,所述起始帧之前的第一帧率的图像帧,抽帧获得的所述第二帧率的图像帧。
- 根据权利要求1-16任一项所述的方法,其特征在于,所述目标视频还包括第二视频段,所述第二视频段采用所述编码帧率对第九图像帧集合中的图像帧编码获得;其中,所述第九图像帧集合为所述第一图像帧集合和所述第二图像帧集合中,所述结束帧之后的第一帧率的图像帧,抽帧获得的所述第二帧率的图像帧。
- 根据权利要求1-17任一项所述的方法,其特征在于,所述方法还包括:在开启录像功能后,以第三帧率采集图像,并在预览界面上以所述第三帧率显示预览图像,所述第三帧率小于所述第一帧率。
- 一种电子设备,其特征在于,包括:摄像头,用于采集图像;显示屏,用于显示界面;一个或多个处理器和一个或多个存储器;所述一个或多个存储器与所述一个或多个处理器耦合,所述一个或多个存储器用于存储计算机程序代码,所述计算机程序代码包括计算机指令,当所述一个或多个处理器执行所述计算机指令时,所述电子设备执行如权利要求1-18中任一项所述的录像方法。
- 一种计算机存储介质,其特征在于,包括计算机指令,当所述计算机指令在电子设备上运行时,使得所述电子设备执行如权利要求1-18中任一项所述的录像方法。
- 一种计算机程序产品,其特征在于,当所述计算机程序产品在计算机上运行时,使得所述计算机执行如权利要求1-18中任一项所述的录像方法。
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US18/253,388 US20240007588A1 (en) | 2020-11-18 | 2021-11-16 | Slow-motion video recording method and device |
EP21893881.9A EP4236300A4 (en) | 2020-11-18 | 2021-11-16 | METHOD AND DEVICE FOR SLOW-MOTION VIDEO RECORDING |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202011296546.8A CN114520886A (zh) | 2020-11-18 | 2020-11-18 | 一种慢动作录像方法及设备 |
CN202011296546.8 | 2020-11-18 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2022105734A1 true WO2022105734A1 (zh) | 2022-05-27 |
Family
ID=81594168
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2021/130849 WO2022105734A1 (zh) | 2020-11-18 | 2021-11-16 | 一种慢动作录像方法及设备 |
Country Status (4)
Country | Link |
---|---|
US (1) | US20240007588A1 (zh) |
EP (1) | EP4236300A4 (zh) |
CN (1) | CN114520886A (zh) |
WO (1) | WO2022105734A1 (zh) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115022679A (zh) * | 2022-05-30 | 2022-09-06 | 北京百度网讯科技有限公司 | 视频处理方法、装置、电子设备和介质 |
CN117479000A (zh) * | 2022-08-08 | 2024-01-30 | 荣耀终端有限公司 | 录像方法和相关装置 |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115225826A (zh) * | 2022-06-30 | 2022-10-21 | 联想(北京)有限公司 | 图像拍摄方法、装置、设备和存储介质 |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107666581A (zh) * | 2016-07-29 | 2018-02-06 | 三星电子株式会社 | 提供视频内容的方法和支持该方法的电子装置 |
CN110086905A (zh) * | 2018-03-26 | 2019-08-02 | 华为技术有限公司 | 一种录像方法及电子设备 |
CN110506415A (zh) * | 2018-03-26 | 2019-11-26 | 华为技术有限公司 | 一种录像方法及电子设备 |
CN111010521A (zh) * | 2018-10-04 | 2020-04-14 | 三星电子株式会社 | 在便携式电子设备中记录超慢动作视频的方法和系统 |
US20200244884A1 (en) * | 2019-01-25 | 2020-07-30 | Samsung Electronics Co., Ltd. | Apparatus and method for producing slow motion video |
Family Cites Families (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107251551B (zh) * | 2015-02-10 | 2020-10-13 | 索尼半导体解决方案公司 | 图像处理设备、图像捕获装置、图像处理方法和存储介质 |
CN112805996B (zh) * | 2018-10-08 | 2022-04-22 | 华为技术有限公司 | 一种用于生成慢动作视频片段的设备和方法 |
-
2020
- 2020-11-18 CN CN202011296546.8A patent/CN114520886A/zh active Pending
-
2021
- 2021-11-16 EP EP21893881.9A patent/EP4236300A4/en active Pending
- 2021-11-16 US US18/253,388 patent/US20240007588A1/en active Pending
- 2021-11-16 WO PCT/CN2021/130849 patent/WO2022105734A1/zh active Application Filing
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107666581A (zh) * | 2016-07-29 | 2018-02-06 | 三星电子株式会社 | 提供视频内容的方法和支持该方法的电子装置 |
CN110086905A (zh) * | 2018-03-26 | 2019-08-02 | 华为技术有限公司 | 一种录像方法及电子设备 |
CN110506415A (zh) * | 2018-03-26 | 2019-11-26 | 华为技术有限公司 | 一种录像方法及电子设备 |
CN111010521A (zh) * | 2018-10-04 | 2020-04-14 | 三星电子株式会社 | 在便携式电子设备中记录超慢动作视频的方法和系统 |
US20200244884A1 (en) * | 2019-01-25 | 2020-07-30 | Samsung Electronics Co., Ltd. | Apparatus and method for producing slow motion video |
Non-Patent Citations (1)
Title |
---|
See also references of EP4236300A4 |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115022679A (zh) * | 2022-05-30 | 2022-09-06 | 北京百度网讯科技有限公司 | 视频处理方法、装置、电子设备和介质 |
CN115022679B (zh) * | 2022-05-30 | 2023-08-29 | 北京百度网讯科技有限公司 | 视频处理方法、装置、电子设备和介质 |
CN117479000A (zh) * | 2022-08-08 | 2024-01-30 | 荣耀终端有限公司 | 录像方法和相关装置 |
Also Published As
Publication number | Publication date |
---|---|
EP4236300A4 (en) | 2024-04-17 |
CN114520886A (zh) | 2022-05-20 |
EP4236300A1 (en) | 2023-08-30 |
US20240007588A1 (en) | 2024-01-04 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2022105734A1 (zh) | 一种慢动作录像方法及设备 | |
JP7371227B2 (ja) | インテリジェントビデオ録画方法及び装置 | |
US20200227089A1 (en) | Method and device for processing multimedia information | |
WO2022062318A1 (zh) | 一种拍摄方法及设备 | |
WO2021052414A1 (zh) | 慢动作视频拍摄方法及电子设备 | |
US9088721B2 (en) | Imaging apparatus and display control method thereof | |
WO2021223500A1 (zh) | 一种拍摄方法及设备 | |
CN113099146B (zh) | 一种视频生成方法、装置及相关设备 | |
WO2019183784A1 (zh) | 一种录像方法及电子设备 | |
WO2021244104A1 (zh) | 一种延时摄影的拍摄方法及设备 | |
CN114339102B (zh) | 一种录像方法及设备 | |
WO2021043061A1 (zh) | 图像处理方法、装置、存储介质及电子设备 | |
CN115176455A (zh) | 功率高效的动态电子图像稳定 | |
WO2022252649A1 (zh) | 一种视频的处理方法及电子设备 | |
EP4109879A1 (en) | Image color retention method and device | |
CN114339101B (zh) | 一种录像方法及设备 | |
CN117278839A (zh) | 一种拍摄方法、电子设备及存储介质 | |
RU2818577C1 (ru) | Способ и устройство видеозаписи с замедленной съемкой | |
WO2022228259A1 (zh) | 一种目标追踪方法及相关装置 | |
EP4258130A1 (en) | Method and apparatus for viewing multimedia content | |
CN114697525B (zh) | 一种确定跟踪目标的方法及电子设备 | |
WO2024114569A1 (zh) | 一种视频处理的方法以及电子设备 | |
US11601588B2 (en) | Take-off capture method and electronic device, and storage medium | |
KR20220080063A (ko) | 골프 스윙 촬영 방법 및 장치 | |
CN117119284A (zh) | 一种拍摄方法 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21893881 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 18253388 Country of ref document: US |
|
ENP | Entry into the national phase |
Ref document number: 2021893881 Country of ref document: EP Effective date: 20230523 |
|
REG | Reference to national code |
Ref country code: BR Ref legal event code: B01A Ref document number: 112023009577 Country of ref document: BR |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
ENP | Entry into the national phase |
Ref document number: 112023009577 Country of ref document: BR Kind code of ref document: A2 Effective date: 20230518 |