CN106998449B - Method and device for realizing seamless video recording in application layer and terminal equipment - Google Patents
Method and device for realizing seamless video recording in application layer and terminal equipment Download PDFInfo
- Publication number
- CN106998449B CN106998449B CN201710228016.1A CN201710228016A CN106998449B CN 106998449 B CN106998449 B CN 106998449B CN 201710228016 A CN201710228016 A CN 201710228016A CN 106998449 B CN106998449 B CN 106998449B
- Authority
- CN
- China
- Prior art keywords
- video
- time
- wrapper
- data
- audio
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
Abstract
The embodiment of the invention provides a method, a device and terminal equipment for realizing seamless video recording on an application layer, which adopt MediaCodec to encode preview video data and audio data to obtain audio and video encoding data, then adopt MediaMuxer to control two packers to alternatively pack the audio and video encoding data, and before one packer stops a packing process, the other packer is started in advance and starts a packing operation, so that two adjacent video files have an overlapping part, and the phenomenon of second leakage is eliminated. In this embodiment, the MediaCodec runs after the start of video recording until the end of seamless video recording, and does not need to cyclically perform the operations of releasing and reconstructing the encoder resources, and the two encapsulators controlled by the mediamux alternately operate, so that even if one of the encapsulators needs to release and reconstruct the resources, the other encapsulator is in an encapsulation operating state, thereby really achieving seamless video recording.
Description
Technical Field
The embodiment of the invention relates to the technical field of video recording, in particular to a method and a device for realizing seamless video recording in an Android system application layer and terminal equipment.
Background
In production and life, continuous video recording is required in many occasions, such as monitoring video recording on streets, monitoring video recording in production workshops, driving recording on automobiles and the like. Since the longer the video recording time is, the larger the recorded video file is, it is impossible to store the recorded video in one file during the continuous video recording process, and it is necessary to perform the sectional video recording according to the time, that is, one video file is produced every time a period of recording is performed. For example, if the recording time of each video file is set to 5 minutes in advance and the recording starts from 9:00:00, the following video files should be obtained in turn theoretically: the video recording method comprises the steps of recording a first video file in a range of 9:00: 00-9: 05:00, recording a second video file in a range of 9:05: 00-9: 10:00, recording a third video file in a range of 9:10: 00-9: 15:00 and the like.
However, the video recording scheme adopted by the existing video recording equipment needs a certain time interval from the storage of a video recording file (stopping video recording) to the new creation of a video recording file (restarting video recording); during this period, the video recording program at least needs to complete the operations of releasing the decoder and the wrapper, and re-creating the wrapper and the decoder, so that video recording cannot be performed, i.e. the so-called missing second phenomenon, and the missing second time is usually about 1 s. For example, in the above example, the second video file may start recording later than 9:05:00, such as 9:05:01, and the actual recording time of the second video file is 9:05:01 to 9:10: 01.
Therefore, the existing video recording scheme cannot really realize continuous video recording, and important events are very likely to occur in the time period of missing seconds, but cannot be recorded into a video recording file. Therefore, a seamless video recording scheme capable of eliminating the second missing phenomenon is needed.
Disclosure of Invention
The embodiment of the invention provides a method, a device and terminal equipment for realizing seamless video recording in an Android system application layer, and aims to solve the problem of seconds missing commonly existing in the conventional continuous video recording scheme.
In a first aspect, an embodiment of the present invention provides a method for implementing seamless video recording in an Android system application layer, where the method includes:
coding preview video data and audio data through a coder interface MediaCodec to obtain audio and video coding data; the preview video data is preview frame data which is used for previewing and displaying on a screen in the original video data acquired by the video acquisition device;
starting a first encapsulator through a encapsulator interface MediaMuxer, and performing encapsulation operation on the audio and video coding data through the first encapsulator;
before the first wrapper stops wrapping, starting a second wrapper through the MediaMuxer, and performing wrapping operation on the audio and video coding data through the second wrapper;
restarting the first encapsulator by the mediamultiplexer before the second encapsulator stops encapsulating.
Optionally, before the first encapsulator stops encapsulating, starting, by the mediamultiplexer, a second encapsulator, comprising:
when the current time reaches a first pre-starting time, starting the second wrapper through the MediaMuxer; the lead of the first pre-starting time relative to the first division time is a first preset interval time, and the first division time is the time when the first wrapper stops wrapping;
restarting, by the mediamultiplexer, the first encapsulator before the second encapsulator stops encapsulating, comprising:
when the current time reaches a second pre-starting time, restarting the first wrapper through the MediaMuxer; and the advance of the second pre-starting time relative to the second division time is a second preset interval time, and the second division time is the time when the second wrapper stops wrapping.
Optionally, the method further includes:
when the current time reaches the first segmentation time, controlling the first packaging device to stop packaging operation, and storing a video file obtained by packaging the first packaging device;
and when the current time reaches the second segmentation time, controlling the second packaging device to stop packaging operation, and storing the video file obtained by packaging the second packaging device.
Optionally, the method further includes:
at any time between the first division time and a fourth pre-starting time, the mediamultiplexer further includes, after controlling the first wrapper to stop the wrapping operation and before reaching the fourth pre-starting time: the MediaMuxer releases the first wrapper and recreates the first wrapper, wherein the fourth pre-start time is a time when the MediaMuxer starts the first wrapper again;
at any time between the second division time and a third pre-starting time, after the MediaMuxer controls the second wrapper to stop the wrapping operation and before the third pre-starting time is reached, the MediaMuxer releases the second wrapper and re-creates the second wrapper, wherein the third pre-starting time is a time when the MediaMuxer starts the second wrapper again.
Optionally, the method further includes:
after a first wrapper is started, detecting whether a key frame arrives in the audio and video coded data, and after the key frame arrives, executing the step of executing a wrapping operation on the audio and video coded data through the first wrapper;
and/or after a second wrapper is started, detecting whether a key frame arrives in the audio and video coded data, and after the key frame arrives, executing the step of performing a wrapping operation on the audio and video coded data through the second wrapper.
Optionally, the method further includes:
and determining the first preset interval time and/or the second preset interval time according to the interval time of two adjacent key frames.
Optionally, the starting the first wrapper includes: creating a first video file, and writing decoding configuration data into the first video file;
the initiating a second wrapper, comprising: and creating a second video file, and writing decoding configuration data into the second video file.
In a second aspect, an embodiment of the present invention provides a device for implementing seamless video recording in an application layer of an Android system, where the device includes: an encoding control unit and an encapsulation control unit;
the coding control unit is used for coding preview video data and audio data through a coder interface MediaCodec to obtain audio and video coding data; the preview video data is preview frame data which is used for previewing and displaying on a screen in the original video data acquired by the video acquisition device;
the encapsulation control unit is used for starting a first encapsulator through an encapsulator interface mediamultiplexer and executing encapsulation operation on the audio and video coding data through the first encapsulator; before the first wrapper stops wrapping, starting a second wrapper through the MediaMuxer, and performing wrapping operation on the audio and video coding data through the second wrapper; restarting the first encapsulator by the mediamultiplexer before the second encapsulator stops encapsulating.
Optionally, the encapsulation control unit may be specifically configured to: when the current time reaches a first pre-starting time, starting a second wrapper through the MediaMuxer; the lead of the first pre-starting time relative to the first division time is a first preset interval time, and the first division time is the time when the first wrapper stops wrapping;
the packaging control unit may be further configured to: when the current time reaches a second pre-starting time, starting a first wrapper through the MediaMuxer; and the advance of the second pre-starting time relative to the second division time is a second preset interval time, and the second division time is the time when the second wrapper stops wrapping.
Optionally, the encapsulation control unit may be further configured to: when the current time reaches the first segmentation time, controlling the first packaging device to stop packaging operation, and storing a video file obtained by packaging the first packaging device; and when the current time reaches the second division time, controlling the second packaging device to stop packaging operation, and storing the video file obtained by packaging the second packaging device.
Optionally, the apparatus further comprises:
the key frame detection unit is used for detecting whether a key frame arrives in the audio and video coded data after the packaging control unit starts a first packaging device, and triggering the packaging control unit to execute the step of performing packaging operation on the audio and video coded data through the first packaging device after the key frame arrives;
and/or, after the second wrapper is started by the wrapping control unit, detecting whether a key frame arrives in the audio and video coded data, and after the key frame arrives, triggering the wrapping control unit to execute the step of performing a wrapping operation on the audio and video coded data through the second wrapper.
In a third aspect, an embodiment of the present invention provides a terminal device, including: the device comprises a camera, a microphone, a processor and a memory;
a computer program executable on the processor is stored in the memory; the processor is connected with the camera, the microphone and the memory through a preset interface;
the camera is used for collecting video data;
the microphone is used for collecting audio data;
the processor is configured to execute the computer program, so as to enable the terminal device to implement the seamless video recording method according to any one of the above embodiments.
According to the technical scheme, the method is different from the integrated packaging type video recording process of the traditional video recording interface MediaRecoder, the embodiment of the invention firstly adopts an encoder interface MediaCodec to encode preview video data and audio data to obtain audio and video encoding data, then adopts a packager interface MediaMuxer to package the audio and video encoding data, and the packaging process comprises the following steps: the two packers alternately execute the packing operation of the audio and video coding data, and before one packers stops the packing process once, the other packers are started in advance and start the packing operation, so that the two adjacent video files have overlapped parts, and the second leakage phenomenon can be completely eliminated. In this embodiment, the encoder interface MediaCodec is operated after the start of video recording until the whole seamless video recording process is finished, and there is no need to cyclically perform the operations of releasing and reconstructing the encoder resources.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive exercise.
Fig. 1 is a flowchart of a method for implementing seamless video recording in an application layer of an Android system according to an embodiment of the present invention;
FIG. 2 is a schematic time-line diagram of a seamless video recording method according to an embodiment of the present invention;
fig. 3 is a signal flow diagram of another method for implementing seamless video recording in an application layer of an Android system according to an embodiment of the present invention;
FIG. 4 is a signal flow diagram corresponding to the seamless video recording method shown in FIG. 3;
fig. 5 is a schematic structural diagram of a device for implementing seamless video recording in an Android system application layer according to an embodiment of the present invention;
fig. 6 is a block diagram of a terminal device with a seamless video recording function according to an embodiment of the present invention.
Detailed Description
The embodiment of the invention provides a method, a device and terminal equipment for realizing seamless video recording in an Android system application layer, and aims to solve the problem of seconds missing commonly existing in the conventional continuous video recording scheme.
The inventor finds that in the prior art, in the original design of an Android system, a recording function is realized through a highly integrated and packaged recording interface MediaRecorder. The problem with such highly integrated packaging is that resource release and re-establishment takes a long time; specifically, the video recording function is realized by using a MediaRecorder, the method integrated in the MediaRecorder is traversed and executed once (including the creation/acquisition of various resources at the beginning of video recording and the release of various resources at the end of video recording), a video recording process is completed once, a video recording file is obtained, namely, when the video recording in a time period is completed (a video recording file is obtained), the actions of releasing an encoder, releasing an encapsulator, releasing a Camera resource and the like are executed in sequence, and then when the video recording in the next time period is started, the actions of creating a Camera resource, creating an encapsulator, creating an encoder and the like are executed in sequence. Therefore, in the process of realizing video recording based on the MediaRecorder in the prior art, the execution complexity of the flow is high when the video recording is finished and restarted, and the time consumption is long, which is an important reason for causing the problem of second leakage.
In view of this, the embodiment of the present invention uses the most basic code of the multimedia codec to perform programming, including using a coder interface MediaCodec to realize coding of audio and video, and working all the time in the whole seamless video recording process, using a wrapper interface mediamux to create two wrappers to realize packaging of audio and video coded data, each wrapper working once to obtain a video file, and controlling the two wrappers to work alternately to realize uninterrupted packaging operation of the audio and video coded data continuously output by the coder, thereby ensuring that there is no missing seconds in the packaging process, ensuring that the video data at each moment after starting video recording can be coded, packaged and written into the video file, eliminating the phenomenon of missing seconds, and realizing seamless continuous video recording. The embodiments of the present invention will be described in detail below with reference to the accompanying drawings.
Fig. 1 is a flowchart of a method for implementing seamless video recording in an application layer of an Android system according to a possible embodiment of the present invention. Referring to fig. 1, the seamless video recording method at least includes the following steps:
s11, coding the preview video data and the audio data through a coder interface MediaCodec to obtain audio and video coding data;
the preview video data is preview frame data used for previewing and displaying on a screen in the original video data acquired by the video acquisition device. In practical applications, in order to enable a user to track and preview current video content in real time, a part of image frames, i.e. the preview video data, is usually captured from an original image acquired by a camera according to a screen refresh rate of a screen and displayed on the screen in real time. Because the original image acquired by the camera can only be acquired at a Native layer of the system, and the preview video data can be acquired at an application layer of the system, in this embodiment, the MediaCodec encodes the preview video data, so that the implementation of the method does not need to change a bottom code (such as the Native layer) of the system, and the degree of freedom of encoding is improved.
S12, starting a first wrapper through a wrapper interface MediaMuxer, and receiving the audio and video coding data through the first wrapper;
s13, performing encapsulation operation on the audio and video coded data through the first encapsulator, and performing S14 before the first encapsulator stops encapsulating;
s14, starting a second packer through the MediaMuxer, and receiving the audio and video coding data through the second packer;
and S15, performing packaging operation on the audio and video coded data through the second packager, and performing step S12 before the second packager stops packaging.
In this embodiment, in step S13, the second wrapper is started before the first wrapper stops wrapping, and the audio/video encoded data is received by the second wrapper, so that, during the period from the start of the second wrapper to the stop of the wrapping of the first wrapper, the first wrapper and the second wrapper receive the audio/video encoded data and perform a wrapping operation at the same time, thereby ensuring that no second leakage occurs during the process of switching from "the first wrapper performs a wrapping operation" to "the second wrapper performs a wrapping operation"; similarly, in step S15, before the second encapsulator stops encapsulating, step S12 is executed again to start the first encapsulator and receive the audio/video encoded data through the first encapsulator, so that during the period from the start of the first encapsulator to the stop of the second encapsulator, the second encapsulator and the first encapsulator receive the audio/video encoded data and perform an encapsulating operation at the same time, thereby ensuring that no second leakage occurs during the process of switching from "performing the encapsulating operation by the second encapsulator" to "performing the encapsulating operation by the first encapsulator".
Referring to the schematic time line diagram of the seamless video recording method in the embodiment shown in fig. 2, it can be known that each wrapper in the embodiment generates a video file each time it performs a wrapping operation; at the time t1 after the video recording is started, firstly, receiving audio and video coding data by a first packaging device and packaging; then, at the time t2 before the current packaging process of the first packaging device stops, the second packaging device is started and packages the audio/video coded data output by the current encoder, and at the time, the first packaging device continues to receive and package the audio/video coded data; then, at the time t3, the first packaging device stops packaging to obtain a video file F1, and the second packaging device continues to execute packaging operation; at time t4 before the second wrapper stops wrapping, the first wrapper starts to perform the wrapping operation again, similarly to time t 2; at the time of t5, the second wrapper stops wrapping to obtain a video file F2, the first wrapper continues to perform wrapping operation … …, it is seen that in the time period of t 2-t 3, the two wrappers work in an overlapping manner, the video files F1 and F2 both store the audio and video data recorded in the time period of t 2-t 3, and similarly, in the time period of t 4-t 5, the two wrappers work in an overlapping manner, so that the audio and video data recorded in the time period of t 4-t 5 are both stored in the video files F2 and F3.
That is, in the embodiment, by the overlapping alternative work between the first wrapper and the second wrapper, the tail part of the previous video file is overlapped with the head part of the next video file in the two adjacent video files obtained in the whole video recording process, so that the second missing phenomenon can be completely avoided, the audio and video data at each moment in the process from the start of video recording to the end of video recording are all packaged into the video files, and seamless video recording is really realized.
It can be known from the above technical solutions that, in the embodiments of the present invention, an integrated packaging type video recording process of the conventional video recording interface mediadecoder is abandoned, a coder interface MediaCodec is first adopted to encode preview video data and audio data to obtain audio and video encoded data, and then a packager interface mediamultiplexer is adopted to package the audio and video encoded data, where the packaging process is: the two packers alternately execute the packing operation of the audio and video coding data, and before one packers stops the packing process once, the other packers are started in advance and start the packing operation, so that the two adjacent video files have overlapped parts, and the second leakage phenomenon can be completely eliminated. In this embodiment, the encoder interface MediaCodec is operated after the start of video recording until the whole seamless video recording process is finished, and there is no need to cyclically perform the operations of releasing and reconstructing the encoder resources.
In a feasible embodiment, because the MediaCodec acquires preview video data from the application layer of the system to perform video coding, the coder interface MediaCodec and the encapsulator interface MediaMuxer can be respectively implemented based on a programming language such as Java, and the like, and are made into an application package (for example, an installation file in an apk format applied to an Android system), so that any terminal device can implement the seamless video recording function described in this embodiment by installing the application package without changing the system bottom code (i.e., without replacing or updating the system). Therefore, the seamless video recording method provided by the embodiment has strong portability, and can be easily transplanted to any terminal device through the application program package.
In a feasible embodiment of the present invention, the starting of the first wrapper in step S12 may specifically include: creating a first video file, and writing decoding configuration data into the first video file; correspondingly, the step S14 of starting the second wrapper may specifically include: and creating a second video file, and writing decoding configuration data into the second video file.
That is, when each packaging device starts a new packaging process, it first creates a video file packaged this time, specifies the file name of the video file (e.g. 20160926090000.mp4, which indicates that recording is started at 9 o' clock 26/9/2016) and writes decoding configuration data into the video file, so that when the video file is played, the video file is decoded according to the decoding configuration data.
Fig. 3 is a flowchart of another method for implementing seamless video recording in an application layer of an Android system according to an embodiment of the present invention, and fig. 4 is a signal flow diagram corresponding to fig. 3. Referring to fig. 3 and 4, the method includes the steps of:
s11, coding the preview video data and the audio data through a coder interface MediaCodec to obtain audio and video coding data;
s12, starting the first wrapper through a wrapper interface MediaMuxer, receiving the audio and video coding data through the first wrapper, and respectively executing the steps S131, S132 and S133;
s131, performing packaging operation on the audio and video coded data through the first packaging device;
s132, judging whether the current time is the first pre-starting time, if so, continuing to execute the step S14, otherwise, repeatedly executing the step S132;
s133, judging whether the current time is the first segmentation time, if so, executing the step S134, otherwise, repeatedly executing the step S133;
s134, controlling the first packaging device to stop packaging operation, and storing the video file obtained by packaging the first packaging device;
s14, starting the second wrapper through a wrapper interface MediaMuxer, receiving the audio and video coding data through the second wrapper, and respectively executing the steps S151, S152 and S153;
s151, performing packaging operation on the audio and video coded data through the second packaging device;
s152, judging whether the current time is the second pre-starting time, if so, returning to the step S12, otherwise, repeatedly executing the step S152;
s153, judging whether the current time is the second division time, if so, executing the step S154, otherwise, repeatedly executing the step S153;
and S154, controlling the second packaging device to stop packaging operation, and storing the video file obtained by packaging the second packaging device.
In this embodiment, the first pre-starting time is earlier than the first dividing time, and a time difference (hereinafter referred to as a first preset interval time) between the first pre-starting time and the first dividing time may be set according to actual requirements; similarly, the second pre-starting time is earlier than the second dividing time, and a time difference (hereinafter referred to as a second preset interval time) between the second pre-starting time and the second dividing time can also be set according to actual requirements. The first preset interval time and the second preset interval time may be the same or different.
In a possible embodiment, before starting recording, the recording duration of each video file, and the first preset interval time and the second preset interval time may be set, so that when packaging is performed by the first wrapper, a first division time is determined according to the start time of the packaging operation of the first wrapper (i.e., the start time of the video file currently stored by the first wrapper) and the start time of the recording duration, and then a first preset start time is determined according to the first division time and the preset first preset interval time; similarly, when the second wrapper is used for wrapping, a second division time is determined according to the start time of the wrapping operation of the second wrapper (i.e. the start time of the video file currently stored by the second wrapper) and the start time of the video recording duration, and a second preset start time is determined according to the second division time and a preset second preset interval time.
It can be seen that, in this embodiment, by using the preset parameters such as the video recording duration and the preset interval time, the start time of one wrapper can be determined while the wrapper executes the packaging operation, so as to ensure that the start time of the other wrapper is earlier than the time when the wrapper stops packaging, thereby realizing real seamless video recording. Meanwhile, the preset interval time can be adjusted, so that the overlapping working time of two encapsulators, namely the overlapping time of two adjacent video files, can be adjusted, and different practical application requirements can be met; for example, when the storage space of the video recording device is small, the preset interval time can be properly reduced, and the overlapping of the video recording files stored for a long time is avoided, so that the storage space of the video recording files is saved.
In a possible embodiment of the present invention, after the MediaMuxer starts the first wrapper (step S12), it may first detect whether a key frame arrives in the audio/video encoded data (specifically, video encoded data) acquired by the first wrapper, and after the key frame arrives, perform a wrapping operation from the key frame (step S13 or S131); accordingly, after the second packetizer is started (step S14), the MediaMuxer may first detect whether a key frame arrives in the encoded audio/video data acquired by the second packetizer, and after the key frame arrives, perform the packetizing operation from the key frame (step S15 or S151).
The video coding process is carried out according to groups, each group of coded data starts from a key frame (namely an I frame), each key frame is a complete picture, but the video frames among the key frames are incomplete and need to be calculated according to the related key frames when playing and decoding, therefore, if the first frame of the video file is not a key frame, the beginning part cannot be correctly decoded and calculated to obtain the complete picture when playing, and a mosaic effect is caused.
In a possible embodiment of the present invention, based on the key frame detection step, the seamless video recording method may further include the following steps:
and determining the first preset interval time and/or the second preset interval time according to the interval time of two adjacent key frames.
Assuming that the interval time between two adjacent key frames in the video coding data is 1s, the first preset interval time and the second preset interval time can be set to any value between [1s,2 s). In conjunction with the time-axis diagram shown in fig. 2, if the last frame of F1 is a key frame, t2 actually falls between the third last key frame and the second last key frame of F1, so that the second wrapper writes the wrapped data in F2 from the time when the second last key frame of F1 arrives, and the overlapping duration of F1 and F2 is an interval time, i.e., 1 s; if the last frame of F1 is not a key frame, t2 falls between the last two key frames of F1, so the second wrapper writes data in F2 starting from the last key frame of F1, with the overlap duration of F1 and F2 being less than 1 s.
Therefore, through the arrangement, on the premise of ensuring that two adjacent video files are overlapped, the overlapping time is not more than 1s (one interval time), so that data redundancy caused by overlong overlapping time of the two adjacent video files is avoided, and the storage pressure of the related storage units on the video files is reduced.
In the embodiment of the invention, the first preset interval time and the second preset interval time can be determined by detecting the interval time of key frames in audio and video coding data in real time in the video recording process through the method, so that the segmentation time and the preset starting time corresponding to each wrapper are determined; before starting video recording, the interval time of the key frame can be determined in advance according to the related parameters of the encoder in the terminal equipment, and then the first preset interval time and the second preset interval time are determined, so that in the video recording process, the segmentation time and the preset starting time corresponding to each packaging device are determined according to the first preset interval time and the second preset interval time, and accurate control over starting and stopping packaging of each packaging device is guaranteed.
In a possible embodiment of the present invention, in the seamless video recording method, after each encapsulator stops encapsulating, the following operations may be performed; and releasing the wrapper, and recreating the wrapper by utilizing the wrapper creation thread.
Specifically, between the first division time and a fourth pre-start time, the fourth pre-start time is a time when the MediaMuxer starts the first wrapper again (of course, a time when the first wrapper starts each time may also be defined as a second pre-start time, that is, between steps S133 to S152 shown in fig. 3 or fig. 4, that is, after the first wrapper stops packaging this time and before the first wrapper starts next time), the MediaMuxer may perform the following steps: releasing the first wrapper, and re-creating the first wrapper, so that the first wrapper can be started in time when the fourth pre-starting time and the second pre-starting time are reached (corresponding to step S152); correspondingly, between the second division time and a third pre-starting time, the third pre-starting time is a time when the MediaMuxer starts the second wrapper again (of course, the time when the first wrapper starts each time may also define a first pre-starting time, i.e., between steps S153 to S132 shown in fig. 3 or fig. 4, i.e., after the second wrapper stops packaging this time and before the second wrapper starts next time), the MediaMuxer may perform the following steps: releasing the second wrapper and re-creating the second wrapper to start the second wrapper in time when the third first preset time is reached (i.e., corresponding to step S132).
As can be seen, in this embodiment, when the first encapsulator completes one encapsulation and needs to perform resource release and reconstruction, the second encapsulator has already taken over its encapsulation work, so that the release and reconstruction operations of the first encapsulator can be completed in the working process of the second encapsulator, and the second encapsulator is ready to take over the second encapsulator to perform the encapsulation work when the second encapsulator completes one encapsulation and needs to perform resource release and reconstruction. Meanwhile, the release and the reconstruction are executed once after the packaging process (obtaining a video file) is finished by the packaging device every time, so that the phenomena of system errors and the like caused by the fact that the same packaging device occupies the same system resource for a long time can be avoided.
In a possible embodiment of the present invention, the saving of the video file in steps S134 and S154 of the above-mentioned seamless video recording method may specifically be: storing the video files into a preset cache unit; correspondingly, the seamless video recording method further comprises the following steps: and transferring the video files in the cache unit to a hard disk or a memory card.
In practical applications, the data writing speed of a storage unit (e.g., a Read Only Memory (ROM) of the terminal device, or various embedded Memory cards such as an sd (secure digital) card and a TF (T-Flash) card) for storing a video file in the terminal device may be slow, and if the writing speed is lower than the frame rate of the video file, the video file may be difficult to be stored in the storage unit in time, or the video file may be correctly stored only by reducing the video frame rate. In view of this, in the embodiment of the present invention, the video files are cached in a buffer (generally, a Random Access Memory (RAM)) with a relatively high writing speed in the terminal device, and then the cached video files are sequentially transferred to the storage unit.
Therefore, the embodiment of the invention solves the problem that the video frame rate is not matched with the writing speed of the storage unit by adding the first-level cache, and ensures that the video files are correctly stored in the corresponding storage unit on the premise of not reducing the video frame rate.
Fig. 5 is a schematic structural diagram of a device for implementing seamless video recording in an Android system application layer according to an embodiment of the present invention. Referring to fig. 5, the apparatus includes at least:
the encoding control unit 510 is configured to encode the preview video data and the audio data through an encoder interface MediaCodec to obtain audio/video encoded data; the preview video data is preview frame data which is used for previewing and displaying on a screen in the original video data acquired by the video acquisition device;
a packaging control unit 520, configured to start a first packager through a packager interface mediamultiplexer, and perform a packaging operation on the audio/video encoded data through the first packager; before the first wrapper stops wrapping, starting a second wrapper through the MediaMuxer, and performing wrapping operation on the audio and video coding data through the second wrapper; restarting the first encapsulator by the mediamultiplexer before the second encapsulator stops encapsulating.
According to the technical scheme, the method is different from the integrated packaging type video recording process of the traditional video recording interface MediaRecoder, the embodiment of the invention firstly adopts an encoder interface MediaCodec to encode preview video data and audio data to obtain audio and video encoding data, then adopts a packager interface MediaMuxer to package the audio and video encoding data, and the packaging process comprises the following steps: the two packers alternately execute the packing operation of the audio and video coding data, and before one packers stops the packing process once, the other packers are started in advance and start the packing operation, so that the two adjacent video files have overlapped parts, and the second leakage phenomenon can be completely eliminated. In this embodiment, the encoder interface MediaCodec is operated after the start of video recording until the whole seamless video recording process is finished, and there is no need to cyclically perform the operations of releasing and reconstructing the encoder resources.
In a feasible embodiment of the present invention, the starting of the first wrapper by the wrapper control unit 520 specifically includes: creating a first video file, and writing decoding configuration data into the first video file; correspondingly, the starting of the second wrapper by the wrapper control unit 520 may specifically include: and creating a second video file, and writing decoding configuration data into the second video file.
In a possible embodiment of the present invention, to enable starting the second wrapper before the first wrapper stops wrapping, the wrapping control unit 520 may specifically be configured to: when the current time reaches a first pre-starting time, starting a second wrapper through the MediaMuxer; the advance of the first pre-starting time relative to the first segmentation time is a first preset interval time, and the first segmentation time is the time when the first wrapper stops wrapping.
Likewise, to enable the first wrapper to be started before the second wrapper stops wrapping, the wrapping control unit 520 may specifically be configured to: when the current time reaches a second pre-starting time, starting a first wrapper through the MediaMuxer; and the advance of the second pre-starting time relative to the second division time is a second preset interval time, and the second division time is the time when the second wrapper stops wrapping.
In a possible embodiment of the present invention, the seamless recording apparatus may further include:
a key frame detection unit, configured to detect whether a key frame arrives in the audio/video encoded data after the first encapsulator is started by the encapsulation control unit 520, and trigger the encapsulation control unit 520 to continue to execute the step of performing the encapsulation operation on the audio/video encoded data by the first encapsulator after the key frame arrives;
and/or, the audio/video encoding device is configured to detect whether a key frame arrives in the audio/video encoding data after the encapsulation control unit 520 starts a second encapsulator, and trigger the encapsulation control unit 520 to execute the step of performing an encapsulation operation on the audio/video encoding data through the second encapsulator after the key frame arrives.
According to the embodiment of the invention, after the encapsulator is started, the key frame is detected, and the encapsulation operation is started when the key frame is detected, so that the first frame of each video file is the key frame, and the mosaic effect can be avoided when the beginning part of the video file is played.
In a possible embodiment of the present invention, the seamless recording apparatus may further include:
and the lead determining unit is used for determining the first preset interval time and/or the second preset interval time according to the interval time of two adjacent key frames.
In one possible embodiment of the invention, the encapsulation control unit 520 is further configured to: at any time between the first segmentation time and a fourth pre-starting time, releasing the first wrapper, and re-creating the first wrapper through the mediaplayer; and, at any time between the second split time and a third pre-start time, releasing the second wrapper and recreating the second wrapper by the mediamultiplexer; the fourth pre-starting time is a time when the mediamultiplexer restarts the first encapsulator, and the third pre-starting time is a time when the mediamultiplexer restarts the second encapsulator.
As can be seen, in this embodiment, when the first encapsulator completes one encapsulation and needs to perform resource release and reconstruction, the second encapsulator has already taken over its encapsulation work, so that the release and reconstruction operations of the first encapsulator can be completed in the working process of the second encapsulator, and the second encapsulator is ready to take over the second encapsulator to perform the encapsulation work when the second encapsulator completes one encapsulation and needs to perform resource release and reconstruction. Meanwhile, the release and the reconstruction are executed once after the packaging process (obtaining a video file) is finished by the packaging device every time, so that the phenomena of system errors and the like caused by the fact that the same packaging device occupies the same system resource for a long time can be avoided.
In a possible embodiment of the present invention, the seamless recording apparatus may further include:
and the cache control unit is used for respectively storing the video files obtained by packaging the first packaging device and the video files obtained by packaging the second packaging device into the cache unit, and then transferring the video files in the cache unit to a hard disk or a memory card.
The embodiment of the invention solves the problem that the video frame rate is not matched with the writing speed of the storage unit by adding the first-level cache, and ensures that the video files are correctly stored in the corresponding storage unit on the premise of not reducing the video frame rate.
Fig. 6 is a block diagram of a terminal device with a seamless video recording function according to an embodiment of the present invention. Referring to fig. 6, the terminal device includes: camera 610, microphone 620, processor 630, and memory 640.
Wherein the memory has stored therein a computer program operable on the processor; the processor is connected with the camera, the microphone and the memory through a preset interface.
The camera 610 is used for collecting video data; the microphone 620 is used for collecting audio data; the processor 630 is configured to execute the computer program stored in the memory, so as to enable the terminal device to implement the seamless video recording method according to any of the foregoing embodiments.
It can be seen that, unlike the integrated packaging type video recording process of the conventional video recording interface mediadecoder, the embodiment of the present invention first uses the coder interface MediaCodec to encode the preview video data and the audio data to obtain the audio/video encoded data, and then uses the packager interface mediamux to package the audio/video encoded data, and the packaging process is: the two packers alternately execute the packing operation of the audio and video coding data, and before one packers stops the packing process once, the other packers are started in advance and start the packing operation, so that the two adjacent video files have overlapped parts, and the second leakage phenomenon can be completely eliminated. In this embodiment, the encoder interface MediaCodec is operated after the start of video recording until the whole seamless video recording process is finished, and there is no need to cyclically perform the operations of releasing and reconstructing the encoder resources.
For convenience of description, the above devices are described as being divided into various units by function, and are described separately. Of course, the functions of the units may be implemented in the same software and/or hardware or in a plurality of software and/or hardware when implementing the invention.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for apparatus or system embodiments, since they are substantially similar to method embodiments, they are described in relative terms, as long as they are described in partial descriptions of method embodiments. The above-described embodiments of the apparatus and system are merely illustrative, and the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
One of ordinary skill in the art can understand and implement it without inventive effort. The foregoing is merely a detailed description of the invention, and it should be noted that modifications and adaptations by those skilled in the art may be made without departing from the principles of the invention, and should be considered as within the scope of the invention.
Claims (10)
1. A method for realizing seamless video recording in an Android system application layer is characterized by comprising the following steps:
coding preview video data and audio data through a coder interface MediaCodec to obtain audio and video coding data; the preview video data is preview frame data which is used for previewing and displaying on a screen in the original video data acquired by the video acquisition device;
starting a first encapsulator through a encapsulator interface MediaMuxer, and performing encapsulation operation on the audio and video coding data through the first encapsulator;
before the first wrapper stops wrapping, starting a second wrapper through the MediaMuxer, and performing wrapping operation on the audio and video coding data through the second wrapper;
restarting the first encapsulator by the mediamultiplexer before the second encapsulator stops encapsulating.
2. The method of claim 1, wherein starting a second encapsulator by the MediaMuxer before the first encapsulator stops encapsulating comprises:
when the current time reaches a first pre-starting time, starting the second wrapper through the MediaMuxer; the lead of the first pre-starting time relative to the first division time is a first preset interval time, and the first division time is the time when the first wrapper stops wrapping;
restarting, by the mediamultiplexer, the first encapsulator before the second encapsulator stops encapsulating, comprising:
when the current time reaches a second pre-starting time, restarting the first wrapper through the MediaMuxer; and the advance of the second pre-starting time relative to the second division time is a second preset interval time, and the second division time is the time when the second wrapper stops wrapping.
3. The method of claim 2, further comprising:
when the current time reaches the first segmentation time, controlling the first packaging device to stop packaging operation, and storing a video file obtained by packaging the first packaging device;
and when the current time reaches the second segmentation time, controlling the second packaging device to stop packaging operation, and storing the video file obtained by packaging the second packaging device.
4. The method of claim 3, further comprising:
at any time between the first division time and a fourth pre-starting time, the mediamultiplexer further includes, after controlling the first wrapper to stop the wrapping operation and before reaching the fourth pre-starting time: the MediaMuxer releases the first wrapper and recreates the first wrapper, wherein the fourth pre-start time is a time when the MediaMuxer starts the first wrapper again;
at any time between the second division time and a third pre-starting time, after the MediaMuxer controls the second wrapper to stop the wrapping operation and before the third pre-starting time is reached, the MediaMuxer releases the second wrapper and re-creates the second wrapper, wherein the third pre-starting time is a time when the MediaMuxer starts the second wrapper again.
5. The method of claim 2, further comprising:
after a first wrapper is started, detecting whether a key frame arrives in the audio and video coded data, and after the key frame arrives, executing the step of executing a wrapping operation on the audio and video coded data through the first wrapper;
and/or after a second wrapper is started, detecting whether a key frame arrives in the audio and video coded data, and after the key frame arrives, executing the step of performing a wrapping operation on the audio and video coded data through the second wrapper.
6. The method of claim 5, further comprising:
and determining the first preset interval time and/or the second preset interval time according to the interval time of two adjacent key frames.
7. The method of claim 1, wherein the initiating a first wrapper comprises: creating a first video file, and writing decoding configuration data into the first video file;
the initiating a second wrapper, comprising: and creating a second video file, and writing decoding configuration data into the second video file.
8. The utility model provides a realize device of seamless video recording at Android system application layer which characterized in that includes: an encoding control unit and an encapsulation control unit;
the coding control unit is used for coding preview video data and audio data through a coder interface MediaCodec to obtain audio and video coding data; the preview video data is preview frame data which is used for previewing and displaying on a screen in the original video data acquired by the video acquisition device;
the encapsulation control unit is used for starting a first encapsulator through an encapsulator interface mediamultiplexer and executing encapsulation operation on the audio and video coding data through the first encapsulator; before the first wrapper stops wrapping, starting a second wrapper through the MediaMuxer, and performing wrapping operation on the audio and video coding data through the second wrapper; restarting the first encapsulator by the mediamultiplexer before the second encapsulator stops encapsulating.
9. The apparatus of claim 8, further comprising:
the key frame detection unit is used for detecting whether a key frame arrives in the audio and video coded data after the packaging control unit starts a first packaging device, and triggering the packaging control unit to execute the step of performing packaging operation on the audio and video coded data through the first packaging device after the key frame arrives;
and/or, after the second wrapper is started by the wrapping control unit, detecting whether a key frame arrives in the audio and video coded data, and after the key frame arrives, triggering the wrapping control unit to execute the step of performing a wrapping operation on the audio and video coded data through the second wrapper.
10. A terminal device, comprising: the device comprises a camera, a microphone, a processor and a memory; a computer program stored in and on the memory and executable on the processor; the processor is connected with the camera, the microphone and the memory through a preset interface;
the camera is used for collecting video data;
the microphone is used for collecting audio data;
the processor, when executing the computer program, implements the method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710228016.1A CN106998449B (en) | 2017-04-10 | 2017-04-10 | Method and device for realizing seamless video recording in application layer and terminal equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710228016.1A CN106998449B (en) | 2017-04-10 | 2017-04-10 | Method and device for realizing seamless video recording in application layer and terminal equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106998449A CN106998449A (en) | 2017-08-01 |
| CN106998449B true CN106998449B (en) | 2020-02-04 |
Family
ID=59435574
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710228016.1A Active CN106998449B (en) | 2017-04-10 | 2017-04-10 | Method and device for realizing seamless video recording in application layer and terminal equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106998449B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107545618A (en) * | 2017-09-07 | 2018-01-05 | 深圳市麦道微电子技术有限公司 | The mechanism that a kind of seamless circulating in drive recorder is made video recording |
| CN111556351B (en) * | 2020-05-15 | 2022-04-15 | 宁波菊风系统软件有限公司 | RTP file playing system |
| CN112689108A (en) * | 2020-12-22 | 2021-04-20 | 山东快课信息科技有限公司 | System and method for recording and playing blackboard writing in sections based on key frames |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103842936A (en) * | 2011-06-03 | 2014-06-04 | 迈克尔·爱德华·扎莱泰尔 | Recording, editing and combining multiple live video clips and still photographs into a finished composition |
| CN105049919A (en) * | 2015-07-27 | 2015-11-11 | 青岛海信移动通信技术股份有限公司 | Method and device for recording multimedia file |
| CN105430312A (en) * | 2015-11-27 | 2016-03-23 | 福州瑞芯微电子股份有限公司 | Method and system for non-frame-loss segmented storage of video record based on Android system |
| CN106027935A (en) * | 2016-07-13 | 2016-10-12 | 深圳市爱培科技术股份有限公司 | Frame-missing-free segmental video-recording system and implementation method |
-
2017
- 2017-04-10 CN CN201710228016.1A patent/CN106998449B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103842936A (en) * | 2011-06-03 | 2014-06-04 | 迈克尔·爱德华·扎莱泰尔 | Recording, editing and combining multiple live video clips and still photographs into a finished composition |
| US9117483B2 (en) * | 2011-06-03 | 2015-08-25 | Michael Edward Zaletel | Method and apparatus for dynamically recording, editing and combining multiple live video clips and still photographs into a finished composition |
| CN105049919A (en) * | 2015-07-27 | 2015-11-11 | 青岛海信移动通信技术股份有限公司 | Method and device for recording multimedia file |
| CN105430312A (en) * | 2015-11-27 | 2016-03-23 | 福州瑞芯微电子股份有限公司 | Method and system for non-frame-loss segmented storage of video record based on Android system |
| CN106027935A (en) * | 2016-07-13 | 2016-10-12 | 深圳市爱培科技术股份有限公司 | Frame-missing-free segmental video-recording system and implementation method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106998449A (en) | 2017-08-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106998449B (en) | Method and device for realizing seamless video recording in application layer and terminal equipment | |
| CN108848406B (en) | Method and device for recording multimedia file | |
| CN110430441B (en) | Cloud mobile phone video acquisition method, system, device and storage medium | |
| JP2015533059A5 (en) | ||
| CN105007442A (en) | Marking method for continuously recording video and image | |
| CN102456376B (en) | Recording and play method for monitoring device | |
| CN107093436B (en) | Prerecorded audio and video data storage method and device and mobile terminal | |
| CN105049920A (en) | Method and device for recording multimedia files | |
| CN103281500A (en) | Method and device for processing videos | |
| KR100847534B1 (en) | Apparatus and method for determining rendering duration of video frame | |
| CN111522620B (en) | Full liquid crystal instrument of quick demonstration and high frame rate of rendering based on two process HMI | |
| CN114040251A (en) | Audio and video playing method, system, storage medium and computer program product | |
| CN105049904A (en) | Play method and play device of multi-media files | |
| CN103747366A (en) | A method for displaying the playing process of a HLS media file and a player | |
| CN109167951A (en) | Powered-off fault based on Android framework automobile data recorder can play video method | |
| CN114501134B (en) | iOS cross-page video playing method, device, equipment and storage medium | |
| CN105702278A (en) | Method, device and terminal for recording sound of conferences | |
| CN107135364B (en) | Method and device for realizing seamless video recording on native layer and terminal equipment | |
| JP2006319956A5 (en) | ||
| JP2007306258A5 (en) | ||
| WO2023083064A1 (en) | Video processing method and apparatus, electronic device, and readable storage medium | |
| CN108566552B (en) | Multimedia playing method and system suitable for digital set top box | |
| CN103414863A (en) | Method and device for processing video | |
| JP4724551B2 (en) | Moving image processing apparatus and moving image processing method | |
| KR101970787B1 (en) | Video decoding apparatus and method based on android platform using dual memory |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP01 | Change in the name or title of a patent holder | ||
| CP01 | Change in the name or title of a patent holder |
Address after: 266071 Shandong city of Qingdao province Jiangxi City Road No. 11 Patentee after: Qingdao Hisense Mobile Communication Technology Co.,Ltd. Address before: 266071 Shandong city of Qingdao province Jiangxi City Road No. 11 Patentee before: HISENSE MOBILE COMMUNICATIONS TECHNOLOGY Co.,Ltd. |