CN113411661B

CN113411661B - Method, apparatus, device, storage medium and program product for recording information

Info

Publication number: CN113411661B
Application number: CN202110659679.5A
Authority: CN
Inventors: 付强
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2021-06-11
Filing date: 2021-06-11
Publication date: 2023-01-31
Anticipated expiration: 2041-06-11
Also published as: CN113411661A

Abstract

The disclosure provides a method, an apparatus, an electronic device, a storage medium and a computer program product for recording information, and relates to the technical field of media cloud. One embodiment of the method comprises: in the process of playing the target data, acquiring a video frame and an audio frame to be played in the target data through an open graphic library and a video display control; receiving a recording operation for representing at least part of data in the recording target data, and recording at least part of data according to the recording operation; and combining at least part of video frames and audio frames in the data to obtain the recorded information. The method can acquire the video frame and the audio frame in the target data in real time based on the open graphic library and the video display control, does not need to depend on the position of the key frame during recording, avoids the problem of screen splash caused by loss of the key frame, and can record the audio and video data in real time according to the recording operation of a user.

Description

Method, apparatus, device, storage medium and program product for recording information

Technical Field

The present disclosure relates to the field of cloud computing technologies, and further relates to the field of media cloud technologies, and in particular, to a method and an apparatus for recording information, an electronic device, a storage medium, and a computer program product.

Background

In some cases, a user has a need to record audio-video data being played during the playing of the audio-video. For example, a user such as a censor, a monitor, a live broadcast house manager needs to record a video in real time while watching the video. At present, generally, the obtained undecoded audio/video source data is combined to record and store information. The media audio and video coding format, resolution and the like of the recorded information obtained in the mode depend on the original media stream, and the recording start time depends on the position of the key frame.

Disclosure of Invention

The present disclosure provides a method, an apparatus, an electronic device, a storage medium, and a computer program product for recording information.

According to a first aspect, there is provided a method for recording information, comprising: in the process of playing the target data, acquiring a video frame and an audio frame to be played in the target data through an open graphic library and a video display control; receiving a recording operation for representing at least part of data in the recording target data, and recording at least part of data according to the recording operation; and combining at least part of video frames and audio frames in the data to obtain the recorded information.

According to a second aspect, there is provided an apparatus for recording information, comprising: the acquisition unit is configured to acquire video frames and audio frames to be played in the target data through the open graphic library and the video display control in the process of playing the target data; a recording unit configured to receive a recording operation characterizing at least part of the data in the recording target data and record at least part of the data according to the recording operation; and the merging unit is configured to merge at least part of the video frames and the audio frames in the data to obtain the recorded information.

According to a third aspect, there is provided an electronic device comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as described in any one of the implementations of the first aspect.

According to a fourth aspect, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to perform a method as described in any one of the implementations of the first aspect.

According to a fifth aspect, there is provided a computer program product comprising: computer program which, when being executed by a processor, carries out the method as described in any of the implementations of the first aspect.

According to the technology disclosed by the invention, the video frame and the audio frame in the target data can be obtained in real time based on the open graphic library and the video display control, and the video frame and the audio frame do not need to depend on the position of the key frame during recording, so that the problem of screen splash caused by loss of the key frame is avoided, and the audio and video data can be recorded in real time according to the recording operation of a user.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

FIG. 1 is an exemplary system architecture diagram in which one embodiment according to the present disclosure may be applied;

FIG. 2 is a flow diagram of one embodiment of a method for recording information, according to the present disclosure;

fig. 3 is a schematic diagram of an application scenario of the method for recording information according to the present embodiment;

FIG. 4 is a flow diagram of yet another embodiment of a method for recording information according to the present disclosure;

FIG. 5 is a schematic structural diagram of yet another embodiment of a method for recording information according to the present disclosure;

FIG. 6 is a block diagram of one embodiment of an apparatus for recording information according to the present disclosure;

FIG. 7 is a schematic block diagram of a computer system suitable for use in implementing embodiments of the present disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of embodiments of the present disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

In the technical scheme of the disclosure, the acquisition, storage, application and the like of the personal information of the related user all accord with the regulations of related laws and regulations, necessary security measures are taken, and the customs of the public order is not violated.

Fig. 1 illustrates an exemplary architecture 100 to which the disclosed method and apparatus for recording information may be applied.

As shown in fig. 1, the system architecture 100 may include

terminal devices

101, 102, 103, a network 104, and a server 105. The communication connections between the

terminal devices

101, 102, 103 form a topological network, and the network 104 serves to provide a medium for communication links between the

terminal devices

101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.

The

terminal devices

101, 102, 103 may be hardware devices or software that support network connections for data interaction and data processing. When the

terminal devices

101, 102, 103 are hardware, they may be various electronic devices supporting network connection, information acquisition, interaction, display, processing, and the like, including but not limited to smart phones, tablet computers, e-book readers, laptop portable computers, desktop computers, and the like. When the

terminal devices

101, 102, 103 are software, they can be installed in the electronic devices listed above. It may be implemented, for example, as multiple software or software modules to provide distributed services, or as a single software or software module. And is not particularly limited herein.

The server 105 may be a server providing various services, for example, a background processing server that acquires played video frames and audio frames in real time and records audio and video in the process of playing target data by the

terminal devices

101, 102, and 103. As an example, the server 105 may be a cloud server.

The server may be hardware or software. When the server is hardware, it may be implemented as a distributed server cluster formed by multiple servers, or may be implemented as a single server. When the server is software, it may be implemented as multiple pieces of software or software modules (e.g., software or software modules for providing distributed services) or as a single piece of software or software module. And is not particularly limited herein.

It should be further noted that the method for recording information provided by the embodiments of the present disclosure may be executed by a server, may also be executed by a terminal device, and may also be executed by the server and the terminal device in cooperation with each other. Accordingly, each part (for example, each unit) included in the apparatus for recording information may be entirely provided in the server, may be entirely provided in the terminal device, and may be provided in the server and the terminal device, respectively.

It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation. The system architecture may comprise only the electronic device (e.g. server or terminal device) on which the method for recording information operates, when the electronic device on which the method for recording information operates does not require data transmission with other electronic devices.

Referring to fig. 2, fig. 2 is a flowchart of a method for recording information according to an embodiment of the disclosure, where the process 200 includes the following steps:

step 201, in the process of playing the target data, a video frame and an audio frame to be played in the target data are acquired through the open graphic library and the video display control.

In this embodiment, an execution subject (for example, a terminal device or a server in fig. 1) of the method for recording information may acquire a video frame and an audio frame to be played in target data through an open graphics library and a video display control in a process of playing the target data.

The target data may be media data including arbitrary content, including corresponding video data and audio data. By way of example, the target data may be media data such as a movie, a television show, a live video, a web short video, and so on.

OpenGL (Open Graphics Library) is a cross-language, cross-platform application programming interface for rendering 2D (two-dimensional), 3D (three-dimensional) vector Graphics. In the process of playing the target data, the video frame data corresponding to each video frame can be rendered to obtain a corresponding video frame image.

The video display control is a UI (User Interface) control for playing media information. And displaying the video frame data in the playing process of the target data. As an example, the video display control may be a TextureView control, a SurfaceView control.

After the source data of the audio and video is acquired, the execution main body may decode the source data based on decoding manners such as soft decoding and hard decoding, and obtain video data in YUV (Y represents brightness, and U, V represents chroma) format and audio data in PCM (Pulse Code Modulation) format. Then, rendering video frame data in the PCM format corresponding to each video frame through the open graphics library, displaying the video frame through the video display control, and playing audio data in the PCM format through the AudioTrack (a class for managing and playing audio data).

Step 202, receiving a recording operation characterizing at least part of the data in the recording target data, and recording at least part of the data according to the recording operation.

In this embodiment, the execution body receives a recording operation representing at least part of the recording target data, and records at least part of the data according to the recording operation.

The recording operation may be a recording instruction issued by the user based on voice or a behavioral action. As an example, when the user clicks a virtual button displaying "start recording" in a screen displaying target data, the execution main body starts recording audio/video data; when the user clicks the virtual button of "stop recording" displayed in the screen, the execution main body stops recording the audio and video data.

And step 203, combining at least part of video frames and audio frames in the data to obtain the recorded information.

In this embodiment, the execution body may combine at least a part of the video frame and the audio frame in the data to obtain the recording information.

As an example, the execution body may merge a video frame and an audio frame in a one-to-one correspondence in at least part of the recorded data by a Muxer (video multiplexer) to obtain recording information.

With continued reference to fig. 3, fig. 3 is a schematic diagram 300 of an application scenario of the method for recording information according to the present embodiment. In the application scenario of fig. 3, after the user 301 acquires the source data through the terminal device 302, the media data represented by the source data is to be played. In the process of playing the media data, the open graphics library 303 renders video frame data corresponding to each video frame in the media data, and the video frame data and the audio frame data are played through the video display control 304, so that the open graphics library 303 and the video display control 304 acquire the video frame and the audio frame to be played in the target data. During playback, the user 301 requests recording of media data, and issues a recording request to the terminal device. The terminal device 302 receives a recording operation characterizing at least part of data in the user recording target data, and records at least part of data according to the recording operation characterized by the recording request. And combining at least part of video frames and audio frames in the data to obtain the recorded information.

In the embodiment, the video frames and the audio frames in the target data can be obtained in real time based on the open graphic library and the video display control, the position of the key frame is not required to be relied on during recording, the problem of screen splash caused by key frame loss is avoided, and the audio and video data can be recorded in real time according to the recording operation of a user.

In some optional implementations of this embodiment, the executing main body may execute the step 201 as follows:

first, a video display control is monitored in response to playing target data.

In this implementation, a video display control TextureView is taken as an example. TextureView has two related classes, surface texture control and Surface control. The essence of the Surface control is a segment of drawing buffer in the memory. The SurfaceTexture control is used to capture image frames in a video stream, which may be camera preview or video decoding data. The SurfaceTexture can be used as a target video data output object of the classes android, hardware, camera2, mediaCodec, mediaPlayer, and can call the updatetex image () method to update the current frame from the video stream data.

The self-contained SurfaceTexture control in the TextureView cannot acquire a refresh callback of a video frame and cannot acquire the video frame for operation, so that a SurfaceTextureListener monitor of the TextureView is set to monitor the life cycle of the TextureView in response to the generation of Activity (APP interface Activity) of play target data to create a new SurfaceTexture control.

Listening to the lifecycle of TextureView by the setSurfaceTextureListener includes:

onsurface textureavailable, which represents that TextureView is available;

onsurface texturizezaChanged, which represents that the size of TextureView changes;

ontsurfaceTextureUpdated, which represents that the TextureView has view data updated;

onSurfaceTextureDestroyed, characterizing TextureView destruction;

setOnFrameAvailableListener, which characterizes the callback after the end of listening to the TextureView to draw each frame of video frame.

Second, in response to determining that the video display control is available, environmental information of the open graphics library is initialized.

In this embodiment, the environment information of OpenGL is initialized in the surface texture list. The Context information includes creating EGLContext (Enterprise Generation Language Context), loading Shader (Shader), and creating new Texture Identity Document (Texture ID). The breadth and height of the SurfaceTexture is updated in the SurfaceTextureListener.

And thirdly, creating a surface texture control and a first drawing surface control in the video display control according to the texture identification in the environment information.

In this implementation manner, the execution main body creates a new SurfaceTexture control (surface texture control) through a texture identifier in the environment information, and sets an OnFrameAvailableListener listener of the SurfaceTexture control to monitor a callback after the TextureView draws each frame of video frame; and creates a first Surface control (first drawing Surface control) through the Surface texture control. The newly created surface texture control replaces the own surface texture control in the TextureView.

And fourthly, in the process of playing the target data, transmitting the video frame data decoded in the target data into the first drawing surface control, and drawing the video frame data corresponding to the video frame to be played through the open graphic library.

In the implementation mode, a newly-built Surface control through a Surface texture control is transmitted to a player of a c/c + + layer through JNI (Java Native Interface ), after video source data are read in the player, the video source data are decoded to obtain video data in a YUV format, the newly-built Surface control (first drawing Surface control) is filled with the video data in the YUV format, updating of each video frame in the first drawing Surface control can call back an onFrameAvailable () method of an OnFrameAvailableListener monitor, and then OpenGL is called in the call-back to draw the video data in the YUV format.

Fifthly, when the audio frame in the target data is played, the audio frame data corresponding to the audio frame to be played is transmitted into the audio encoder through the audio callback interface of the video display control.

In the implementation mode, an audio callback interface OnReceiveAudioDataCallback of the video display control is registered to a player of a c/c + + layer through JNI. And (3) when the decoded audio PCM source data read in the player is written into the system AudioTrack to play the audio data, calling back the audio PCM source data through the OnReceiveAudioDataCallback interface and transmitting the audio PCM source data into an audio encoder.

In the implementation mode, a specific mode of acquiring the video frame and the audio frame to be played in the target data through the open graphic library and the video display control can be used for acquiring the original audio and video data in real time while playing the audio and video data.

In some optional implementations of this embodiment, the executing main body executes the step 202 by:

in response to the start of the recording operation, drawing video frame data corresponding to a video frame to be played through the open graphics library, simultaneously drawing the video frame data on a second drawing surface control corresponding to the video encoder, encoding the drawn video frame data through the video encoder, and recording the video frame; and coding the received audio frame data through an audio coder, and recording the audio frame.

When a video is played, a preview frame needs to be drawn on a Surface control corresponding to video preview; after the recording of the audio and video data is started, the video data which is the same as the preview frame picture is drawn off the screen on the Surface control of the MediaCodec (a system API) at the same time.

And after the video encoder and the audio encoder obtain video frame data and audio frame data, encoding the audio and video frames. Specifically, the video encoder thread runs as follows: dequeuoutputbuffer () → getOutputBuffer (); meanwhile, the audio encoder thread runs as follows: dequeueInputBuffer () → queueInputBuffer () → dequeuOutputBuffer () → getOutputBuffer ().

In the implementation mode, a specific recording mode for the audio and video data is provided, and the convenience of recording the audio and video data is improved.

In some optional implementations of this embodiment, the executing main body may execute the step 203 by: and combining at least part of video frames and audio frames in the data to obtain the recording information in the preset format.

The preset format may be a file in any format. As examples, the preset format includes MP4 (Media Portable 4, portable Media 4), RMVB (real Media Variable Bitrate), MKV (Multimedia Container), AVI (Audio Video Interleaved).

In the implementation mode, the recorded audio and video data is original frame data, format processing can be performed on the original frame data to obtain a file with a preset format, and flexibility of generating the recording information is improved.

In some optional implementation manners of this embodiment, after the execution main body obtains the recording information, the execution main body may further perform data processing on at least part of the data to obtain at least part of the processed data.

In this implementation, at least part of the recorded data may be processed by any data processing method, including but not limited to adding a watermark, a filter, and other data processing methods. In the implementation mode, at least part of recorded data can be flexibly processed to obtain required recording information, and the diversity of the obtained recording information is improved.

With continuing reference to FIG. 4, an exemplary flow 400 of one method embodiment for recording information is shown in accordance with the methods of the present disclosure, including the steps of:

and step 401, responding to the playing target data, and monitoring a video display control.

In response to determining that the video display control is available, environment information of the open graphics library is initialized, STEP 402.

Step 403, creating a surface texture control and a first drawing surface control in the video display control according to the texture identifier in the environment information.

Step 404, in the process of playing the target data, transmitting the video frame data decoded in the target data into the first drawing surface control, and drawing the video frame data corresponding to the video frame to be played through the open graphics library.

Step 405, while playing the audio frame in the target data, transmitting the audio frame data corresponding to the audio frame to be played to the audio encoder through the audio callback interface of the video display control.

Step 406, in response to the start of the recording operation, drawing the video frame data corresponding to the video frame to be played through the open graphics library, simultaneously drawing the video frame data on the second drawing surface control corresponding to the video encoder, and encoding the drawn video frame data through the video encoder to record the video frame.

Step 407, the received audio frame data is encoded by the audio encoder, and the audio frame is recorded.

And step 408, performing data processing on at least part of the data to obtain at least part of processed data.

Step 409, combining at least part of the video frames and audio frames in the data to obtain the recording information in the preset format.

As can be seen from this embodiment, compared with the embodiment corresponding to fig. 2, the flow 400 of the method for recording information in this embodiment specifically illustrates an acquisition process of an audio/video frame, a recording process of information, and a processing process of at least part of recorded data, so that flexibility, convenience, and diversity of information recording are improved.

With continued reference to fig. 5, to further illustrate the method for recording information of the present disclosure, an embodiment 500 is described as follows:

1. acquiring source media data comprising source video 501 and source audio 502; in a player under a C/C + + layer, decoding a source video 501 in a hard decoding or soft decoding mode to obtain video data in a YUV format; for the source audio 502, decoding is performed in a soft decoding manner, so as to obtain audio data in a PCM format, so that the audio data is played through an audio track while the video data is played.

2. In response to the Activity of the play source video 501, the lifecycle of the textreviview control 504 is listened through the setsurfacetextilelistener 503, and the context information of OpenGL505 is initialized in the surfacetextilenceronener.

3. According to texture identification TextureID in the environment information of OpenGL505, a new SurfaceTexture control 506 is created, and a Surface control 507 is newly created through the SurfaceTexture control 506. The created Surface texture control 506 and Surface control 507 replace the self-contained Surface texture control and Surface control in the TextureView control 504.

4. Transmitting the Surface control 507 into a layer of a c/c + + layer through JNI, reading YUV video data in the layer, filling the YUV video data into the Surface control 507, and calling an onFrameAvailable () method of an OnFrameAvailableListener monitor 508 for updating each video frame in the Surface control 507, then calling OpenGL505 to draw a preview frame of the YUV video data in the call-back, and drawing the YUV video data in OpenGL 505.

5. In response to the recording operation of the receiving user, the Surface control 509 corresponding to the video encoder is acquired, the same video frame data as the preview frame picture is drawn on the Surface509 by MediaCodec off-screen, and the video frame data is transmitted to the video track channel.

6. Meanwhile, the received audio frame data is encoded by an audio encoder, audio frames are recorded, and video frame data is transmitted to an audio track channel.

7. Finally, the Muxer merges the video frame data in the video track channel and the audio frame data in the audio track channel to obtain the merged recording information.

With continuing reference to fig. 6, as an implementation of the methods illustrated in the above figures, the present disclosure provides one embodiment of an apparatus for recording information, which corresponds to the method embodiment illustrated in fig. 2, and which is particularly applicable in various electronic devices.

As shown in fig. 6, the apparatus for recording information includes: an obtaining unit 601 configured to obtain a video frame and an audio frame to be played in target data through an open graphics library and a video display control in a process of playing the target data; a recording unit 602 configured to receive a recording operation characterizing at least part of data in the recording target data, and record at least part of data according to the recording operation; a merging unit 603 configured to merge at least part of the video frames and audio frames in the data to obtain the recording information.

In some optional implementations of this embodiment, the obtaining unit 601 is further configured to: monitoring a video display control in response to the playing target data; initializing environmental information of an open graphics library in response to determining that the video display control is available; according to the texture identification in the environment information, a surface texture control and a first drawing surface control are established in the video display control; in the process of playing the target data, transmitting the video frame data decoded in the target data into a first drawing surface control, and drawing the video frame data corresponding to the video frame to be played through an open graphic library; and transmitting the audio frame data corresponding to the audio frame to be played into an audio encoder through an audio callback interface of the video display control while the audio frame in the target data is played.

In some optional implementations of this embodiment, the recording unit 602 is further configured to: in response to the start of recording operation, drawing video frame data corresponding to a video frame to be played through the open graphic library, simultaneously drawing the video frame data on a second drawing surface control corresponding to the video encoder, encoding the drawn video frame data through the video encoder, and recording the video frame; and coding the received audio frame data through an audio coder, and recording the audio frame.

In some optional implementations of this embodiment, the merging unit 603 is further configured to: and combining at least part of video frames and audio frames in the data to obtain the recording information in the preset format.

In some optional implementations of this embodiment, the apparatus further includes: and a processing unit (not shown in the figure) configured to perform data processing on at least part of the data to obtain at least part of the processed data.

In the embodiment, the video frame and the audio frame in the target data can be acquired in real time based on the open graphic library and the video display control, and the video frame and the audio frame do not need to be dependent on the position of the key frame during recording, so that the problem of screen splash caused by loss of the key frame is avoided, and the audio and video data can be recorded in real time according to the recording operation of a user.

According to an embodiment of the present disclosure, the present disclosure also provides an electronic device including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to cause the at least one processor to perform the method for recording information as described in any of the above embodiments when executed by the at least one processor.

According to an embodiment of the present disclosure, there is also provided a readable storage medium storing computer instructions for enabling a computer to implement the method for recording information described in any of the above embodiments when executed.

The embodiments of the present disclosure provide a computer program product, which when executed by a processor is capable of implementing the method for recording information described in any of the embodiments above.

FIG. 7 illustrates a schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 7, the device 700 comprises a computing unit 701, which may perform various suitable actions and processes according to a computer program stored in a Read Only Memory (ROM) 702 or a computer program loaded from a storage unit 708 into a Random Access Memory (RAM) 703. In the RAM703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM702, and the RAM703 are connected to each other by a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.

Various components in the device 700 are connected to the I/O interface 705, including: an input unit 706 such as a keyboard, a mouse, or the like; an output unit 707 such as various types of displays, speakers, and the like; a storage unit 708 such as a magnetic disk, optical disk, or the like; and a communication unit 709 such as a network card, modem, wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.

Computing unit 701 may be a variety of general purpose and/or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The calculation unit 701 executes the respective methods and processes described above, such as a method for recording information. For example, in some embodiments, the method for recording information may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of a computer program may be loaded onto and/or installed onto device 700 via ROM702 and/or communications unit 709. When the computer program is loaded into the RAM703 and executed by the computing unit 701, one or more steps of the method for recording information described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured by any other suitable means (e.g., by means of firmware) to perform the method for recording information.

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the Internet.

The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The Server may be a cloud Server, which is also called a cloud computing Server or a cloud host, and is a host product in a cloud computing service system, so as to solve the defects of high management difficulty and weak service extensibility in the conventional physical host and Virtual Private Server (VPS) service.

According to the technical scheme of the embodiment of the disclosure, the video frame and the audio frame in the target data can be obtained in real time based on the open graphic library and the video display control, and when recording, the position of the key frame is not required to be relied on, so that the problem of screen splash caused by key frame loss is avoided, and the audio and video data can be recorded in real time according to the recording operation of a user.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in this disclosure may be performed in parallel or sequentially or in a different order, as long as the desired results of the technical solutions provided by this disclosure can be achieved, and are not limited herein.

The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. A method for recording information, comprising:

in the process of playing the target data, acquiring a video frame and an audio frame to be played in the target data through an open graphic library and a video display control, wherein the process comprises the following steps: monitoring the video display control in response to playing the target data; initializing environmental information of the open graphics library in response to determining that the video display control is available; according to the texture identification in the environment information, a surface texture control and a first drawing surface control are established in the video display control, and the surface texture control and the drawing surface control which are carried in the video display control are replaced; in the process of playing the target data, transmitting the decoded video frame data in the target data into the first drawing surface control, and drawing the video frame data corresponding to the video frame to be played through the open graphic library; transmitting audio frame data corresponding to the audio frame to be played into an audio encoder through an audio callback interface of the video display control while playing the audio frame in the target data;

receiving a recording operation for recording at least part of data in the target data in a characterizing way, and recording the at least part of data according to the recording operation, wherein the recording operation comprises the following steps: in response to the start of recording operation, drawing video frame data corresponding to a video frame to be played through the open graphics library, simultaneously drawing the video frame data on a second drawing surface control corresponding to a video encoder, encoding the drawn video frame data through the video encoder, and recording the video frame; coding the received audio frame data through the audio coder, and recording the audio frame;

and combining the video frame and the audio frame in at least part of data to obtain the recorded information.

2. The method of claim 1, wherein said merging the video frames and the audio frames in the at least part of the data to obtain the recorded information comprises:

and combining the video frame and the audio frame in at least part of the data to obtain the recording information with a preset format.

3. The method of claim 1, wherein after said recording said at least part of data according to said recording operation, further comprising:

and carrying out data processing on at least part of data to obtain at least part of processed data.

4. An apparatus for recording information, comprising:

the acquiring unit is configured to acquire a video frame and an audio frame to be played in target data through an open graphics library and a video display control in the process of playing the target data, and comprises: monitoring the video display control in response to playing the target data; initializing environmental information of the open graphics library in response to determining that the video display control is available; according to the texture identification in the environment information, a surface texture control and a first drawing surface control are established in the video display control, and the surface texture control and the drawing surface control which are carried in the video display control are replaced; in the process of playing the target data, transmitting the video frame data decoded from the target data into the first drawing surface control, and drawing the video frame data corresponding to the video frame to be played through the open graphic library; when the audio frame in the target data is played, transmitting audio frame data corresponding to the audio frame to be played into an audio encoder through an audio callback interface of the video display control;

a recording unit configured to receive a recording operation characterizing at least a part of the target data, and record the at least a part of the data according to the recording operation, including: in response to the start of recording operation, drawing video frame data corresponding to a video frame to be played through the open graphics library, simultaneously drawing the video frame data on a second drawing surface control corresponding to a video encoder, encoding the drawn video frame data through the video encoder, and recording the video frame; coding the received audio frame data through the audio coder, and recording the audio frame;

and the merging unit is configured to merge the video frame and the audio frame in the at least part of data to obtain the recording information.

5. The apparatus of claim 4, wherein the merging unit is further configured to:

6. The apparatus of claim 4, further comprising:

and the processing unit is configured to perform data processing on the at least part of data to obtain processed at least part of data.

7. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein, the first and the second end of the pipe are connected with each other,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-3.

8. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-3.