CN113411661B - Method, apparatus, device, storage medium and program product for recording information - Google Patents

Abstract

The disclosure provides a method, device, electronic equipment, storage medium and computer program product for recording information, and relates to the technical field of media cloud. A specific embodiment of the method includes: during the process of playing the target data, acquiring the video frame and audio frame to be played in the target data through the open graphics library and the video display control; receiving a record representing and recording at least part of the data in the target data operation, and record at least part of the data according to the recording operation; combine video frames and audio frames in at least part of the data to obtain recording information. Based on the open graphics library and the video display control, this disclosure can obtain the video frame and audio frame in the target data in real time. When recording, it does not need to depend on the position of the key frame, and avoids the blurred screen problem caused by the loss of the key frame. According to The user's recording operation records audio and video data in real time.

Description

Method, device, device, storage medium and program product for recording information

technical field

The present disclosure relates to the technical field of cloud computing, further to the technical field of media cloud, and in particular to a method, device, electronic device, storage medium and computer program product for recording information.

Background technique

In some cases, the user needs to record the played audio and video data during the playback of the audio and video. For example, users such as inspectors, monitors, and live room management need to record the video they see in real time while watching the video. At present, the obtained undecoded audio and video source data are generally combined to record and save the information. The media audio and video encoding format and resolution of the recorded information obtained in this way depend on the original media stream, and the recording start time depends on the key frame position.

Contents of the invention

The present disclosure provides a method, apparatus, electronic equipment, storage medium, and computer program product for recording information.

According to the first aspect, there is provided a method for recording information, including: in the process of playing the target data, obtaining the video frame and audio frame to be played in the target data through the open graphics library and the video display control; receiving the characterization Recording the recording operation of at least part of the data in the target data, and recording at least part of the data according to the recording operation; combining video frames and audio frames in at least part of the data to obtain recording information.

According to the second aspect, there is provided a device for recording information, including: an acquisition unit configured to acquire video frames to be played in the target data through an open graphics library and a video display control during the process of playing the target data and an audio frame; a recording unit configured to receive a recording operation representing at least part of the data in the recording target data, and record at least part of the data according to the recording operation; a merging unit configured to merge at least a video frame and an audio frame in the at least part of the data , get record information.

According to a third aspect, there is provided an electronic device, comprising: at least one processor; and a memory communicatively connected to at least one processor; wherein, the memory stores instructions executable by at least one processor, and the instructions are processed by at least one processor executed by a processor, so that at least one processor can execute the method described in any implementation manner of the first aspect.

According to a fourth aspect, there is provided a non-transitory computer-readable storage medium storing computer instructions, the computer instructions are used to cause a computer to execute the method described in any implementation manner of the first aspect.

According to a fifth aspect, a computer program product is provided, including: a computer program, and when the computer program is executed by a processor, the method described in any implementation manner of the first aspect is implemented.

According to the technology of the present disclosure, the video frame and audio frame in the target data can be obtained in real time based on the open graphics library and the video display control. When recording, it does not need to depend on the position of the key frame, avoiding the blurred screen caused by the loss of the key frame Question, audio and video data can be recorded in real time according to the user's recording operation.

It should be understood that what is described in this section is not intended to identify key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will be readily understood through the following description.

Description of drawings

The accompanying drawings are used to better understand the present solution, and do not constitute a limitation to the present disclosure. in:

FIG. 1 is an exemplary system architecture diagram to which an embodiment of the present disclosure can be applied;

Figure 2 is a flowchart of one embodiment of a method for recording information according to the present disclosure;

FIG. 3 is a schematic diagram of an application scenario of a method for recording information according to this embodiment;

FIG. 4 is a flowchart of another embodiment of a method for recording information according to the present disclosure;

Fig. 5 is a schematic structural diagram of another embodiment of a method for recording information according to the present disclosure;

Fig. 6 is a structural diagram of an embodiment of a device for recording information according to the present disclosure;

FIG. 7 is a schematic structural diagram of a computer system suitable for implementing the embodiments of the present disclosure.

Detailed ways

Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and they should be regarded as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

In the disclosed technical solution, the acquisition, storage, and application of user personal information involved are in compliance with relevant laws and regulations, necessary confidentiality measures have been taken, and they do not violate public order and good customs.

FIG. 1 shows an exemplary architecture 100 to which the method and apparatus for recording information of the present disclosure can be applied.

As shown in FIG. 1 , a system architecture 100 may include terminal devices 101 , 102 , 103 , a network 104 and a server 105 . The communication connections between the terminal devices 101 , 102 , and 103 constitute a topological network, and the network 104 is used to provide a communication link medium between the terminal devices 101 , 102 , 103 and the server 105 . Network 104 may include various connection types, such as wires, wireless communication links, or fiber optic cables, among others.

The terminal devices 101, 102, and 103 may be hardware devices or software that support network connections for data interaction and data processing. When the terminal devices 101, 102, 103 are hardware, they can be various electronic devices that support network connection, information acquisition, interaction, display, processing and other functions, including but not limited to smartphones, tablet computers, e-book readers, Laptops and desktop computers and more. When the terminal devices 101, 102, 103 are software, they can be installed in the electronic devices listed above. It can be implemented, for example, as a plurality of software or software modules for providing distributed services, or as a single software or software module. No specific limitation is made here.

The server 105 may be a server that provides various services, such as a background processing server that obtains the played video frames and audio frames in real time and performs audio and video recording during the process of playing target data by the terminal devices 101 , 102 , and 103 . As an example, server 105 may be a cloud server.

It should be noted that the server may be hardware or software. When the server is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software or software modules (such as software or software modules for providing distributed services), or as a single software or software module. No specific limitation is made here.

It should also be noted that the method for recording information provided by the embodiments of the present disclosure may be executed by a server, may also be executed by a terminal device, and may also be executed by the server and the terminal device in cooperation with each other. Correspondingly, each part (for example, each unit) included in the apparatus for recording information may be all set in the server, all may be set in the terminal device, or may be respectively set in the server and the terminal device.

It should be understood that the numbers of terminal devices, networks and servers in Fig. 1 are only illustrative. According to the implementation needs, there can be any number of terminal devices, networks and servers. When the electronic device on which the method for recording information operates does not need to perform data transmission with other electronic devices, the system architecture may only include the electronic device on which the method for recording information operates (such as a server or terminal device) ).

Please refer to FIG. 2. FIG. 2 is a flow chart of a method for recording information provided by an embodiment of the present disclosure, wherein the process 200 includes the following steps:

Step 201 , during the process of playing the target data, the video frame and audio frame to be played in the target data are obtained through the open graphics library and the video display control.

In this embodiment, the execution subject of the method for recording information (for example, the terminal device or server in Figure 1) can obtain the target data to be played through the open graphics library and the video display control during the process of playing the target data. video frames and audio frames.

The target data may be media data including arbitrary content, including corresponding video data and audio data. As an example, the target data may be media data such as movies, TV dramas, live videos, and short online videos.

OpenGL (Open Graphics Library, Open Graphics Library) is a cross-language, cross-platform application programming interface for rendering 2D (two-dimensional) and 3D (three-dimensional) vector graphics. During the playing process of the target data, the video frame data corresponding to each video frame may be rendered to obtain the corresponding video frame image.

The video display control is a UI (User Interface, user interface) control for playing media information. During the playback of target data, it is used to display video frame data. As an example, the video display control may be a TextureView control or a SurfaceView control.

After obtaining the source data of audio and video, firstly, the above-mentioned executive body can decode the source data based on decoding methods such as soft decoding and hard decoding, and obtain video data in YUV (Y represents brightness, U and V represent chroma) format and audio data in PCM (Pulse Code Modulation, Pulse Code Modulation) format. Then, render the video frame data in PCM format corresponding to each video frame through the open graphics library, display the video frame through the video display control, and play the audio data in PCM format through AudioTrack (the class that manages and plays audio data).

Step 202, receiving a recording operation representing at least part of the data in the recording target data, and recording at least part of the data according to the recording operation.

In this embodiment, the execution subject receives a recording operation representing at least part of the data in the recording target data, and records at least part of the data according to the recording operation.

Wherein, the recording operation may be a recording instruction issued by the user based on voice or behavior. As an example, when the user clicks the virtual button of "Start Recording" displayed on the screen displaying the target data, the above-mentioned execution subject starts recording audio and video data; when the user clicks the virtual button of "Stop Recording" displayed on the screen, the above-mentioned execution subject stops recording Audio and video data.

Step 203, combining video frames and audio frames in at least part of the data to obtain recording information.

In this embodiment, the execution subject may combine video frames and audio frames in at least part of the data to obtain record information.

As an example, the execution subject may use a Muxer (video multiplexer) to combine the one-to-one corresponding video frames and audio frames in at least part of the recorded data to obtain the recording information.

Continue referring to FIG. 3 , which is a schematic diagram 300 of an application scenario of the method for recording information according to this embodiment. In the application scenario of FIG. 3 , after the user 301 obtains the source data through the terminal device 302 , he wants to play the media data represented by the source data. In the process of playing media data, the video frame data corresponding to each video frame in the media data is rendered by the open graphics library 303, and the video frame data and audio frame data are played by the video display control 304, so that the open graphics library 303 And the video display control 304 obtains the video frame and audio frame to be played in the target data. During playing, the user 301 generates a demand for recording media data, and sends a recording request to the terminal device. The terminal device 302 receives a recording operation representing at least part of the data in the user's recording target data, and records at least part of the data according to the recording operation represented by the recording request. Combining video frames and audio frames in at least part of the data to obtain recording information.

In this embodiment, based on the open graphics library and the video display control, the video frame and audio frame in the target data can be obtained in real time. When recording, it does not need to depend on the position of the key frame, which avoids the blurred screen problem caused by the loss of the key frame , audio and video data can be recorded in real time according to the user's recording operation.

In some optional implementation manners of this embodiment, the above execution subject may perform the above step 201 in the following manner:

First, the video display controls are listened for in response to playing object data.

In this implementation manner, the video display control TextureView is taken as an example. TextureView has two related classes SurfaceTexture control and Surface control. The Surface control is essentially a drawing buffer in memory. The SurfaceTexture control is used to capture image frames in a video stream, which can be camera preview or video decoding data. SurfaceTexture can be used as the target video data output object of android.hardware.camera2, MediaCodec, and MediaPlayer, and the updateTexImage() method can be called to update the current frame from the video stream data.

TextureView comes with a SurfaceTexture control, but the built-in SurfaceTexture control cannot obtain the refresh callback of the video frame, nor can it obtain the video frame for operation. Therefore, the SurfaceTextureListener listener of TextureView is set to respond to the Activity (APP Interface activity) generated, monitor the life cycle of TextureView to create a new SurfaceTexture control.

The life cycle of listening to TextureView through setSurfaceTextureListener includes:

onSurfaceTextureAvailable, indicating that TextureView is available;

onSurfaceTextureSizeChanged, indicating that the size of TextureView has changed;

onSurfaceTextureUpdated, indicating that TextureView has view data updated;

onSurfaceTextureDestroyed, which represents the destruction of TextureView;

setOnFrameAvailableListener, which represents the callback after listening to TextureView drawing each frame of video frame.

Second, in response to determining that the video display control is available, environment information for the Open Graphics Library is initialized.

In this embodiment, the OpenGL environment information is initialized in the SurfaceTextureListener.onSurfaceTextureAvailable() callback. Environmental information includes creating EGLContext (Enterprise Generation Language Context, business programming language context), loading Shader (shader), and creating a new TextureID (Texture Identity Document, texture identification). Update the width and height of SurfaceTexture in SurfaceTextureListener.onSurfaceTextureSizeChanged() callback.

Thirdly, create a surface texture control and a first drawing surface control in the video display control according to the texture identification in the environment information.

In this implementation mode, the above-mentioned execution subject creates a new SurfaceTexture control (surface texture control) through the texture identification in the environment information, and sets the OnFrameAvailableListener listener of the SurfaceTexture control to monitor the callback after the TextureView draws each frame of video frame; And create a first Surface control (first drawing surface control) through the SurfaceTexture control. The new surface texture control replaces the surface texture control that comes with TextureView.

Fourth, in the process of playing the target data, the decoded video frame data in the target data is transferred to the first drawing surface control, and the video frame data corresponding to the video frame to be played is drawn through the open graphics library.

In this implementation mode, the Surface control newly created by the SurfaceTexture control is passed to the player (player) of the c/c++ layer through JNI (JavaNative Interface, Java local interface). After the video source data is read in the player, the video source data is The video data in YUV format is decoded, and the video data in YUV format is filled into the new Surface control (the first drawing surface control). The update of each video frame in the first drawing surface control will call back the onFrameAvailable( ) method, and then in the callback, call OpenGL to draw the video data in YUV format.

Fifth, while playing the audio frame in the target data, the audio frame data corresponding to the audio frame to be played is transmitted to the audio encoder through the audio callback interface of the video display control.

In this implementation, the audio callback interface OnReceiveAudioDataCallback of the video display control is registered to the player of the c/c++ layer through JNI. The decoded audio PCM source data read in the player is written into the system AudioTrack to play the audio data, and at the same time, the audio PCM source data is called back to the audio encoder through the OnReceiveAudioDataCallback interface.

In this implementation mode, a specific method of obtaining the video frame and audio frame to be played in the target data through the open graphics library and the video display control can obtain the original audio and video data in real time while playing the audio and video data.

In some optional implementation manners of this embodiment, the above execution subject executes the above step 202 in the following manner:

In response to starting the recording operation, while drawing the video frame data corresponding to the video frame to be played through the open graphics library, draw the video frame data on the second drawing surface control corresponding to the video encoder, and pass the video encoding The device encodes the drawn video frame data to record the video frame; the audio encoder encodes the received audio frame data to record the audio frame.

When the video is playing, it is necessary to draw a preview frame on the Surface control corresponding to the video preview; when recording audio and video data, at the same time draw the same video data as the preview frame off-screen on the Surface control of MediaCodec (a system API).

After the video encoder and the audio encoder obtain the video frame data and the audio frame data, they encode the audio and video frames. Specifically, the video encoder thread operates as follows: dequeueOutputBuffer()→getOutputBuffer(); meanwhile, the audio encoder thread operates as follows: dequeueInputBuffer()→queueInputBuffer()→dequeueOutputBuffer()→getOutputBuffer().

In this implementation manner, a specific recording method for audio and video data is provided, which improves the convenience of recording audio and video data.

In some optional implementations of this embodiment, the execution subject may perform the above step 203 in the following manner: combine video frames and audio frames in at least part of the data to obtain record information in a preset format.

Wherein, the preset format may be a file of any format. As an example, the preset format includes MP4 (MediaPortable 4, portable media 4), RMVB (RealMedia Variable Bitrate, variable bit rate), MKV (Multimedia Container, multimedia container), AVI (Audio Video Interleaved, audio and video interleaving).

In this implementation, the recorded audio and video data is the original frame data, and the original frame data can be format-processed to obtain a file with a preset format, which improves the flexibility of generating record information.

In some optional implementation manners of this embodiment, after obtaining the record information, the execution subject may further perform data processing on at least part of the data to obtain at least part of the processed data.

In this implementation, at least part of the recorded data may be processed by any data processing method, including but not limited to adding watermarks, filters and other data processing methods. In this implementation manner, at least part of the recorded data can be flexibly processed to obtain required record information, which increases the diversity of the obtained record information.

Continuing to refer to FIG. 4 , it shows a schematic flow 400 of an embodiment of a method for recording information according to the method of the present disclosure, including the following steps:

Step 401, monitor the video display control in response to the playing target data.

Step 402, in response to determining that the video display control is available, initialize the environment information of the open graphics library.

Step 403, according to the texture identifier in the environment information, create a surface texture control and a first drawing surface control in the video display control.

Step 404, in the process of playing the target data, transfer the decoded video frame data in the target data to the first drawing surface control, and use the open graphics library to draw the video frame data corresponding to the video frame to be played.

Step 405, while playing the audio frame in the target data, transmit the audio frame data corresponding to the audio frame to be played to the audio encoder through the audio callback interface of the video display control.

Step 406, in response to starting the recording operation, draw the video frame data on the second drawing surface control corresponding to the video encoder while using the open graphics library to draw the video frame data corresponding to the video frame to be played, and The video frame data after rendering is encoded by a video encoder to record the video frame.

Step 407: Encode the received audio frame data by an audio encoder to record the audio frame.

Step 408, performing data processing on at least part of the data to obtain at least part of the data after processing.

Step 409, combining video frames and audio frames in at least part of the data to obtain record information in a preset format.

It can be seen from this embodiment that, compared with the embodiment corresponding to FIG. 2 , the flow 400 of the method for recording information in this embodiment specifically illustrates the acquisition process of audio and video frames, the recording process of information, and all The processing process of at least part of the recorded data improves the flexibility, convenience and diversity of information recording.

Continuing to refer to FIG. 5, in order to further illustrate the method for recording information of the present disclosure, the specific implementation 500 is described as follows:

1. Obtain source media data including source video 501 and source audio 502; in the player under the C/C++ layer, for source video 501, decode through hard decoding or soft decoding to obtain video data in YUV format; for source Audio 502 is decoded by soft decoding to obtain audio data in PCM format, so as to play audio data through audiotrack while playing video data.

2. In response to the Activity playing the source video 501, monitor the life cycle of the TextureView control 504 through the setSurfaceTextureListener listener 503, and initialize the environment information of the OpenGL 505 in the SurfaceTextureListener.onSurfaceTextureAvailable() callback.

3. Create a new SurfaceTexture control 506 according to the texture identifier TextureID in the environment information of the OpenGL 505 , and create a new Surface control 507 through the SurfaceTexture control 506 . The created SurfaceTexture control 506 and Surface control 507 replace the SurfaceTexture control and Surface control in the TextureView control 504 .

4. Pass the Surface control 507 into the player of the c/c++ layer through JNI. After the YUV video data is read in the player, fill the YUV video data into the Surface control 507, and the update of each video frame in the Surface control 507 will be called back The onFrameAvailable() method of the OnFrameAvailableListener listener 508, and then, in the callback, call OpenGL 505 to draw the preview frame of the YUV video data, and draw the video YUV data in OpenGL 505 .

5. In response to receiving the user's recording operation, obtain the surface control 509 corresponding to the video encoder, draw the same video frame data as the preview frame on the Surface 509 off-screen through MediaCodec, and transmit the video frame data to the video track channel.

6. At the same time, the received audio frame data is encoded by the audio encoder, the audio frame is recorded, and the video frame data is transmitted to the audio track channel.

7. Finally, the Muxer merges the video frame data in the video track channel and the audio frame data in the audio track channel to obtain the combined record information.

Continuing to refer to FIG. 6 , as an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a device for recording information, which corresponds to the method embodiment shown in FIG. 2 . The device can be specifically applied to various electronic devices.

As shown in Figure 6, the device for recording information includes: an acquisition unit 601 configured to acquire the video frame and audio frame to be played in the target data through the open graphics library and the video display control during the process of playing the target data The recording unit 602 is configured to receive a recording operation representing at least part of the data in the recording target data, and record at least part of the data according to the recording operation; the merging unit 603 is configured to merge video frames and audio frames in at least part of the data, Get record information.

In some optional implementations of this embodiment, the acquiring unit 601 is further configured to: monitor the video display control in response to the playing target data; and initialize the environment information of the open graphics library in response to determining that the video display control is available; According to the texture identification in the environment information, create a surface texture control and a first drawing surface control in the video display control; in the process of playing the target data, transfer the decoded video frame data in the target data to the first drawing surface control, And use the open graphics library to draw the video frame data corresponding to the video frame to be played; while playing the audio frame in the target data, through the audio callback interface of the video display control, the audio frame corresponding to the audio frame to be played Data is passed into the audio encoder.

In some optional implementations of this embodiment, the recording unit 602 is further configured to: in response to starting the recording operation, draw the video frame data corresponding to the video frame to be played through the open graphics library , draw the video frame data on the second drawing surface control corresponding to the video encoder, and encode the drawn video frame data through the video encoder to record the video frame; use the audio encoder to encode the received audio frame data Encoding to record audio frames.

In some optional implementation manners of this embodiment, the combining unit 603 is further configured to: combine video frames and audio frames in at least part of the data to obtain record information in a preset format.

In some optional implementation manners of this embodiment, the foregoing apparatus further includes: a processing unit (not shown in the figure), configured to perform data processing on at least part of the data, to obtain at least part of the processed data.

In this embodiment, based on the open graphics library and the video display control, the video frame and audio frame in the target data can be obtained in real time. When recording, it does not need to depend on the position of the key frame, which avoids the blurred screen problem caused by the loss of the key frame , audio and video data can be recorded in real time according to the user's recording operation.

According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, the electronic device includes: at least one processor; and a memory connected in communication with the at least one processor; wherein, the memory stores information executable by the at least one processor. An instruction, the instruction is executed by at least one processor, so that the at least one processor can implement the method for recording information described in any of the above embodiments when executed.

According to an embodiment of the present disclosure, the present disclosure also provides a readable storage medium, where computer instructions are stored in the readable storage medium, and the computer instructions are used to enable a computer to implement the method described in any of the above-mentioned embodiments for recording method of information.

An embodiment of the present disclosure provides a computer program product. When the computer program is executed by a processor, the method for recording information described in any of the foregoing embodiments can be implemented.

FIG. 7 shows a schematic block diagram of an example electronic device 700 that may be used to implement embodiments of the present disclosure. Electronic device is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are by way of example only, and are not intended to limit implementations of the disclosure described and/or claimed herein.

As shown in FIG. 7, the device 700 includes a computing unit 701 that can execute according to a computer program stored in a read-only memory (ROM) 702 or loaded from a storage unit 708 into a random-access memory (RAM) 703. Various appropriate actions and treatments. In the RAM 703, various programs and data necessary for the operation of the device 700 can also be stored. The computing unit 701 , ROM 702 , and RAM 703 are connected to each other through a bus 704 . An input/output (I/O) interface 705 is also connected to the bus 704 .

Multiple components in the device 700 are connected to the I/O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a magnetic disk, an optical disk, etc. ; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 709 allows the device 700 to exchange information/data with other devices over a computer network such as the Internet and/or various telecommunication networks.

The computing unit 701 may be various general-purpose and/or special-purpose processing components having processing and computing capabilities. Some examples of computing units 701 include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processing processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 executes the various methods and processes described above, such as a method for recording information. For example, in some embodiments, the method for recording information may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 708 . In some embodiments, part or all of the computer program may be loaded and/or installed on the device 700 via the ROM 702 and/or the communication unit 709 . When the computer program is loaded into RAM 703 and executed by computing unit 701, one or more steps of the method for recording information described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured in any other appropriate way (for example, by means of firmware) to execute the method for recording information.

Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips Implemented in a system of systems (SOC), load programmable logic device (CPLD), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include being implemented in one or more computer programs executable and/or interpreted on a programmable system including at least one programmable processor, the programmable processor Can be special-purpose or general-purpose programmable processor, can receive data and instruction from storage system, at least one input device, and at least one output device, and transmit data and instruction to this storage system, this at least one input device, and this at least one output device an output device.

Program codes for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special purpose computer, or other programmable data processing devices, so that the program codes, when executed by the processor or controller, make the functions/functions specified in the flow diagrams and/or block diagrams Action is implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, portable computer discs, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, compact disk read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing.

To provide for interaction with the user, the systems and techniques described herein can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user. ); and a keyboard and pointing device (eg, a mouse or a trackball) through which a user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and can be in any form (including Acoustic input, speech input or, tactile input) to receive input from the user.

The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., as a a user computer having a graphical user interface or web browser through which a user can interact with embodiments of the systems and techniques described herein), or including such backend components, middleware components, Or any combination of front-end components in a computing system. The components of the system can be interconnected by any form or medium of digital data communication, eg, a communication network. Examples of communication networks include: Local Area Network (LAN), Wide Area Network (WAN) and the Internet.

A computer system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the management difficulties existing in traditional physical host and virtual private server (VPS, Virtual Private Server) services Large and weak business expansion.

According to the technical solution of the embodiment of the present disclosure, the video frame and audio frame in the target data can be obtained in real time based on the open graphics library and the video display control. When recording, it does not need to depend on the position of the key frame, avoiding the loss of the key frame For the blurry screen problem caused, the audio and video data can be recorded in real time according to the user's recording operation.

It should be understood that steps may be reordered, added or deleted using the various forms of flow shown above. For example, each step described in the present disclosure may be executed in parallel, sequentially, or in a different order, as long as the desired result of the technical solution provided by the present disclosure can be achieved, no limitation is imposed herein.

The specific implementation manners described above do not limit the protection scope of the present disclosure. It should be apparent to those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made depending on design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present disclosure shall be included within the protection scope of the present disclosure.

Claims (8)

1. A method for recording information, comprising:

in the process of playing the target data, acquiring a video frame and an audio frame to be played in the target data through an open graphic library and a video display control, wherein the process comprises the following steps: monitoring the video display control in response to playing the target data; initializing environmental information of the open graphics library in response to determining that the video display control is available; according to the texture identification in the environment information, a surface texture control and a first drawing surface control are established in the video display control, and the surface texture control and the drawing surface control which are carried in the video display control are replaced; in the process of playing the target data, transmitting the decoded video frame data in the target data into the first drawing surface control, and drawing the video frame data corresponding to the video frame to be played through the open graphic library; transmitting audio frame data corresponding to the audio frame to be played into an audio encoder through an audio callback interface of the video display control while playing the audio frame in the target data;

receiving a recording operation for recording at least part of data in the target data in a characterizing way, and recording the at least part of data according to the recording operation, wherein the recording operation comprises the following steps: in response to the start of recording operation, drawing video frame data corresponding to a video frame to be played through the open graphics library, simultaneously drawing the video frame data on a second drawing surface control corresponding to a video encoder, encoding the drawn video frame data through the video encoder, and recording the video frame; coding the received audio frame data through the audio coder, and recording the audio frame;

and combining the video frame and the audio frame in at least part of data to obtain the recorded information.

2. The method of claim 1, wherein said merging the video frames and the audio frames in the at least part of the data to obtain the recorded information comprises:

and combining the video frame and the audio frame in at least part of the data to obtain the recording information with a preset format.

3. The method of claim 1, wherein after said recording said at least part of data according to said recording operation, further comprising:

and carrying out data processing on at least part of data to obtain at least part of processed data.

4. An apparatus for recording information, comprising:

the acquiring unit is configured to acquire a video frame and an audio frame to be played in target data through an open graphics library and a video display control in the process of playing the target data, and comprises: monitoring the video display control in response to playing the target data; initializing environmental information of the open graphics library in response to determining that the video display control is available; according to the texture identification in the environment information, a surface texture control and a first drawing surface control are established in the video display control, and the surface texture control and the drawing surface control which are carried in the video display control are replaced; in the process of playing the target data, transmitting the video frame data decoded from the target data into the first drawing surface control, and drawing the video frame data corresponding to the video frame to be played through the open graphic library; when the audio frame in the target data is played, transmitting audio frame data corresponding to the audio frame to be played into an audio encoder through an audio callback interface of the video display control;

a recording unit configured to receive a recording operation characterizing at least a part of the target data, and record the at least a part of the data according to the recording operation, including: in response to the start of recording operation, drawing video frame data corresponding to a video frame to be played through the open graphics library, simultaneously drawing the video frame data on a second drawing surface control corresponding to a video encoder, encoding the drawn video frame data through the video encoder, and recording the video frame; coding the received audio frame data through the audio coder, and recording the audio frame;

and the merging unit is configured to merge the video frame and the audio frame in the at least part of data to obtain the recording information.

5. The apparatus of claim 4, wherein the merging unit is further configured to:

and combining the video frame and the audio frame in at least part of the data to obtain the recording information with a preset format.

6. The apparatus of claim 4, further comprising:

and the processing unit is configured to perform data processing on the at least part of data to obtain processed at least part of data.

7. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-3.

8. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-3.

Images