WO2012016406A1

WO2012016406A1 - Method and mobile terminal for adding background sound in recording procedure

Info

Publication number: WO2012016406A1
Application number: PCT/CN2010/079242
Authority: WO
Inventors: 刘洪霞; 李�昊
Original assignee: 中兴通讯股份有限公司
Priority date: 2010-08-03
Filing date: 2010-11-29
Publication date: 2012-02-09
Also published as: CN102348086A

Abstract

The present invention discloses a method and mobile terminal for adding a background sound in a recording procedure. Said mobile terminal includes: a storage module, a human-computer interaction module, an audio file parsing module, a video frame encoding module and a recording file production module. Said method includes the following steps: when a user uses a mobile terminal to start recording after choosing to add the background sound, said mobile terminal parses the background sound file selected by the user and encodes the video frames obtained in the recording procedure, then the parsed audio frames and the encoded video frames are written into the recording file alternatively, and the basic information needed for playing said audio frames and video frames are written into said recording file after the recording is completed. Herein, the user experience is enriched, and a stronger camera function of a mobile phone is enabled.

Description

Method for loading background sound in video recording process and mobile terminal

Technical field

The present invention relates to the field of mobile communications, and in particular, to a method for loading a background sound during a video recording process and a mobile terminal.

Background technique

With the development of technology, mobile phones are being given more and more additional functions. Whether or not these additional features, and whether these additional features provide a better user experience, has become one of the key factors for consumers to consider when purchasing a mobile phone. The mobile phone camera function is such a key mobile phone add-on function.

At present, the video file format commonly used by mobile phones is 3gp. As shown in Figure 1, a file in 3gp format consists of several boxes and can be nested, that is, a box can store multiple other boxes. Among them, the 4 bytes at the beginning of each box are an integer, which stores the length of the box; the next 4 bytes are a string, which stores the type of the box, such as "moov", "mvhd"; The remaining bytes store the content information of this box. Here are a few basic boxes:

1) File type package:

The box is 24 bytes long and has the type "ftyp". The remaining bytes are used to record the version number and compatible version number of the 3gp file. In a 3gp file, there is only one ftyp type box, and it appears in the file header. The related decoder can judge the file by reading whether the file header has the "ftyp" keyword and whether it meets the ftyp box standard. Whether it is a 3gp file.

2) Media information package:

The length of the box is variable, the type is "moov", and the remaining bytes are used to record the basic information of the 3Gp file video and audio, including: Which encoding method is used, the total duration, the starting position of the audio and video frames in the file, Information such as the position of the key frame, the time stamp of the audio and video frame playback, and so on.

In a 3gp file, there is only one media packet, which usually appears in the header of the 3gp file (that is, after the ftyp type box) or at the end of the 3gp file. It is the most important part of the 3gp file format. Read 3gp When the file is played, the media information package should be analyzed first, and the 3gp file should be created. (such as recording) Generally, the information collected before is written at the end of the file.

3) Media packets

The length of the box is variable, the type is "mdat", and the remaining bytes are generally used by users to store video frames and audio frame data, while audio frames and video frames are generally interleaved and stored in chronological order.

The number of media packets is variable and the location in the file is not fixed. It is used to store actual audio and video data, as well as other related data.

Compared with the traditional digital camera, the mobile phone camera effect is slightly rough, but with the powerful processing function of the mobile phone, if the camera function can be enriched and the user experience is better, the mobile phone camera function will be more powerful and more attractive to consumers. .

In many scenarios, the user has the willingness to re-edit the video and replace the background sound in the video. In this case, the user needs to first use a digital camera or a mobile phone to complete the camera operation, and then import the captured file into the PC, and then use the professional video editing software to process the recorded file. This method is very professional and time consuming, making the average user unwilling to involve. Summary of the invention

The technical problem to be solved by the present invention is to provide a method for loading background sounds in a video recording process and a mobile terminal, so as to solve the problem in the prior art that a user needs to synchronize a video file to a PC and edit with a professional video processing software to replace the video file. Defects in the background sound.

In order to solve the above problems, the present invention provides a method for loading a background sound during a recording process, including:

After the user selects to load the background sound and starts recording by using the mobile terminal, the mobile terminal parses the background sound file selected by the user and encodes the video frame acquired during the recording, and then parses the audio frame that is parsed. And the encoded video frames are alternately written into the video file, and the basic information required to play the audio frame and the video frame is written into the video file after the recording is completed.

The above method may also have the following features:

The basic information includes: an encoding mode, a total duration, a starting position of the audio and video and the video frame in the recording file, a position of the key frame, and a time stamp of the audio and video frame playing. The above method may further include:

The mobile terminal adds a first identification information to each audio frame that is parsed;

When parsing the background sound file selected by the user, the mobile terminal encodes the audio data acquired by the microphone into an audio frame, and adds a second identification information to the audio frame; and the mobile terminal encodes the audio file. The audio frame is alternately written into the video file along with the parsed audio frame and the encoded video frame.

The above method may also have the following features:

The background sound file selected by the user refers to: an audio file selected by the user as an background sound from an interface provided by the mobile terminal.

The invention also provides a mobile terminal for loading a background sound during a recording process, comprising: a storage module, a human-computer interaction module, an audio file parsing module, a video frame encoding module and a video file producing module;

The storage module is configured to: save an audio file;

The human-computer interaction module is configured to: provide a human-computer interaction interface to display an audio file saved in the storage module to a user, and send information of the selected audio file to the user when the user selects one of them as a background sound Audio file parsing module;

The audio file parsing module is configured to: obtain a corresponding audio file from the storage module according to the received information of the audio file, and then parse the audio file into an audio frame and send the audio file to the video file creation module;

The video frame coding module is configured to: encode the video frame obtained during the recording process, and send the encoded video frame to the video file creation module;

The video file creation module is configured to: write the received audio frame and video frame into the video file alternately, and write the basic information required to play the audio frame and the video frame to the office after the recording is completed. In the video file.

The above method may also have the following features:

The basic information includes: an encoding method, a total duration, audio and video, and a video frame in the video The starting position in the piece, the position of the key frame, and the timestamp of the audio and video frame playback.

The mobile terminal may further include: an audio data encoding module;

The audio file parsing module is further configured to: add a first identification information for each audio frame that is parsed;

The audio data encoding module is configured to: encode the audio data acquired by the microphone on the mobile terminal into an audio frame, and add a second identification information to the audio frame;

The recording file creation module is configured to: the encoded audio frame is alternately written into the video file together with the parsed audio frame and the encoded video frame.

The invention is simple and easy to operate, and the user is very convenient to use. At the same time, since the real background sound can be recorded without being recorded, the overhead of audio acquisition and encoding of the mobile phone is saved, which can make the camera more power-saving and the recording time longer. Loading background sound recording adds an additional camera mode, which makes the user's special needs satisfied. Especially for young users who are pursuing fashion and individuality, users can shoot their favorite scenes at the same time and choose their favorite music for synthesis. After the shooting is completed, the recorded multimedia file can be sent to your friends for sharing via MMS (Multimedia Messaging Service). This new feature enriches the user experience and makes the camera's camera function even more powerful. BRIEF abstract

1 is a schematic diagram of a format of a 3GP file in the prior art;

2 is a flowchart of a method for loading a background sound during a recording process according to an embodiment of the present invention; FIG. 3 is a flowchart of a method for simultaneously recording a real-time background sound and a background sound file selected by a user according to an embodiment of the present invention;

FIG. 4 is a structural diagram of a mobile terminal according to an embodiment of the present invention. Preferred embodiment of the invention

The basic idea of the method of the present invention is: after the user chooses to load the background sound and uses the mobile terminal When the terminal starts recording, the mobile terminal parses the background sound file selected by the user and encodes the video frame obtained during the recording process, and then alternately parses the parsed audio frame and the encoded video frame into the video file, After the recording is completed, the basic information required to play the above audio frame and video frame is written into the video file, wherein the basic information includes: the encoding mode, the total duration, the starting position of the audio and video frame in the file, and the position of the key frame. And timestamps such as audio and video frame playback.

In specific implementation, the mobile terminal needs to provide a corresponding menu selection interface for the user to select whether to load the background sound for the video file; after the user selects to load the background sound, the user also needs to provide a music selection interface for the user to pass through the interface. The mobile terminal confirms selection of an audio file as a background sound.

As shown in FIG. 2, after the user selects the background sound and selects to start recording, the mobile terminal performs the following process:

1) The camera on the mobile terminal transmits the obtained video original frame data (such as YUV data) to the screen on the mobile terminal for display, and the mobile terminal encodes the video original frame data according to a preset video coding manner;

2) The mobile terminal parses the background sound file selected by the user into an audio frame. It should be noted that steps 1) and 2) can be performed simultaneously, regardless of time;

3) The parsed audio frame and the encoded video frame are alternately written into the video file.

According to the file coding specification, video frames and audio frames are continuously written into the video file in chronological order. After the recording is completed, index information such as the starting position, size, and time stamp of each audio frame and video frame in the recording file is written into the corresponding field at the end of the file.

As can be seen from the above description, the audio data collected by the microphone on the mobile terminal is not processed by the above recording method. If the user intends to retain the voice data of the microphone, that is, the data collected by the microphone is retained in the video file, and the data of the background sound added by the user is also selected, as shown in FIG. 3, the following process is performed:

1) The camera on the mobile terminal transmits the acquired video original frame data (such as YUV data) to the screen on the mobile terminal for display, and the mobile terminal encodes the video original frame data according to a preset video coding manner; 2) The mobile terminal parses the background sound file selected by the user into an audio frame, and adds a first identification information for each audio frame, where the first identification information is used to indicate that the audio frame is a background sound file selected by the user. Audio frame

3) The mobile terminal encodes the audio data (such as PCM (Pulse Code Modulation) data) acquired by the microphone into an audio frame according to a preset audio coding mode, and adds a second to each of the above audio frames. Identification information, the second identification information is used to indicate that the audio frame is an audio frame corresponding to the voice data acquired by the microphone;

It should be noted that steps 1) to 3) can be performed simultaneously, regardless of time;

4) The audio frame and the video frame obtained through the above steps are alternately written into the video file.

According to the file coding specification, video frames and two audio frames are continuously written into the video file in chronological order. After the recording is completed, index information such as the starting position, size, and time stamp of each audio frame and video frame in the recording file is written into the corresponding field at the end of the file.

During playback, the decoder may correspondingly decode the audio frame corresponding to the background sound file and the real background sound collected during the recording according to the first identification information and the second identification information.

The mobile terminal that loads the background sound in the recording process according to the present invention, as shown in FIG. 4, includes: a storage module, a human-computer interaction module, an audio file parsing module, a video frame encoding module, and a video file creating module;

The storage module is used to save the audio file;

The human-computer interaction module is configured to provide a human-computer interaction interface to display the audio file saved in the storage module to the user, and send the information of the selected audio file to the audio file parsing module when the user selects one as the background sound;

The audio file parsing module is configured to obtain a corresponding audio file from the storage module according to the received information of the audio file, and then parse the audio file into an audio frame and send the audio file to the video file making module;

The video frame coding module is configured to encode the video frame obtained during the recording process, and send the encoded video frame to the video file creation module;

The video file creation module is used to alternately receive the received audio frame and video frame into the video file. Medium, and write the basic information required to play the audio frame and video frame to the video file after the recording is completed.

The foregoing basic information may include: an encoding mode, a total duration, a starting position of the audio and video and the video frame in the video file, a position of the key frame, and a time stamp of the audio and video frame playing.

In addition, the foregoing mobile terminal may further include: an audio data encoding module;

The audio file parsing module may be further configured to add a first identification information to each of the parsed audio frames;

The audio data encoding module may be configured to encode the audio data acquired by the microphone on the mobile terminal into an audio frame, and add a second identification information to the audio frame;

The recorded file creation module is configured to alternately write the encoded audio frame with the parsed audio frame and the encoded video frame.

While the preferred embodiments of the present invention have been disclosed for purposes of illustration, those skilled in the art will recognize that various modifications, additions and substitutions are possible, and the scope of the invention should not be limited to the embodiments described above.

One of ordinary skill in the art will appreciate that all or a portion of the above steps may be accomplished by a program instructing the associated hardware, such as a read-only memory, a magnetic disk, or an optical disk. Alternatively, all or part of the steps of the above embodiments may also be implemented using one or more integrated circuits. Correspondingly, each module/unit in the above embodiment may be implemented in the form of hardware or in the form of a software function module. The invention is not limited to any specific form of combination of hardware and software.

Industrial applicability

The invention is simple and easy to operate, and the user is very convenient to use. At the same time, since the real background sound can be recorded without being recorded, the overhead of audio acquisition and encoding of the mobile phone is saved, which can make the camera more power-saving and the recording time longer. Loading background sound recording adds an additional camera mode, which makes the user's special needs satisfied. Especially for young users who are pursuing fashion and individuality, users can shoot their favorite scenes at the same time and choose their favorite music for synthesis. , after shooting is complete Immediately send the recorded multimedia files to your friends for sharing via MMS SMS. This new feature enriches the user experience and makes the camera's camera function even more powerful.

Claims

Claim

1. A method of loading a background sound during a recording process, the method comprising:

2. The method of claim 1 wherein:

The basic information includes: an encoding mode, a total duration, a starting position of the audio and video and the video frame in the video file, a position of the key frame, and a time stamp of the audio and video frame playing.

3. The method of claim 1 or 2, the method further comprising:

4. The method of claim 1, wherein

5. A mobile terminal that loads a background sound during a recording process, the mobile terminal comprising: a storage module, a human-machine interaction module, an audio file parsing module, a video frame encoding module, and a video file creating module;

The storage module is configured to: save an audio file;

The audio file parsing module is configured to: according to the received information of the audio file Obtaining a corresponding audio file in the storage module, and then parsing the audio file into an audio frame and sending the audio file to the video file creation module;

6. The mobile terminal of claim 5, wherein

The mobile terminal according to claim 5 or 6, wherein the mobile terminal further comprises: an audio data encoding module;

The recording file creation module is configured to: write the encoded audio frame to the video file alternately with the parsed audio frame and the encoded video frame.